Hill diversity
What it measures
Hill numbers come from ecology. There the kinds are species and the population is the individuals in a sample, and they read diversity — how many kinds are present and how evenly — without forcing richness and evenness into a single index.
These are notes on what we want to study, not measurements we have taken. The intent is to borrow Hill numbers to read semantic diversity over the substrate: treat the kinds as frame types and the population as a set of active frames, and ask how varied that set is. A Hill number is an effective count — the number of equally-common types that would produce the diversity you actually see. Ten frame types may be present, but if one dominates, the effective count drops well below ten, and the rare ones barely register.
We expect to use this as a general reading of substrate state — whether a region keeps a varied vocabulary of framings or collapses onto a few — and to read it across scales, since healthy diversity for the whole store, a membrane, and a single agent’s active context need not be the same number.
One thing it should expose is a diversity signature that differs by task: a narrow task should range over few framings, an open one over many. Whether that can be turned into a judgment of task-appropriate behavior is a separate and still-open question.
How it's calculated
Hill numbers are a family, not one number. A single formula, read at different orders , recovers several of the diversity indices used elsewhere as special cases. The order is a dial for how much weight rare types get.
| Order | Equivalent index | What it weights |
|---|---|---|
| richness (type count) | every type equally, rare or common | |
| Shannon, | each type by its frequency | |
| inverse Simpson | the common types; rare ones nearly ignored | |
| Berger–Parker | only the single most dominant type |
We mostly read the first three. Raising shifts weight from rare types toward common ones, so the effective count can only fall or hold as grows — never rise.
For type frequencies (each the share of frames of that type):
The case is taken as the limit, .
A small example. Five frame types with frequencies . Richness is . The Shannon number comes to about 3.2, and the inverse-Simpson to about 2.4. So although five types are present, the substrate behaves like it holds between two and three — one type carries most of the weight.
Reading the three orders together is the point. When sits well above , there is a long tail of rare types that the common ones are drowning out. Watching that gap is how we’d tell whether rare framings survive or get buried.