GainVersusPenaltyPlot

What the criterion gained by adding each component, against what it charged for it.

The criterion turns where the two cross: to the left of the crossing an extra component buys more likelihood than it costs, and to the right it does not. The vertical distance between the curves is how firmly the criterion holds its view at that count.

That distance is not a measure of how likely the criterion is to be right, and this class says so in its own caption rather than leaving it to the caller. Measured across the designed experiment, when these criteria pick the wrong component count the median gap to the truth is about 19 units, and only about 4.6% of wrong calls fall inside a conventional weak-evidence margin of two. A wide gap here means the criterion is not indifferent; it does not mean the criterion is correct. The half of the reading that measurement supports is the other one — a narrow gap really does mean the criterion barely preferred one count to its neighbour.

The analyst's own count is marked when they supplied one, because the firmness there is the number they came for.

A falling line is not a fault. The partition is regenerated at each count rather than split from the one before, so the models do not nest and the gain can be negative: the search found a worse partition at the larger count. That is worth seeing, so it is plotted rather than clipped.

Parameters