ConfirmationReplay

Re-runs the confirmation stage of a finished benchmark against stored results, under a different selection rule, without re-running the search.

Why this exists. Evaluating a change to the selection rule against a completed study used to mean running the study again — days of compute to answer a question about the last few minutes of it. The search is the expensive part and it does not depend on the rule: the same cells, the same bests. Only the choice made among those bests changes, and the estimates that choice needs are in the database.

What makes it possible is tblRunResponse. tblRun preserves each cell best's inputs, which is enough to re-simulate a point but not to re-rank it — ranking needs each response's average, variance and count. With those stored, candidates sufficient for selection are rebuilt from the database and only the finalists the new rule chooses need fresh simulation.

The source experiment is never mutated. A replay is written as its own experiment under a new name, through the same sink an ordinary experiment uses, so it is queryable by every existing helper and can be compared with its source by the same queries.

What a replay is not. The rebuilt candidates carry the estimates the search produced, not the penalty state it carried — penalty memory is not stored, so a replayed selection under a rule that reads penalties would not reproduce the original. That is not a limitation in practice: FeasibilityFirstComparator, and any rule fit for cross-iteration selection, is clock- and penalty-independent by design. A rule that reads the penalized objective is the thing this library deliberately does not use for selection.