Benchmark
Parameters
the experiment's name; flows into every run record
the problem cases; names must be unique
the solver configurations; labels must be unique
the number of macro-replications per (problem, solver) pair in the study this experiment belongs to; it fixes the addressing of starting points, so blocks of one study must all declare the same value
which of those macro-replications THIS experiment runs. Defaults to all of them. Running a study as several experiments over disjoint sub-ranges is cell-for-cell identical to running it as one, which is how a long study is checkpointed: each block is saved on completion and a resumed run skips the blocks already present
the per-cell replication budget
confirmation-stage options; null disables confirmation
when true, every cell solver's per-iteration progress (iteration, cumulative replications, best penalized objective) is captured into the summary's traces, keyed by cell label — opt-in because traces grow with the budget
when true, each captured trace point also carries the cell solver's algorithm-specific state for that iteration. Gated separately from captureIterationTraces because the volume is an order of magnitude larger: a solver publishing six state values turns a study's trace rows into millions of state rows. Requires captureIterationTraces, since solver state rides on trace points.
when non-null, each problem's winning point is re-simulated at this replication count on a dedicated evaluator and recorded — the classic verify-at-elevated-replications step
the maximum number of cells running at the same time; null uses the smaller of the cell count and the available processors
the stream provider for experiment-level draws (currently: the common starting points); defaults to a fresh provider so identically configured experiments reproduce each other exactly
when supplied, receives each problem's result as it completes rather than only at the end, so a long study is checkpointed per problem instead of holding everything in heap and writing once. A sink also enables resume: a re-run attaches to an unfinished record of the same name and skips the problems already in it. run() still returns a BenchmarkSummary, so existing callers are unaffected — but note that on a resumed run the summary covers only the problems THIS pass ran; the skipped ones are in the sink, not in the returned value.
invoked with each freshly created cell solver before it runs, on the cell's worker thread — the attachment hook for per-cell trackers and instrumentation; anything it touches must be safe to use from worker threads