Benchmark
The results database for benchmark experiments: SQLite by default (via the SQLiteDb base), holding the tables described by the table-data classes of this package.
Append semantics: by default the database is NOT deleted if it exists, and only missing tables are created — so successive experiments (including a later trace-enabled rerun of a problem) accumulate in one file under fresh experiment and run ids. Set deleteIfExists true to start clean.
Capture is post-run and bulk: run the ksl.simopt.benchmark.BenchmarkExperiment, then hand its summary to saveSummary. The returned experiment id keys every table.
Post-processing helpers pull typed rows back out (one function per table), feed a MultipleComparisonAnalyzer with per-problem final objectives via mcbDataMap, and compute performance-profile data from captured traces via performanceProfile.
Parameters
the database file name
the directory holding the database file
true deletes an existing file first; the default (false) appends to it
Constructors
Functions
Confirmation rows, optionally restricted to one experiment.
All experiment rows.
A MultipleComparisonAnalyzer over one problem's final objectives (see mcbDataMap); null when fewer than two solver cases have complete data.
The multiple-comparison feed for one problem of one experiment: solver case label mapped to the final objective values (or gaps) across the completed, valid macro-replications, ordered by replication number.
Performance-profile data across an experiment's traced runs: for each solver case and each budget fraction on a grid, the fraction of (problem, macro-rep) cells whose best penalized objective reached the problem's gap basis plus tau within that fraction of the replication budget.
Problem rows, optionally restricted to one experiment.
Run rows, optionally restricted to one experiment.
Persists a benchmark summary: the experiment row, its problems, solver cases and their captured parameters, every cell run, confirmation finalists, verification estimates, and (when captured) iteration traces keyed by run id.
Solver case parameter rows, optionally restricted to one experiment.
Solver case rows, optionally restricted to one experiment.
Iteration-trace rows, optionally restricted to one experiment's runs.
Verification rows, optionally restricted to one experiment.