KSLCore/ksl.simopt.benchmark

Package-level declarations

Types

class BenchmarkExperiment(val name: String, val problems: List<ProblemCase>, val solverCases: List<SolverCase>, val macroReplications: Int, val replicationBudgetPerRun: Int, val confirmation: ConfirmationOptions? = ConfirmationOptions(), val captureIterationTraces: Boolean = false, val verificationReplications: Int? = null, val numWorkers: Int? = null, experimentStreamProvider: RNStreamProviderIfc = RNStreamProvider(), cellSolverDecorator: (solver: Solver, problemName: String, solverLabel: String, repNum: Int) -> Unit? = null)

The benchmark grid: problems × solver configurations × macro-replications, run under the experiment-level fairness policies and concurrent execution, producing a BenchmarkSummary of cell-level run records, per-problem confirmation outcomes, and optimality gaps.

BenchmarkRunResult

data class BenchmarkRunResult(val experimentName: String, val problemName: String, val solverLabel: String, val repNum: Int, val cellLabel: String, val status: MemberStatus, val startingPoint: Map<String, Double>, val bestInputs: Map<String, Double>, val bestObjective: Double, val bestPenalizedObjective: Double, val isBestValid: Boolean, val isInputFeasible: Boolean, val responseConstraintViolation: Double, val numOracleCalls: Int, val numReplicationsRequested: Int, val totalIterations: Int?, val wallClockMillis: Long?, val gap: Double?, val gapType: GapType?, val errorMessage: String? = null)

The recorded outcome of one benchmark cell: one solver configuration run once (one macro-replication) on one problem under the experiment's replication budget.

BenchmarkSolverFactoryIfc

fun interface BenchmarkSolverFactoryIfc

Creates a fresh solver instance for one benchmark cell, bound to the cell's problem definition and private evaluator.

BenchmarkSummary

data class BenchmarkSummary(val experimentName: String, val macroReplications: Int, val replicationBudgetPerRun: Int, val confirmation: ConfirmationOptions?, val verificationReplications: Int?, val startTime: Instant, val endTime: Instant, val problemResults: List<ProblemBenchmarkResult>, val solverCaseDescriptions: Map<String, String>, val solverConfigurations: Map<String, Map<String, String>>, val traces: Map<String, List<IterationTracePoint>>)

The in-memory result of a benchmark experiment: one entry per problem, in problem order, each holding its cell-level run records, confirmation outcome, and gap basis.

FunctionMemberEvaluatorFactory

class FunctionMemberEvaluatorFactory(problemDefinition: ProblemDefinition, responseFunctionBuilder: ResponseFunctionBuilderIfc, microRepSampleSize: Int = 1, substreamBlockSize: Int = ConcurrentRunOptions.DEFAULT_SUBSTREAM_BLOCK_SIZE, solutionCacheFactory: () -> SolutionCacheIfc? = { MemorySolutionCache() }) : MemberEvaluatorFactoryIfc

Provisions per-member evaluators over a response function — the synthetic and static Monte Carlo counterpart of ksl.simopt.solvers.concurrent.PooledMemberEvaluatorFactory. Each member gets its own ResponseFunctionOracle with a private solution cache and a stream tape starting at the member's block offset, so concurrently running members draw from non-overlapping regions of the sub-stream tape regardless of scheduling.

GapType

enum GapType : Enum<GapType>

The basis against which a run's optimality gap was computed. Runs on problems with a reference solution record the reference's type; runs on problems without one are gapped against the best objective value found across all runs of that problem within the experiment.

IterationTracePoint

data class IterationTracePoint(val iteration: Int, val cumulativeReplications: Int, val bestPenalizedObjective: Double)

One point of a cell's iteration trace: the solver's progress at the end of an iteration, recorded when the experiment captures traces. Iteration 0 is the initialized state before the first main iteration.

ProblemBenchmarkResult

data class ProblemBenchmarkResult(val problemName: String, val tags: Map<String, String>, val dimension: Int, val optimizationType: OptimizationType, val numResponseConstraints: Int, val runs: List<BenchmarkRunResult>, val confirmation: ConfirmationOutcome?, val winner: Solution?, val verification: Solution?, val gapBasisObjective: Double?, val gapType: GapType?)

The recorded outcome of all cells of one problem within a benchmark experiment, plus the problem-level confirmation stage and the gap basis shared by the runs.

ProblemCase

class ProblemCase(val name: String, val problemDefinitionFactory: () -> ProblemDefinition, val evaluatorFactoryProvider: (problemDefinition: ProblemDefinition) -> MemberEvaluatorFactoryIfc, val referenceSolution: ReferenceSolution? = null, val tags: Map<String, String> = emptyMap())

A named, self-contained problem for benchmarking: fresh problem definitions on demand, fresh per-member evaluation resources on demand, an optional reference solution, and descriptive tags for analysis grouping.

ReferenceSolution

data class ReferenceSolution(val inputs: Map<String, Double>, val objectiveValue: Double, val type: ReferenceType)

A reference solution for a benchmark problem: the inputs and objective value that runs are measured against, and whether that reference is a proven optimum or merely the best known point.

ReferenceType

enum ReferenceType : Enum<ReferenceType>

Whether a problem's reference solution is a proven optimum or just the best point known so far.

SolverCase

class SolverCase(val label: String, val solverFactory: BenchmarkSolverFactoryIfc, val description: String = "")

A named solver configuration for benchmarking. The factory creates a fresh, member-safe solver instance per cell (a new instance on every call, bound only to the supplied evaluator, with its own solver-side streams), for whatever problem the experiment hands it — the same configuration races on every problem in the grid. The harness reads each instance's configurationProperties so the configuration that actually ran is recorded rather than assumed.