ComparisonWithStandardProcedure

Kim's (2005) fully-sequential comparison with a standard procedure, used by COMPASS as the local-optimality test: the current center x* is the standard, its neighbors are the alternatives, and the procedure decides — with a controlled error probability — whether any neighbor is better than x* by more than the indifference amount delta.

Each system starts with n0 replications. For each alternative i the per-replication difference d_j = alt_i_j - standard_j (minimization, so a negative mean means the alternative is better) is accumulated into Z_i(r) = Σ d_j. A triangular continuation region with half-width a_i(r) = max(0, h² S²_i / (2δ) − (δ/2) r) brackets the walk, where S²_i is the first-stage sample variance of the differences and h² = 2 η (n0 − 1). At each stage:

• Z_i(r) < −a_i(r) ⇒ alternative i is significantly better than the standard ⇒ the standard loses and i is the winner;

• Z_i(r) > +a_i(r) ⇒ alternative i is significantly worse ⇒ eliminate i;

• otherwise keep sampling i.

When every alternative has been eliminated the standard is declared best. Bonferroni splits the error across the k alternatives: β = α / k and η = ½ ((2β)^(−2/(n0−1)) − 1) (the closed form for the single-constant case, c = 1).

This procedure requires a positive delta; with delta == 0 the boundary degenerates and the walk never terminates, which is why COMPASS uses it only when δ_L > 0 (see the ISC degraded-mode documentation).

Feasibility is settled before the test, not inside it. Systems are screened with ksl.simopt.evaluator.Solution.isResponseConstraintFeasible at feasibilityCILevel, following the same ordering FeasibilityFirstComparator applies: a confidently-feasible neighbour beats an infeasible standard outright, an infeasible neighbour cannot displace a feasible standard, and when nothing is confidently feasible the comparison falls back to least total violation with no statistical claim attached. Only among confidently-feasible systems does the sequential test run, and there it compares raw objectives against a variance built from the objective's own sampling error. An earlier version compared PENALIZED objectives against that same variance, which placed a term in Z whose variance is absent from S² -- narrowing the continuation region and inflating the error rate rather than merely blurring the decision -- and did so at whatever clocks the two systems had drifted to as alternatives were eliminated at different rates.

Parameters