KSLCore/ksl.utilities.distributions.fitting.mixture.scoring/HeldOutLogLikelihoodCriterion

HeldOutLogLikelihoodCriterion

class HeldOutLogLikelihoodCriterion(holdOutData: DoubleArray, worstValue: Double = LogLikelihoodCriterion.defaultWorstValue) : MixtureCriterionIfc(source)

The mean log-likelihood of a fitted mixture on observations it was not fitted to.

This is the only measure immune to the optimism introduced by selecting the partition, the families, and the number of components from the same data, which makes it the arbiter when the penalized criteria disagree.

Larger values indicate a better fit, unlike every other criterion here. Ranking machinery must consult the direction rather than assume it.

Parameters

holdOutData

the observations to evaluate on, disjoint from those used to fit

Constructors

HeldOutLogLikelihoodCriterion

constructor(holdOutData: DoubleArray, worstValue: Double = LogLikelihoodCriterion.defaultWorstValue)

Properties

name

open override val name: String

A short name for the criterion, used as a column heading and a factor level.

smallerIsBetter

open override val smallerIsBetter: Boolean = false

Whether smaller values indicate a better fit. Information criteria are smaller-is-better; a held-out mean log-likelihood is not. Ranking machinery must consult this rather than assuming a direction.

Functions

evaluate

open override fun evaluate(candidate: MixtureCandidate, data: DoubleArray): MixtureCriterionValue

Evaluates the criterion for the supplied candidate against the supplied observations.

metric

open override fun metric(): MetricIfc

A metric describing this criterion, for use with the library's multi-objective decision analysis machinery. The direction is taken from this interface.

toString

open override fun toString(): String