LookaheadPolicy

abstract class LookaheadPolicy(val search: ActionSearch = ExhaustiveSearch) : ShapeAwarePolicyIfc(source)

The skeleton shared by every policy that chooses among available actions rather than constructing one directly: value-function approximations, and cost-function approximations that score candidates.

A modeler supplies the three model-specific pieces and nothing else:

class MyRule : LookaheadPolicy(ExhaustiveSearch) {
override fun contribution(obs, action, ctx) = ... // C(s, a), the immediate cost
override fun postDecision(obs, action) = ... // S^x(s, a)
override fun value(postDecision, ctx) = ... // V̄
}

The enumeration, the feasibility filter, the argmin and the empty-set fallback come from the library. Contrast §8.2.9's first VFA, which inlined all four inside one action method — the reason §8.2.11 argues the design was one abstraction where the problem has five.

Not every policy fits this shape, and that is deliberate. A rule that constructs an action — the greedy allocator of §8.2.8 sorts regions and fills them — implements plain PolicyIfc instead. Two shapes, because there are two ways to decide.

Constructors

Link copied to clipboard
constructor(search: ActionSearch = ExhaustiveSearch)

Properties

Link copied to clipboard
protected val search: ActionSearch

Functions

Link copied to clipboard
override fun action(observation: DoubleArray, ctx: DecisionContext): DoubleArray
Link copied to clipboard
open override fun configure(surface: DecisionSurfaceDescriptor)
Link copied to clipboard
protected abstract fun contribution(observation: DoubleArray, action: DoubleArray, ctx: DecisionContext): Double

C(s, a): the cost incurred by taking action now.

Link copied to clipboard
protected open fun postDecision(observation: DoubleArray, action: DoubleArray): DoubleArray

S^x(s, a): the state immediately after the decision and before the exogenous information. Defaults to the observation unchanged, which is right for a rule whose action does not move the observed state.

Link copied to clipboard
protected abstract fun value(postDecision: DoubleArray, ctx: DecisionContext): Double

: estimated cost-to-go from the post-decision state.

Link copied to clipboard

What to do when 𝒳(s) is empty (§4.4.6.3). Zeros by default; §8.2.3's declared neutral value is what should replace this once it exists.