DTMC
The structure and the exact properties of a finite discrete-time Markov chain.
This class answers questions about a transition matrix. It holds no random number stream and generates nothing, because none of what it computes needs randomness: a stationary distribution or an expected absorption time is a property of the matrix, not of a sample path. To simulate a chain, use DMarkovChain, which holds one of these and exposes it as its dtmc property.
States are numbered 1 through numStates, matching DMarkovChain. Row i of the transition matrix is the distribution of the next state given the chain is in state i, so every row must be a valid probability mass function.
Results that require a structural precondition say so and check it. A stationary distribution is unique only for an irreducible chain; the absorbing-chain results require an absorbing chain in the usual sense, that every recurrent state is absorbing. Asking for one of these on a chain that does not qualify raises an exception naming the reason rather than returning a number that means nothing.
Parameters
a square matrix whose rows are probability mass functions
Properties
The states the chain never leaves, in increasing order.
The communicating classes, each a set of state numbers that all reach one another. Returned in order of their smallest member.
Whether the chain has at least one absorbing state.
True when every recurrent state is absorbing, which is what the absorbing-chain results below assume. A chain with a recurrent class of two or more states is not an absorbing chain, however many absorbing states it also has.
Whether every recurrent state has period 1.
Whether every state can reach every other state.
Entry (i, j) is true when state j+1 can be reached from state i+1 in zero or more transitions. Every state reaches itself, so the diagonal is true.
The recurrent states: those in a communicating class the chain cannot leave. In a finite chain these are exactly the states the chain returns to infinitely often.
The stationary distribution: the unique probability vector with pi = pi P. Entry k is the long-run proportion of transitions spent in state k+1.
The transient states: every state that is not recurrent.
The transition matrix. A copy: mutating it does not affect this chain.
Functions
Entry (i, j) is the probability that the chain is eventually absorbed in state j+1 having started in state i+1. Columns for states that are not absorbing are zero, and an absorbing state is absorbed in itself with probability one.
Whether the chain can leave state and come back to it in one or more transitions.
The expected number of transitions before absorption, one entry per state in state order. An absorbing state is already absorbed, so its entry is zero.
The fundamental matrix N = (I - Q) inverted, where Q holds the transition probabilities among the transient states. Entry (i, j) is the expected number of visits to the j-th transient state, starting from the i-th, before absorption. Rows and columns follow the order of transientStates.
Whether the chain, once in state, never leaves.
Whether to can be reached from from in zero or more transitions. A state is always reachable from itself; see canReturnTo for reachability in one or more.
Whether the chain returns to state infinitely often.
Whether the chain visits state only finitely often.
Entry (i, j) is the expected number of transitions to reach state j+1 for the first time, having started in state i+1. The diagonal holds the mean recurrence times, the expected number of transitions to return to a state, which is the reciprocal of that state's stationary probability.
The n-step transition matrix: entry (i, j) is the probability of being in state j+1 after n transitions, having started in state i+1. Zero steps gives the identity.