The n-step transition matrix: entry (i, j) is the probability of being in state j+1 after n transitions, having started in state i+1. Zero steps gives the identity.