Markov chains

Question 1

What is a Markov chain?

Answer 1

A Markov Chain is a sequence of events or states where the probability of moving to the next state depends only on the current state, not on any previous states. This property is known as the “memoryless” or Markov property.

Question 2

What are transition probabilities?

Answer 2

Transition probabilities quantify the likelihood of moving from one state to another. They are organized into a transition matrix.

Question 3

What is a transition matrix?

Answer 3

• a square array where each entry represents the probability of transitioning from a specific row state to a specific column state
• for a system with n states, the matrix is an n×n grid
• each row sums to 1