204. Markov Chains

204.1. Checkout Counter Model

• Discrete time ($𝑛=0,1,\dots$)

• Custommer arrivals: Benoulli($𝑝$)

  • Geometric arrival times
• Custommer service times: Geometric($𝑞$)
• State ${𝑋}_{𝑛}$: Number of cusommers at time $𝑛$

Where:

• $𝑝$: Probability of having an arrival
• $𝑞$: Probability of having an departure
• $𝑝𝑞$: Probability of having an arrival and a departure simultaneously
• $1-𝑝$: Probability of not having a custommer arrival
• $1-𝑞$: Probability of not having a custommer departure
• $𝑝\left(1-𝑞\right)$: Probability of custommer arrival without a custommer departure
• $𝑞\left(1-𝑝\right)$: Probability of custommer departure without a custommer arrival

204.2. $𝒏$-Step Transition Probabilities

State occupance probabilities given initial state $𝑖$:

204.3. Key Recursion

The total probability of ending at with state $𝑗$ is the sum of probabilities of the different paths to get to state $𝑗$

204.4. Random Initial State