Compute Markov steady state and expectations
Overview
This question evaluates understanding of Markov chains and stationary distributions, derivation of expectations for continuous and discrete distributions from first principles, and computation of mixed-strategy Nash equilibria in 2×2 zero-sum games, testing competencies in linear algebra, calculus, probability theory, and game-theoretic reasoning within the Statistics & Math domain. It is commonly asked because it probes both conceptual understanding (uniqueness conditions for stationary measures and theoretical expectation derivations) and practical application (symbolic manipulation and numeric illustration of equilibrium probabilities), making it a hybrid conceptual-and-applied problem that assesses mathematical rigor and probabilistic reasoning.
Probability and Game Theory: Three Sub-questions
Context: The exact transition matrix P (for Q1) and the 2×2 payoff matrix (for Q3) are not provided. Below, you will (a) solve generically in symbolic form, and (b) see a small numeric example to illustrate the procedure.
- Finite-state Markov chain
- Given a finite-state, irreducible, aperiodic Markov chain with transition matrix P, compute its stationary distribution π by solving πP = π with ∑ᵢ πᵢ = 1. Explain why the solution is unique.
- Expectations from first principles
- Let X ~ Exponential(λ) and N ~ Poisson(λ) with λ > 0. Derive E[X] and E[N] from first principles (integration/summation), showing intermediate steps.
- 2×2 zero-sum game
- For a 2×2 zero-sum game with Player A’s payoff matrix [ [a, b], [c, d] ], find the mixed-strategy Nash equilibrium. Report the probability each player assigns to their first action (row 1 for A, column 1 for B), and justify the equilibrium with probability calculations.
Solution
Login required