Simulate and Compare Multi-Armed Bandit Strategies

Build a small simulation that compares common multi-armed bandit strategies. Explain each strategy, record reward and regret, and use the experiment to discuss exploration versus exploitation. ### Constraints & Assumptions The source reports a bandit simulation exercise with AI and search tools allowed, but gives no arm distributions, horizon, or required output. For this practice discussion, use stationary independent Bernoulli arms with known probabilities available only to the simulator. Compare epsilon-greedy, UCB1, and Thompson sampling over repeated random seeds. This is an experimental simulation exercise rather than a fixed-output console task. ### Clarifying Questions Are rewards binary or continuous? Is the environment stationary? What is the horizon? Does the policy observe only the chosen arm's reward? How should reproducibility and uncertainty be reported? ### What a Strong Answer Covers Separate policy knowledge from simulator knowledge, implement each update correctly, distinguish realized reward from expected pseudo-regret, and compare repeated runs rather than a single lucky trajectory. Explain any generated code before relying on it. ### Follow-up Questions How do the results change with a small reward gap, a short horizon, nonstationary rewards, or a poorly chosen exploration rate? Why is sharing a seed alone insufficient to guarantee identical random consumption across policies?