# Explain Importance Sampling in LLM Training Define importance sampling and explain how it supports estimation under one distribution using samples from another. Connect the likelihood ratio to off-policy or reused language-model data and discuss variance and bias controls. ### Constraints & Assumptions - The target distribution must be absolutely continuous with respect to the proposal on relevant events. - Sequence-level ratios can become extreme as token count grows. - State whether ratios are normalized, clipped, or truncated. ### Clarifying Questions to Ask - Which expectation is the estimator targeting? - Which policy generated the samples? - What correction or clipping changes the estimator's bias? ```hint Write the ratio Identify target probability divided by proposal probability for the same sampled event before describing an algorithm. ``` ### What a Strong Answer Covers - Derivation of the weighted expectation and support condition. - Unbiasedness under ideal ratios and the source of high variance. - Token- or sequence-level use in off-policy language-model objectives. - Clipping, normalization, effective sample size, and diagnostics. ### Follow-up Questions 1. Why do long generated sequences make ratios unstable? 2. When is collecting fresh on-policy data better than correcting old samples?