# Explain Pipeline Parallelism in GRPO Training Explain how pipeline parallelism can be used around a group-relative policy optimization training workflow. Cover the distinction between rollout generation and policy training, stage partitioning, microbatches, pipeline bubbles, model versions, and failure or throughput trade-offs. ### Constraints & Assumptions - Separate GRPO's learning objective from the systems mechanism of pipeline parallelism. - Do not assume rollout workers and training stages share one synchronous loop. - State which model version generated each sampled response. ### Clarifying Questions to Ask - Is parallelism being applied during inference rollouts, training, or both? - Which latency or throughput bottleneck motivates it? - How much policy staleness is acceptable between generation and update? ```hint Track versions and bubbles Draw the movement of one rollout group and one training microbatch, including idle time and the policy version attached to each. ``` ### What a Strong Answer Covers - A correct high-level account of GRPO's grouped sampling and relative advantages. - Pipeline stage and microbatch scheduling with bubble and communication costs. - Rollout-versus-training resource allocation and model-version consistency. - Checkpointing, backpressure, failure recovery, and useful throughput metrics. ### Follow-up Questions 1. How would uneven Transformer layers affect stage placement? 2. When would tensor or data parallelism be a better first choice?