How SAPO mitigates extreme fluctuations in importance ratio

The Qwen team proposed Soft Adaptive Policy Optimization to improve the stability of large model RL

The paper on the Soft Adaptive Policy Optimization (SAPO) algorithm was published on arXiv, and then the Qwen team introduced this reinforcement learn...