caiyuchen commited on
Commit
a16895a
·
verified ·
1 Parent(s): 73c962b

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +2 -2
README.md CHANGED
@@ -18,13 +18,13 @@ base_model:
18
  # On Predictability of Reinforcement Learning Dynamics for Large Language Models
19
 
20
 
21
- ![Paper Overview](/project/ugmathllm/caiyuchen/LIMO/eval/auto_temperature/overview.png)
22
 
23
  This repository provides one of the models used in our paper **"On Predictability of Reinforcement Learning Dynamics for Large Language Models"** for evaluating and predicting reinforcement learning (RL) dynamics in large language models (LLMs).
24
 
25
  Recent advances in LLM reasoning capabilities are largely driven by RL, yet the parameter dynamics during RL training remain poorly understood. Our work identifies two key properties of RL-induced parameter updates: **Rank-1 Dominance**, where the top singular subspace of the parameter update matrix captures nearly all reasoning improvements, and **Rank-1 Linear Dynamics**, where this subspace evolves linearly across training, allowing accurate prediction from early checkpoints. Based on these insights, we propose **AlphaRL**, a plug-in acceleration framework that extrapolates final parameter updates from a short early training window, achieving up to 2.5× speedup while retaining over 96% of reasoning performance.
26
 
27
- This model and code are provided to support further research on interpretable and efficient RL training for LLMs. Our full codebase is available at: [AlphaRL GitHub](https://github.com/caiyuchen-ustc/Alpha-RL).
28
 
29
 
30
 
 
18
  # On Predictability of Reinforcement Learning Dynamics for Large Language Models
19
 
20
 
21
+ ![Paper Overview](overview.png)
22
 
23
  This repository provides one of the models used in our paper **"On Predictability of Reinforcement Learning Dynamics for Large Language Models"** for evaluating and predicting reinforcement learning (RL) dynamics in large language models (LLMs).
24
 
25
  Recent advances in LLM reasoning capabilities are largely driven by RL, yet the parameter dynamics during RL training remain poorly understood. Our work identifies two key properties of RL-induced parameter updates: **Rank-1 Dominance**, where the top singular subspace of the parameter update matrix captures nearly all reasoning improvements, and **Rank-1 Linear Dynamics**, where this subspace evolves linearly across training, allowing accurate prediction from early checkpoints. Based on these insights, we propose **AlphaRL**, a plug-in acceleration framework that extrapolates final parameter updates from a short early training window, achieving up to 2.5× speedup while retaining over 96% of reasoning performance.
26
 
27
+ This model is one of the training checkpoints used in our paper and is provided to support research on evaluating and predicting parameter dynamics during RL training of LLMs. The full codebase is available at: [AlphaRL GitHub](https://github.com/caiyuchen-ustc/Alpha-RL).
28
 
29
 
30