caiyuchen commited on
Commit
73c962b
·
verified ·
1 Parent(s): 434c168

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +8 -1
README.md CHANGED
@@ -17,11 +17,18 @@ base_model:
17
 
18
  # On Predictability of Reinforcement Learning Dynamics for Large Language Models
19
 
 
 
 
20
  This repository provides one of the models used in our paper **"On Predictability of Reinforcement Learning Dynamics for Large Language Models"** for evaluating and predicting reinforcement learning (RL) dynamics in large language models (LLMs).
21
 
22
  Recent advances in LLM reasoning capabilities are largely driven by RL, yet the parameter dynamics during RL training remain poorly understood. Our work identifies two key properties of RL-induced parameter updates: **Rank-1 Dominance**, where the top singular subspace of the parameter update matrix captures nearly all reasoning improvements, and **Rank-1 Linear Dynamics**, where this subspace evolves linearly across training, allowing accurate prediction from early checkpoints. Based on these insights, we propose **AlphaRL**, a plug-in acceleration framework that extrapolates final parameter updates from a short early training window, achieving up to 2.5× speedup while retaining over 96% of reasoning performance.
23
 
24
- This model and code are provided to support further research on interpretable and efficient RL training for LLMs. Our full codebase is available at: [AlphaRL GitHub](https://github.com/caiyuchen-ustc/Alpha-RL)
 
 
 
 
25
 
26
  ## 🔧 Prompt Format (Chat Template)
27
 
 
17
 
18
  # On Predictability of Reinforcement Learning Dynamics for Large Language Models
19
 
20
+
21
+ ![Paper Overview](/project/ugmathllm/caiyuchen/LIMO/eval/auto_temperature/overview.png)
22
+
23
  This repository provides one of the models used in our paper **"On Predictability of Reinforcement Learning Dynamics for Large Language Models"** for evaluating and predicting reinforcement learning (RL) dynamics in large language models (LLMs).
24
 
25
  Recent advances in LLM reasoning capabilities are largely driven by RL, yet the parameter dynamics during RL training remain poorly understood. Our work identifies two key properties of RL-induced parameter updates: **Rank-1 Dominance**, where the top singular subspace of the parameter update matrix captures nearly all reasoning improvements, and **Rank-1 Linear Dynamics**, where this subspace evolves linearly across training, allowing accurate prediction from early checkpoints. Based on these insights, we propose **AlphaRL**, a plug-in acceleration framework that extrapolates final parameter updates from a short early training window, achieving up to 2.5× speedup while retaining over 96% of reasoning performance.
26
 
27
+ This model and code are provided to support further research on interpretable and efficient RL training for LLMs. Our full codebase is available at: [AlphaRL GitHub](https://github.com/caiyuchen-ustc/Alpha-RL).
28
+
29
+
30
+
31
+
32
 
33
  ## 🔧 Prompt Format (Chat Template)
34