|
|
--- |
|
|
base_model: |
|
|
- mistralai/Mistral-7B-Instruct-v0.2 |
|
|
datasets: |
|
|
- mzhaoshuai/Llama-3.3-70B-Inst-awq_ultrafeedback_1in3 |
|
|
license: apache-2.0 |
|
|
library_name: transformers |
|
|
pipeline_tag: text-generation |
|
|
--- |
|
|
|
|
|
# RefAlign: RL with Similarity-based Rewards |
|
|
|
|
|
**GitHub repository**: https://github.com/mzhaoshuai/RefAlign |
|
|
|
|
|
**Paper**: [Learning from Reference Answers: Versatile Language Model Alignment without Binary Human Preference Data](https://huggingface.co/papers/2504.09895). |
|
|
|
|
|
The training data is [mzhaoshuai/Llama-3.3-70B-Inst-awq_ultrafeedback_1in3](https://huggingface.co/datasets/mzhaoshuai/Llama-3.3-70B-Inst-awq_ultrafeedback_1in3). |
|
|
|
|
|
When conducting Reinforcement Learning with Similarity-based Rewards, the reward function is BERTScore. |
|
|
|
|
|
| Hyper-Parameters | Value | |
|
|
|:---------------------------------------------------------|--------------------------------------------------------| |
|
|
|LR|8e-7| |
|
|
|Batch Size| 512 | |
|
|
|Epoch| 1 | |
|
|
|Prompt Length| 400 | |
|
|
|Generation Length|800| |
|
|
|Advantage CLIP|0.5| |
|
|
|Sampled Generations (K)|2| |
|
|
|BertScore Model|bart-large-mnli| |
