metadata
license: apache-2.0
DistillQwen-ThoughtY: Enhanced Chain-of-Thought Reasoning Models
Key Contributions
- Advanced Reasoning Models: DistillQwen-ThoughtY series (4B/8B/32B) outperform previous versions (ThoughtX) and Qwen3 in thinking mode, achieving state-of-the-art results on mathematical, scientific, and coding tasks.
- OmniThought-0528 Dataset: New 365K high-quality Chain-of-Thought (CoT) dataset distilled from DeepSeek-R1-0528 (top-tier Chinese model) with:
- Cognitive Difficulty (CD) and Reasoning Verbosity (RV) annotations
- Multi-teacher integration (DeepSeek-R1, DeepSeek-R1-0528, QwQ-32B)
Performance Highlights
| Model | AIME2024 | MATH500 | GPQA Diamond | LiveCodeBench V2 | Avg. |
|---|---|---|---|---|---|
| DistillQwen-ThoughtY-4B | 76.7 | 95.2 | 56.1 | 75.8 | 76.0 |
| DistillQwen-ThoughtY-8B | 76.7 | 94.6 | 62.1 | 78.1 | 77.9 |
| DistillQwen-ThoughtY-32B | 90.0 | 95.2 | 63.6 | 76.3 | 81.3 |
| OpenThinker2-32B | 76.7 | 90.8 | 64.1 | 72.5 | 76.0 |
| DistillQwen-ThoughtX-32B | 80.0 | 92.6 | 64.0 | 73.4 | 77.5 |
Quick Start
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained(
"alibaba-pai/DistillQwen-ThoughtY-4B",
device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained(model_name)
prompt = "Solve ∫x e^x dx. Show your reasoning step-by-step."
messages = [
{"role": "user", "content": prompt}
]
text = tokenizer.apply_chat_template(
messages,
tokenize=False,
add_generation_prompt=True,
enable_thinking=True # Switches between thinking and non-thinking modes. Default is True.
)
inputs = tokenizer([text], return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=32768)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
Resources
Reference
For more detailed information about the model, we encourage you to refer to our paper:
- Reasoning with OmniThought: A Large CoT Dataset with Verbosity and Cognitive Difficulty Annotations
Wenrui Cai, Chengyu Wang, Junbing Yan, Jun Huang, Xiangzhong Fang arXiv:2505.10937
You can cite the paper using the following citation format:
@misc{cai2025reasoningomnithoughtlargecot,
title={Reasoning with OmniThought: A Large CoT Dataset with Verbosity and Cognitive Difficulty Annotations},
author={Wenrui Cai and Chengyu Wang and Junbing Yan and Jun Huang and Xiangzhong Fang},
year={2025},
eprint={2505.10937},
archivePrefix={arXiv},
primaryClass={cs.CL},
url={https://arxiv.org/abs/2505.10937}
}