Hugging Face's logo
‌ ​ ‍ ​ ‎ ‏ ‌ ​ ‍ ​'s picture
19 20 132

‌ ​ ‍ ​ ‎ ‏ ‌ ​ ‍ ​

Maani
lnxdx's profile picture tahamajs's profile picture Gargaz's profile picture
·

AI & ML interests

Novel Architectures | System Design

Recent Activity

reacted to codelion's post with 🚀 10 minutes ago
Recently, Essential AI released a new 8B base model https://huggingface.co/EssentialAI/rnj-1 they highlighted the importance of data mix for pretraning - "In the long run, we expect our methods to automatically represent, transform, and blend data to optimize measurable abilities in pre-training. Our work on modeling data taxonomies led to new approaches for jointly clustering and mixing data distributions under data repetition penalties. Many improvements in our STEM abilities can be traced back to this. " This resonates with the recent work we did around optimal dataset mixing for pretraining where we saw have the right mix can increase the efficiency of training - https://huggingface.co/blog/codelion/optimal-dataset-mixing
liked a model 12 minutes ago
mistralai/Devstral-Small-2-24B-Instruct-2512
liked a model 2 days ago
mistralai/Ministral-3-8B-Instruct-2512
View all activity

Organizations

None yet