Update README.md
Browse files
README.md
CHANGED
|
@@ -74,17 +74,29 @@ print(pipe("What is prime intellect ?"))
|
|
| 74 |
- **Optimizer**: Diloco/LocalSGD - Inner Optimizer: AdamW, Outer Optmizer: Nesterov SGD
|
| 75 |
|
| 76 |
**Performance on benchmarks**
|
|
|
|
|
|
|
| 77 |
| Model | Size | Tokens | MMLU | GPQA | GSM8K | ARC-C | Hellaswag |
|
| 78 |
|---|---|---|---|---|---|---|---|
|
| 79 |
-
| INTELLECT
|
| 80 |
-
| LLaMA-7B | 7B | 1T | 35.1 | 23.1 | 9.7 | 50.43 | 78.19 |
|
| 81 |
-
| LLaMA-13B | 13B | 1T | 46.9 | 26.34 | 17.3 | 56.14 | 81.05 |
|
| 82 |
-
| LLaMA2-7B | 7B | 2T | 45.3 | 25.89 | 13.5 | 54.10 | 78.64 |
|
| 83 |
-
| LLaMA2-13B | 13B | 2T | 54.8 | 25.67 | 24.3 | 59.81 | 82.58 |
|
| 84 |
| MPT-7B | 7B | 1T | 26.8 | 25.67 | 8.3 | 46.67 | 77.41 |
|
| 85 |
| Falcon-7B | 7B | 1.5T | 26.2 | 23.66 | 4.9 | 47.61 | 78.23 |
|
| 86 |
| Pythia-12B | 12B | 300B | 26.5 | 24.33 | 4.09 | 40.61 | 68.83 |
|
| 87 |
-
| LLM360-Amber | 7B | 1.3T | 24.5 | 27.01 | 4.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 88 |
|
| 89 |
## **Citations**
|
| 90 |
If you use this model in your research, please cite it as follows:
|
|
|
|
| 74 |
- **Optimizer**: Diloco/LocalSGD - Inner Optimizer: AdamW, Outer Optmizer: Nesterov SGD
|
| 75 |
|
| 76 |
**Performance on benchmarks**
|
| 77 |
+
|
| 78 |
+
Base Models:
|
| 79 |
| Model | Size | Tokens | MMLU | GPQA | GSM8K | ARC-C | Hellaswag |
|
| 80 |
|---|---|---|---|---|---|---|---|
|
| 81 |
+
| INTELLECT | 10B | 1T | 37.5 | 26.12 | 8.1 | 52.13 | 72.26 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 82 |
| MPT-7B | 7B | 1T | 26.8 | 25.67 | 8.3 | 46.67 | 77.41 |
|
| 83 |
| Falcon-7B | 7B | 1.5T | 26.2 | 23.66 | 4.9 | 47.61 | 78.23 |
|
| 84 |
| Pythia-12B | 12B | 300B | 26.5 | 24.33 | 4.09 | 40.61 | 68.83 |
|
| 85 |
+
| LLM360-Amber | 7B | 1.3T | 24.5 | 27.01 | 4.32 | 42.75 | 74.08 |
|
| 86 |
+
| LLaMA-7B | 7B | 1T | 35.1 | 23.21 | 9.7 | 50.43 | 78.19 |
|
| 87 |
+
| LLaMA-13B | 13B | 1T | 46.9 | 26.34 | 17.3 | 56.14 | 81.05 |
|
| 88 |
+
| LLaMA2-7B | 7B | 2T | 45.3 | 25.89 | 13.5 | 54.10 | 78.64 |
|
| 89 |
+
| LLaMA2-13B | 13B | 2T | 54.8 | 25.67 | 24.3 | 59.81 | 82.58 |
|
| 90 |
+
|
| 91 |
+
[Instruction-Tuned Models](https://huggingface.co/PrimeIntellect/INTELLECT-1-Instruct):
|
| 92 |
+
| Model | Size | Tokens | MMLU | GPQA | GSM8K | ARC-C | Hellaswag |
|
| 93 |
+
|---|---|---|---|---|---|---|---|
|
| 94 |
+
| INTELLECT-Instruct | 10B | 1T | 49.89 | 28.32 | 38.58 | 54.52 | 71.42 |
|
| 95 |
+
| MPT-7B-Chat | 7B | 1T | 36.29 | 26.79 | 8.26 | 51.02 | 75.88 |
|
| 96 |
+
| Falcon-7B-Instruct | 7B | 1.5T | 25.21 | 26.34 | 4.93 | 45.82 | 70.61 |
|
| 97 |
+
| LLM360-AmberChat | 7B | 1.4T | 36.02 | 27.23 | 6.14 | 43.94 | 73.94 |
|
| 98 |
+
| LLaMA2-7B-Chat | 7B | 2T | 47.20 | 28.57 | 23.96 | 53.33 | 78.69 |
|
| 99 |
+
| LLaMA2-13B-Chat | 13B | 2T | 53.51 | 28.35 | 37.15 | 59.73 | 82.47 |
|
| 100 |
|
| 101 |
## **Citations**
|
| 102 |
If you use this model in your research, please cite it as follows:
|