zhouyiqing commited on
Commit ·
f6324d8
1
Parent(s): e54f191
update readme
Browse files
README.md
CHANGED
|
@@ -34,85 +34,85 @@ With this model, we transform a linear discourse sequence into a condensed hiera
|
|
| 34 |
<tr>
|
| 35 |
<td>GPT-4o</td>
|
| 36 |
<td rowspan="12">General LLM*</td>
|
| 37 |
-
<td>
|
| 38 |
-
<td>
|
| 39 |
<td>0.0210</td>
|
| 40 |
<td>-</td>
|
| 41 |
</tr>
|
| 42 |
<tr>
|
| 43 |
<td>GPT-4.1</td>
|
| 44 |
-
<td>
|
| 45 |
<td>37.90%</td>
|
| 46 |
<td>0.0168</td>
|
| 47 |
<td>-</td>
|
| 48 |
</tr>
|
| 49 |
<tr>
|
| 50 |
<td>OpenAI o3</td>
|
| 51 |
-
<td>
|
| 52 |
-
<td>
|
| 53 |
<td>0.0168</td>
|
| 54 |
<td>-</td>
|
| 55 |
</tr>
|
| 56 |
<tr>
|
| 57 |
<td>OpenAI o4-mini</td>
|
| 58 |
-
<td>
|
| 59 |
-
<td>
|
| 60 |
<td>0.0092</td>
|
| 61 |
<td>-</td>
|
| 62 |
</tr>
|
| 63 |
<tr>
|
| 64 |
<td>Claude-3.7-Sonnet</td>
|
| 65 |
-
<td>
|
| 66 |
-
<td>35.
|
| 67 |
<td>0.0286</td>
|
| 68 |
<td>-</td>
|
| 69 |
</tr>
|
| 70 |
<tr>
|
| 71 |
<td>Claude-4</td>
|
| 72 |
-
<td><u>
|
| 73 |
-
<td><u>
|
| 74 |
<td>0.0286</td>
|
| 75 |
<td>-</td>
|
| 76 |
</tr>
|
| 77 |
<tr>
|
| 78 |
<td>Gemini-2.5-flash</td>
|
| 79 |
-
<td>
|
| 80 |
-
<td>
|
| 81 |
<td>0.0040</td>
|
| 82 |
<td>-</td>
|
| 83 |
</tr>
|
| 84 |
<tr>
|
| 85 |
<td>Gemini-2.5-pro</td>
|
| 86 |
-
<td>
|
| 87 |
-
<td>
|
| 88 |
<td>0.0162</td>
|
| 89 |
<td>-</td>
|
| 90 |
</tr>
|
| 91 |
<tr>
|
| 92 |
<td>DeepSeek-V3</td>
|
| 93 |
-
<td>
|
| 94 |
-
<td>
|
| 95 |
<td>0.0012</td>
|
| 96 |
<td>-</td>
|
| 97 |
</tr>
|
| 98 |
<tr>
|
| 99 |
<td>DeepSeek-R1</td>
|
| 100 |
-
<td>
|
| 101 |
-
<td>
|
| 102 |
<td>0.0046</td>
|
| 103 |
<td>-</td>
|
| 104 |
</tr>
|
| 105 |
<tr>
|
| 106 |
<td>Qwen3-32B</td>
|
| 107 |
-
<td>
|
| 108 |
-
<td>
|
| 109 |
<td>0.0012</td>
|
| 110 |
<td>10.17<sup>†</sup></td>
|
| 111 |
</tr>
|
| 112 |
<tr>
|
| 113 |
<td>Qwen3-235B</td>
|
| 114 |
-
<td>
|
| 115 |
-
<td>
|
| 116 |
<td>0.0012</td>
|
| 117 |
<td>-</td>
|
| 118 |
</tr>
|
|
@@ -134,8 +134,8 @@ With this model, we transform a linear discourse sequence into a condensed hiera
|
|
| 134 |
<tr>
|
| 135 |
<td><b>Our Method (LingoEDU)</b></td>
|
| 136 |
<td><b>Specialized</b></td>
|
| 137 |
-
<td><b>
|
| 138 |
-
<td><b>
|
| 139 |
<td><u>0.0007</u></td>
|
| 140 |
<td><b>1.20<sup>†</sup></b></td>
|
| 141 |
</tr>
|
|
@@ -152,7 +152,7 @@ With this model, we transform a linear discourse sequence into a condensed hiera
|
|
| 152 |
- Article input: an input string built from sentence-segmented article
|
| 153 |
- Guidance grammar: an lark grammar built from sentence-segmented article
|
| 154 |
|
| 155 |
-
See in our Github repository [DeepLangAI/LingoEDU](https://github.com/DeepLangAI/LingoEDU/
|
| 156 |
|
| 157 |
### Generate with vLLM
|
| 158 |
|
|
|
|
| 34 |
<tr>
|
| 35 |
<td>GPT-4o</td>
|
| 36 |
<td rowspan="12">General LLM*</td>
|
| 37 |
+
<td>6.22</td>
|
| 38 |
+
<td>29.03%</td>
|
| 39 |
<td>0.0210</td>
|
| 40 |
<td>-</td>
|
| 41 |
</tr>
|
| 42 |
<tr>
|
| 43 |
<td>GPT-4.1</td>
|
| 44 |
+
<td>6.35</td>
|
| 45 |
<td>37.90%</td>
|
| 46 |
<td>0.0168</td>
|
| 47 |
<td>-</td>
|
| 48 |
</tr>
|
| 49 |
<tr>
|
| 50 |
<td>OpenAI o3</td>
|
| 51 |
+
<td>5.51</td>
|
| 52 |
+
<td>28.63%</td>
|
| 53 |
<td>0.0168</td>
|
| 54 |
<td>-</td>
|
| 55 |
</tr>
|
| 56 |
<tr>
|
| 57 |
<td>OpenAI o4-mini</td>
|
| 58 |
+
<td>5.87</td>
|
| 59 |
+
<td>32.66%</td>
|
| 60 |
<td>0.0092</td>
|
| 61 |
<td>-</td>
|
| 62 |
</tr>
|
| 63 |
<tr>
|
| 64 |
<td>Claude-3.7-Sonnet</td>
|
| 65 |
+
<td>6.65</td>
|
| 66 |
+
<td>35.08%</td>
|
| 67 |
<td>0.0286</td>
|
| 68 |
<td>-</td>
|
| 69 |
</tr>
|
| 70 |
<tr>
|
| 71 |
<td>Claude-4</td>
|
| 72 |
+
<td><u>5.08</u></td>
|
| 73 |
+
<td><u>43.15%</u></td>
|
| 74 |
<td>0.0286</td>
|
| 75 |
<td>-</td>
|
| 76 |
</tr>
|
| 77 |
<tr>
|
| 78 |
<td>Gemini-2.5-flash</td>
|
| 79 |
+
<td>5.82</td>
|
| 80 |
+
<td>27.82%</td>
|
| 81 |
<td>0.0040</td>
|
| 82 |
<td>-</td>
|
| 83 |
</tr>
|
| 84 |
<tr>
|
| 85 |
<td>Gemini-2.5-pro</td>
|
| 86 |
+
<td>5.61</td>
|
| 87 |
+
<td>32.66%</td>
|
| 88 |
<td>0.0162</td>
|
| 89 |
<td>-</td>
|
| 90 |
</tr>
|
| 91 |
<tr>
|
| 92 |
<td>DeepSeek-V3</td>
|
| 93 |
+
<td>6.32</td>
|
| 94 |
+
<td>33.47%</td>
|
| 95 |
<td>0.0012</td>
|
| 96 |
<td>-</td>
|
| 97 |
</tr>
|
| 98 |
<tr>
|
| 99 |
<td>DeepSeek-R1</td>
|
| 100 |
+
<td>6.26</td>
|
| 101 |
+
<td>30.65%</td>
|
| 102 |
<td>0.0046</td>
|
| 103 |
<td>-</td>
|
| 104 |
</tr>
|
| 105 |
<tr>
|
| 106 |
<td>Qwen3-32B</td>
|
| 107 |
+
<td>6.52</td>
|
| 108 |
+
<td>26.21%</td>
|
| 109 |
<td>0.0012</td>
|
| 110 |
<td>10.17<sup>†</sup></td>
|
| 111 |
</tr>
|
| 112 |
<tr>
|
| 113 |
<td>Qwen3-235B</td>
|
| 114 |
+
<td>7.67</td>
|
| 115 |
+
<td>19.10%</td>
|
| 116 |
<td>0.0012</td>
|
| 117 |
<td>-</td>
|
| 118 |
</tr>
|
|
|
|
| 134 |
<tr>
|
| 135 |
<td><b>Our Method (LingoEDU)</b></td>
|
| 136 |
<td><b>Specialized</b></td>
|
| 137 |
+
<td><b>4.77</b></td>
|
| 138 |
+
<td><b>49.60%</b></td>
|
| 139 |
<td><u>0.0007</u></td>
|
| 140 |
<td><b>1.20<sup>†</sup></b></td>
|
| 141 |
</tr>
|
|
|
|
| 152 |
- Article input: an input string built from sentence-segmented article
|
| 153 |
- Guidance grammar: an lark grammar built from sentence-segmented article
|
| 154 |
|
| 155 |
+
See in our Github repository [DeepLangAI/LingoEDU](https://github.com/DeepLangAI/LingoEDU/blob/main/inference/src/edu_core/edu_func.py).
|
| 156 |
|
| 157 |
### Generate with vLLM
|
| 158 |
|