Upload README.md
Browse files
README.md
CHANGED
|
@@ -40,7 +40,7 @@ We are excited to share the release schedule for our latest models:
|
|
| 40 |

|
| 41 |
|
| 42 |
This repository provides large language models developed by [TokyoTech-LLM](https://tokyotech-llm.github.io/).
|
| 43 |
-
Read our [blog post](https://zenn.dev/tokyotech_lm/articles/d6cb3a8fdfc907) or our [paper](https://
|
| 44 |
|
| 45 |
## Model Details
|
| 46 |
|
|
@@ -224,7 +224,7 @@ The following datasets were used for continual pre-training.
|
|
| 224 |
|
| 225 |
- [Japanese Wikipedia](https://dumps.wikimedia.org/other/cirrussearch)
|
| 226 |
- [RefinedWeb](https://huggingface.co/datasets/tiiuae/falcon-refinedweb)
|
| 227 |
-
- Swallow Corpus
|
| 228 |
- [The Pile](https://huggingface.co/datasets/EleutherAI/pile)
|
| 229 |
|
| 230 |
|
|
@@ -265,3 +265,15 @@ Here are the team members:
|
|
| 265 |
- [Rio Yokota](https://twitter.com/rioyokota)
|
| 266 |
- [Kazuki Fujii](https://twitter.com/okoge_kaz)
|
| 267 |
- [Taishi Nakamura](https://twitter.com/Setuna7777_2)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 40 |

|
| 41 |
|
| 42 |
This repository provides large language models developed by [TokyoTech-LLM](https://tokyotech-llm.github.io/).
|
| 43 |
+
Read our [blog post](https://zenn.dev/tokyotech_lm/articles/d6cb3a8fdfc907) or our [paper](https://arxiv.org/abs/2404.17790)
|
| 44 |
|
| 45 |
## Model Details
|
| 46 |
|
|
|
|
| 224 |
|
| 225 |
- [Japanese Wikipedia](https://dumps.wikimedia.org/other/cirrussearch)
|
| 226 |
- [RefinedWeb](https://huggingface.co/datasets/tiiuae/falcon-refinedweb)
|
| 227 |
+
- [Swallow Corpus](https://arxiv.org/abs/2404.17733)
|
| 228 |
- [The Pile](https://huggingface.co/datasets/EleutherAI/pile)
|
| 229 |
|
| 230 |
|
|
|
|
| 265 |
- [Rio Yokota](https://twitter.com/rioyokota)
|
| 266 |
- [Kazuki Fujii](https://twitter.com/okoge_kaz)
|
| 267 |
- [Taishi Nakamura](https://twitter.com/Setuna7777_2)
|
| 268 |
+
|
| 269 |
+
## How to cite
|
| 270 |
+
```
|
| 271 |
+
@misc{fujii2024continual,
|
| 272 |
+
title={Continual Pre-Training for Cross-Lingual LLM Adaptation: Enhancing Japanese Language Capabilities},
|
| 273 |
+
author={Kazuki Fujii and Taishi Nakamura and Mengsay Loem and Hiroki Iida and Masanari Ohi and Kakeru Hattori and Hirai Shota and Sakae Mizuki and Rio Yokota and Naoaki Okazaki},
|
| 274 |
+
year={2024},
|
| 275 |
+
eprint={2404.17790},
|
| 276 |
+
archivePrefix={arXiv},
|
| 277 |
+
primaryClass={cs.CL}
|
| 278 |
+
}
|
| 279 |
+
```
|