| | --- |
| | base_model: |
| | - Moraliane/NekoMix-12B |
| | library_name: exllamav2 |
| | tags: |
| | - mergekit |
| | - merge |
| | - rp |
| | - russian |
| | - role-play |
| | language: |
| | - ru |
| | - en |
| | --- |
| | # NekoMix-12B-exl2 |
| | Original model: [NekoMix-12B](https://huggingface.co/Moraliane/NekoMix-12B) by [Moraliane](https://huggingface.co/Moraliane) |
| |
|
| | ## Quants |
| | [4bpw h6 (main)](https://huggingface.co/cgus/NekoMix-12B-exl2/tree/main) |
| | [4.5bpw h6](https://huggingface.co/cgus/NekoMix-12B-exl2/tree/4.5bpw-h6) |
| | [5bpw h6](https://huggingface.co/cgus/NekoMix-12B-exl2/tree/5bpw-h6) |
| | [6bpw h6](https://huggingface.co/cgus/NekoMix-12B-exl2/tree/6bpw-h6) |
| | [8bpw h8](https://huggingface.co/cgus/NekoMix-12B-exl2/tree/8bpw-h8) |
| |
|
| | ## Quantization notes |
| | Made with Exllamav2 0.2.8 with default dataset. |
| | It seems to be primarily a Russian RP model. No clue how it performs at all. |
| | It can be used with TabbyAPI or Text-Generation-WebUI with RTX GPU on Windows or RTX/ROCm on Linux. |
| | Exllamav2 doesn't support offloading to RAM, so make sure it fits your GPU. Otherwise use GGUF quants instead. |
| | For example, with 12GB VRAM it can be used at 6bpw/Q6 cache at 16k context. |
| |
|
| | Эта модель может использоваться с TabbyAPI или Text-Generation-WebUI. |
| | Для работы с ней требуется Nvidia RTX (Windows) или RTX/ROCm (Linux). |
| | Exl2 формат требует, чтобы модель полностью помещалась в видеопамяти. |
| | Например, с 12ГБ видеопамяти можно использовать 6bpw версию с Q6 кэшем с 16k контекстом. |
| |
|
| | # Original model card |
| | # NekoMix-12B |
| |
|
| |  |
| | # GGUF: |
| | https://huggingface.co/mradermacher/NekoMix-12B-GGUF |
| |
|
| | # GGUF imatrix: |
| | Soon... |
| |
|
| | # Presets: |
| | https://huggingface.co/Moraliane/NekoMix-12B/blob/main/pres/NekoMixRUS.json |
| |  |
| |  |
| | Так же рекомендую использовать Mistral V3-Tekken в качестве Context Template и Instruct Template (!!Спорно!!) |
| | # Sampler: |
| |  |
| |
|
| |  |
| |
|
| |
|
| | Рекомендую для начала использовать стоковый пресет simple-1 а так же Parameters_Top(A)Kek из https://huggingface.co/MarinaraSpaghetti/SillyTavern-Settings/tree/main/Parameters |
| | |
| | ``` |
| | Temp - 0,7 - 1,25 ~ |
| | TopA - 0,1 |
| | DRY - 0,8 1,75 2 0 |
| | I recommend trying the stock presets from SillyTavern, such as simple-1. |
| | ``` |
| | |
| | |
| | |
| | |
| | |
| | # Testmrg |
| | |
| | This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit). |
| | |
| | ## Merge Details |
| | ### Merge Method |
| | |
| | This model was merged using the della_linear merge method using E:\Programs\TextGen\text-generation-webui\models\IlyaGusev_saiga_nemo_12b as a base. |
| | |
| | ### Models Merged |
| | |
| | The following models were included in the merge: |
| | * E:\Programs\TextGen\text-generation-webui\models\MarinaraSpaghetti_NemoMix-Unleashed-12B |
| | * E:\Programs\TextGen\text-generation-webui\models\Vikhrmodels_Vikhr-Nemo-12B-Instruct-R-21-09-24 |
| | * E:\Programs\TextGen\text-generation-webui\models\TheDrummer_Rocinante-12B-v1.1 |
| |
|
| | ### Configuration |
| |
|
| | The following YAML configuration was used to produce this model: |
| |
|
| | ```yaml |
| | models: |
| | - model: E:\Programs\TextGen\text-generation-webui\models\IlyaGusev_saiga_nemo_12b |
| | parameters: |
| | weight: 0.5 # Основной акцент на русском языке |
| | density: 0.4 |
| | - model: E:\Programs\TextGen\text-generation-webui\models\MarinaraSpaghetti_NemoMix-Unleashed-12B |
| | parameters: |
| | weight: 0.2 # РП модель, чуть меньший вес из-за ориентации на английский |
| | density: 0.4 |
| | - model: E:\Programs\TextGen\text-generation-webui\models\TheDrummer_Rocinante-12B-v1.1 |
| | parameters: |
| | weight: 0.2 # Увеличенный вес для усиления РП аспектов |
| | density: 0.5 # Повышенная плотность для более сильного влияния |
| | - model: E:\Programs\TextGen\text-generation-webui\models\Vikhrmodels_Vikhr-Nemo-12B-Instruct-R-21-09-24 |
| | parameters: |
| | weight: 0.25 # Русскоязычная поддержка и баланс |
| | density: 0.4 |
| | |
| | merge_method: della_linear |
| | base_model: E:\Programs\TextGen\text-generation-webui\models\IlyaGusev_saiga_nemo_12b |
| | parameters: |
| | epsilon: 0.05 |
| | lambda: 1 |
| | dtype: float16 |
| | tokenizer_source: base |
| | |
| | ``` |
