yamatazen commited on
Commit
8bfdb55
·
verified ·
1 Parent(s): 2bae013

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +55 -51
README.md CHANGED
@@ -1,51 +1,55 @@
1
- ---
2
- base_model:
3
- - shisa-ai/shisa-v2-mistral-nemo-12b
4
- - nbeerbower/mistral-nemo-gutenberg-12B-v4
5
- - natong19/Mistral-Nemo-Instruct-2407-abliterated
6
- - Elizezen/Himeyuri-v0.1-12B
7
- library_name: transformers
8
- tags:
9
- - mergekit
10
- - merge
11
-
12
- ---
13
- # LinearWriter-12B
14
-
15
- This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
16
-
17
- ## Merge Details
18
- ### Merge Method
19
-
20
- This model was merged using the [Linear](https://arxiv.org/abs/2203.05482) merge method.
21
-
22
- ### Models Merged
23
-
24
- The following models were included in the merge:
25
- * [shisa-ai/shisa-v2-mistral-nemo-12b](https://huggingface.co/shisa-ai/shisa-v2-mistral-nemo-12b)
26
- * [nbeerbower/mistral-nemo-gutenberg-12B-v4](https://huggingface.co/nbeerbower/mistral-nemo-gutenberg-12B-v4)
27
- * [natong19/Mistral-Nemo-Instruct-2407-abliterated](https://huggingface.co/natong19/Mistral-Nemo-Instruct-2407-abliterated)
28
- * [Elizezen/Himeyuri-v0.1-12B](https://huggingface.co/Elizezen/Himeyuri-v0.1-12B)
29
-
30
- ### Configuration
31
-
32
- The following YAML configuration was used to produce this model:
33
-
34
- ```yaml
35
- merge_method: linear
36
- dtype: bfloat16
37
- out_dtype: bfloat16
38
- models:
39
- - model: natong19/Mistral-Nemo-Instruct-2407-abliterated # Uncensor
40
- parameters:
41
- weight: 1.0
42
- - model: nbeerbower/mistral-nemo-gutenberg-12B-v4 # Writing
43
- parameters:
44
- weight: [0.25, 0.3, 0.5, 0.6, 0.75]
45
- - model: Elizezen/Himeyuri-v0.1-12B # Japanese
46
- parameters:
47
- weight: [0.25, 0.3, 0.6, 0.3, 0.25]
48
- - model: shisa-ai/shisa-v2-mistral-nemo-12b # Japanese
49
- parameters:
50
- weight: [0.25, 0.3, 0.5, 0.3, 0.25]
51
- ```
 
 
 
 
 
1
+ ---
2
+ base_model:
3
+ - shisa-ai/shisa-v2-mistral-nemo-12b
4
+ - nbeerbower/mistral-nemo-gutenberg-12B-v4
5
+ - natong19/Mistral-Nemo-Instruct-2407-abliterated
6
+ - Elizezen/Himeyuri-v0.1-12B
7
+ library_name: transformers
8
+ tags:
9
+ - mergekit
10
+ - merge
11
+ - linear
12
+ language:
13
+ - en
14
+ - ja
15
+ ---
16
+ ![image/png](https://huggingface.co/yamatazen/LinearWriter-12B/resolve/main/LinearWriter-12B.png?download=true)
17
+ # LinearWriter-12B
18
+
19
+ This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
20
+
21
+ ## Merge Details
22
+ ### Merge Method
23
+
24
+ This model was merged using the [Linear](https://arxiv.org/abs/2203.05482) merge method.
25
+
26
+ ### Models Merged
27
+
28
+ The following models were included in the merge:
29
+ * [shisa-ai/shisa-v2-mistral-nemo-12b](https://huggingface.co/shisa-ai/shisa-v2-mistral-nemo-12b)
30
+ * [nbeerbower/mistral-nemo-gutenberg-12B-v4](https://huggingface.co/nbeerbower/mistral-nemo-gutenberg-12B-v4)
31
+ * [natong19/Mistral-Nemo-Instruct-2407-abliterated](https://huggingface.co/natong19/Mistral-Nemo-Instruct-2407-abliterated)
32
+ * [Elizezen/Himeyuri-v0.1-12B](https://huggingface.co/Elizezen/Himeyuri-v0.1-12B)
33
+
34
+ ### Configuration
35
+
36
+ The following YAML configuration was used to produce this model:
37
+
38
+ ```yaml
39
+ merge_method: linear
40
+ dtype: bfloat16
41
+ out_dtype: bfloat16
42
+ models:
43
+ - model: natong19/Mistral-Nemo-Instruct-2407-abliterated # Uncensor
44
+ parameters:
45
+ weight: 1.0
46
+ - model: nbeerbower/mistral-nemo-gutenberg-12B-v4 # Writing
47
+ parameters:
48
+ weight: [0.25, 0.3, 0.5, 0.6, 0.75]
49
+ - model: Elizezen/Himeyuri-v0.1-12B # Japanese
50
+ parameters:
51
+ weight: [0.25, 0.3, 0.6, 0.3, 0.25]
52
+ - model: shisa-ai/shisa-v2-mistral-nemo-12b # Japanese
53
+ parameters:
54
+ weight: [0.25, 0.3, 0.5, 0.3, 0.25]
55
+ ```