Hugging Face's logo

M-CLIP
/
XLM-Roberta-Large-Vit-B-32

Transformers
PyTorch
TensorFlow
M-CLIP
Model card Files
xet
Community
7
Lauler commited on
Commit
358a8bb
·
1 Parent(s): e421c71

Add yaml and adjust table

Browse files
Files changed (1) hide show
  1. README.md +7 -2
README.md CHANGED
@@ -1,10 +1,14 @@
 
 
 
 
1
  ## Model card: multilingual-clip
2
 
3
  Multilingual-CLIP extends OpenAI's English text encoders to multiple other languages. This model *only* contains the multilingual text encoder. The corresponding image model `ViT-B-32` needs to be retrieved via instructions found on from OpenAI's [CLIP repository on Github](https://github.com/openai/CLIP). We provide a usage example below.
4
 
5
  ## Requirements
6
 
7
- To use both the multilingual text encoder and corresponding image encoder, we need to install the packages `multilingual-clip` and `clip`.
8
 
9
  ```
10
  pip install multilingual-clip
@@ -59,6 +63,7 @@ print("Image features shape:", image_features.shape)
59
  ## Evaluation results
60
 
61
  None of the M-CLIP models have been extensivly evaluated, but testing them on Txt2Img retrieval on the humanly translated MS-COCO dataset, we see the following **R@10** results:
 
62
  | Name | En | De | Es | Fr | Zh | It | Pl | Ko | Ru | Tr | Jp |
63
  | ----------------------------------|:-----: |:-----: |:-----: |:-----: | :-----: |:-----: |:-----: |:-----: |:-----: |:-----: |:-----: |
64
  | [OpenAI CLIP Vit-B/32](https://github.com/openai/CLIP)| 90.3 | - | - | - | - | - | - | - | - | - | - |
@@ -66,7 +71,7 @@ None of the M-CLIP models have been extensivly evaluated, but testing them on Tx
66
  | [LABSE Vit-L/14](https://huggingface.co/M-CLIP/LABSE-Vit-L-14)| 91.6 | 89.6 | 89.5 | 89.9 | 88.9 | 90.1 | 89.8 | 80.8 | 85.5 | 89.8 | 73.9 |
67
  | [XLM-R Large Vit-B/32](https://huggingface.co/M-CLIP/XLM-Roberta-Large-Vit-B-32)| 91.8 | 88.7 | 89.1 | 89.4 | 89.3 | 89.8| 91.4 | 82.1 | 86.1 | 88.8 | 81.0 |
68
  | [XLM-R Vit-L/14](https://huggingface.co/M-CLIP/XLM-Roberta-Large-Vit-L-14)| 92.4 | 90.6 | 91.0 | 90.0 | 89.7 | 91.1 | 91.3 | 85.2 | 85.8 | 90.3 | 81.9 |
69
- | [XLM-R Large Vit-B/16+](https://huggingface.co/M-CLIP/XLM-Roberta-Large-Vit-B-16Plus)| <b>95.0</b> | <b>93.0</b> | <b>93.6</b> | <b>93.1</b> | <b>94.0</b> | <b>93.1</b> | <b>94.4</b> | <b>89.0</b> | <b>90.0</b> | <b>93.0</b> | <b>84.2</b> |
70
 
71
 
72
  ## Training/Model details
 
1
+ ---
2
+ language: multilingual
3
+ ---
4
+
5
  ## Model card: multilingual-clip
6
 
7
  Multilingual-CLIP extends OpenAI's English text encoders to multiple other languages. This model *only* contains the multilingual text encoder. The corresponding image model `ViT-B-32` needs to be retrieved via instructions found on from OpenAI's [CLIP repository on Github](https://github.com/openai/CLIP). We provide a usage example below.
8
 
9
  ## Requirements
10
 
11
+ To use both the multilingual text encoder and corresponding image encoder, we need to install the packages [`multilingual-clip`](https://github.com/FreddeFrallan/Multilingual-CLIP) and [`clip`](https://github.com/openai/CLIP).
12
 
13
  ```
14
  pip install multilingual-clip
 
63
  ## Evaluation results
64
 
65
  None of the M-CLIP models have been extensivly evaluated, but testing them on Txt2Img retrieval on the humanly translated MS-COCO dataset, we see the following **R@10** results:
66
+
67
  | Name | En | De | Es | Fr | Zh | It | Pl | Ko | Ru | Tr | Jp |
68
  | ----------------------------------|:-----: |:-----: |:-----: |:-----: | :-----: |:-----: |:-----: |:-----: |:-----: |:-----: |:-----: |
69
  | [OpenAI CLIP Vit-B/32](https://github.com/openai/CLIP)| 90.3 | - | - | - | - | - | - | - | - | - | - |
 
71
  | [LABSE Vit-L/14](https://huggingface.co/M-CLIP/LABSE-Vit-L-14)| 91.6 | 89.6 | 89.5 | 89.9 | 88.9 | 90.1 | 89.8 | 80.8 | 85.5 | 89.8 | 73.9 |
72
  | [XLM-R Large Vit-B/32](https://huggingface.co/M-CLIP/XLM-Roberta-Large-Vit-B-32)| 91.8 | 88.7 | 89.1 | 89.4 | 89.3 | 89.8| 91.4 | 82.1 | 86.1 | 88.8 | 81.0 |
73
  | [XLM-R Vit-L/14](https://huggingface.co/M-CLIP/XLM-Roberta-Large-Vit-L-14)| 92.4 | 90.6 | 91.0 | 90.0 | 89.7 | 91.1 | 91.3 | 85.2 | 85.8 | 90.3 | 81.9 |
74
+ | [XLM-R Large Vit-B/16+](https://huggingface.co/M-CLIP/XLM-Roberta-Large-Vit-B-16Plus)| **95.0** | **93.0** | **93.6** | **93.1** | **94.0** | **93.1** | **94.4** | **89.0** | **90.0** | **93.0** | **84.2** |
75
 
76
 
77
  ## Training/Model details