Upload folder using huggingface_hub

README.md

@@ -1,3 +1,124 @@
----
-license: apache-2.0
----
+---
+library_name: transformers
+base_model: roberta-large-emopillars-contextual
+metrics:
+- f1
+model-index:
+- name: roberta-large-emopillars-contextual-emocontext
+  results: []
+---
+
+# roberta-large-emopillars-contextual-emocontext
+
+This model is a fine-tuned version of [roberta-large-emopillars-contextual](https://huggingface.co/alex-shvets/roberta-large-emopillars-contextual) on the [EmoContext dataset](https://paperswithcode.com/dataset/emocontext).
+
+<img src="https://huggingface.co/datasets/alex-shvets/images/resolve/main/emopillars_color_2.png" width="450">
+
+## Model description
+
+The model is a multi-label classifier over 28 emotional classes for a context-aware scenario, fine-tuned on a dataset of 4 classes (_angry_, _sad_, _happy_, and _others_) that we initially relabelled (see _Training data_ for details). The resulting model takes as input either a context concatenated with a character description and an utterance, and extracts emotions only from the utterance, or a three-turn dialogue and identifies emotions in the last utterance.
+
+## How to use
+
+Here is how to use this model:
+
+```python
+>>> import torch
+>>> from transformers import pipeline
+>>> model_name = "roberta-large-emopillars-contextual-emocontext"
+>>> threshold = 0.10
+>>> emotions = [
+>>>     "admiration", "amusement", "anger", "annoyance", "approval", "caring", "confusion",
+>>>     "curiosity", "desire", "disappointment", "disapproval", "disgust", "embarrassment",
+>>>     "excitement", "fear", "gratitude", "grief", "joy", "love", "nervousness", "optimism",
+>>>     "pride", "realization", "relief", "remorse", "sadness", "surprise", "neutral"
+>>> ]
+>>> label_to_emotion = dict(zip(list(range(len(emotions))), emotions))
+>>> emotion_to_emocontext = dict(zip(emotions, ["others"]*len(emotions)))
+>>> emotion_to_emocontext.update({
+>>>     "anger": "angry",
+>>>     "sadness": "sad",
+>>>     "joy": "happy"
+>>> })
+>>> device = torch.device("cuda" if torch.cuda.is_available() else "CPU")
+>>> pipe = pipeline("text-classification", model=model_name, truncation=True,
+>>>                  return_all_scores=True, device=-1 if device.type=="cpu" else 0)
+>>> # input in a format f"{context} {character}: \"{utterance}\""
+>>> # alternative input format: f"{persona1}: {utterance1}, {persona2}:\"{utterance2}\"\n{persona1}: \"{utterance3}\""
+>>> utterances_in_contexts = [
+>>>     "A user watched a video of a musical performance on YouTube. This user expresses an opinion and thoughts. User: \"Ok is it just me or is anyone else getting goosebumps too???\"",
+>>>     "User: \"But...\", Conversational Agent: \"then\"\nUser: \"I’m feeling nervous\""
+>>> ]
+>>> outcome = pipe(utterances_in_contexts)
+>>> dominant_classes = [
+>>>     [prediction for prediction in example if prediction['score'] >= threshold]
+>>>     for example in outcome
+>>> ]
+>>> for example in dominant_classes:
+>>>     print(", ".join([
+>>>         "%s (%s): %.2lf" % (
+>>>             label_to_emotion[int(prediction['label'])], 
+>>>             emotion_to_emocontext[label_to_emotion[int(prediction['label'])]], 
+>>>             prediction['score']
+>>>         ) 
+>>>         for prediction in sorted(example, key=lambda x: x['score'], reverse=True)
+>>>     ]))
+excitement (others): 0.73, joy (happy): 0.23
+sadness (sad): 0.89, nervousness (others): 0.12
+```
+
+## Training data
+
+The training data consists of 30,157 samples of the [EmoContext dataset](https://paperswithcode.com/dataset/emocontext). We relabelled the _others_ examples in the training set by choosing the most probable label predicted by our base 28-class contextual model (see our paper for details).
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 2e-05
+- train_batch_size: 16
+- eval_batch_size: 8
+- seed: 752
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: linear
+- num_epochs: 3.0
+
+### Framework versions
+
+- Transformers 4.45.0.dev0
+- Pytorch 2.4.0a0+gite3b9b71
+- Datasets 2.21.0
+- Tokenizers 0.19.1
+
+## Evaluation
+
+Scores for the evaluation on the EmoContext dev split:
+
+| **class** | **precision**| **recall** | **f1-score** | **support** |
+| :--- | :---: | :---: | :---: | ---: |
+| angry | 0.76 | 0.80 | 0.78 |  150 |
+| sad | 0.81 | 0.80 | 0.81 |  125 |
+| happy | 0.72 | 0.75 | 0.73 |  142 |
+| others | 0.96 | 0.96 | 0.96 | 2335 |
+| **micro avg** | 0.93 | 0.93 | 0.93 | 2752 |
+| **macro avg** | 0.81 | 0.83 | 0.82 | 2752 |
+| **weighted avg** | 0.93 | 0.93 | 0.93 | 2752 |
+| **samples avg** | 0.93 | 0.93 | 0.93 | 2752 |
+
+For more details on the evaluation, please visit our [GitHub repository](https://github.com/alex-shvets/emopillars).
+
+
+## Disclaimer
+
+<details>
+
+<summary>Click to expand</summary>
+
+The model published in this repository is intended for a generalist purpose and is available to third parties. This model may have bias and/or any other undesirable distortions.
+
+When third parties deploy or provide systems and/or services to other parties using this model (or using systems based on this model) or become users of the model, they should note that it is their responsibility to mitigate the risks arising from its use and, in any event, to comply with applicable regulations, including regulations regarding the use of Artificial Intelligence.
+
+In no event shall the creator of the model be liable for any results arising from the use made by third parties of this model.
+
+</details>

config.json

@@ -0,0 +1,89 @@
+{
+  "_name_or_path": "roberta_mistfull_64b_10ep752_context_actor_rewutt",
+  "architectures": [
+    "RobertaForSequenceClassification"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "bos_token_id": 0,
+  "classifier_dropout": null,
+  "eos_token_id": 2,
+  "finetuning_task": "text-classification",
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 1024,
+  "id2label": {
+    "0": "0",
+    "1": "1",
+    "2": "10",
+    "3": "11",
+    "4": "12",
+    "5": "13",
+    "6": "14",
+    "7": "15",
+    "8": "16",
+    "9": "17",
+    "10": "18",
+    "11": "19",
+    "12": "2",
+    "13": "20",
+    "14": "21",
+    "15": "22",
+    "16": "23",
+    "17": "24",
+    "18": "25",
+    "19": "26",
+    "20": "27",
+    "21": "3",
+    "22": "4",
+    "23": "5",
+    "24": "6",
+    "25": "7",
+    "26": "8",
+    "27": "9"
+  },
+  "initializer_range": 0.02,
+  "intermediate_size": 4096,
+  "label2id": {
+    "0": 0,
+    "1": 1,
+    "10": 2,
+    "11": 3,
+    "12": 4,
+    "13": 5,
+    "14": 6,
+    "15": 7,
+    "16": 8,
+    "17": 9,
+    "18": 10,
+    "19": 11,
+    "2": 12,
+    "20": 13,
+    "21": 14,
+    "22": 15,
+    "23": 16,
+    "24": 17,
+    "25": 18,
+    "26": 19,
+    "27": 20,
+    "3": 21,
+    "4": 22,
+    "5": 23,
+    "6": 24,
+    "7": 25,
+    "8": 26,
+    "9": 27
+  },
+  "layer_norm_eps": 1e-05,
+  "max_position_embeddings": 514,
+  "model_type": "roberta",
+  "num_attention_heads": 16,
+  "num_hidden_layers": 24,
+  "pad_token_id": 1,
+  "position_embedding_type": "absolute",
+  "problem_type": "multi_label_classification",
+  "torch_dtype": "float32",
+  "transformers_version": "4.45.0.dev0",
+  "type_vocab_size": 1,
+  "use_cache": true,
+  "vocab_size": 50265
+}

label_to_emocontext_classes.json

@@ -0,0 +1,30 @@
+{
+    "anger": "angry",
+    "sadness": "sad",
+    "joy": "happy",
+    "admiration": "others",
+    "amusement": "others",
+    "annoyance": "others",
+    "approval": "others",
+    "caring": "others",
+    "confusion": "others",
+    "curiosity": "others",
+    "desire": "others",
+    "disappointment": "others",
+    "disapproval": "others",
+    "disgust": "others",
+    "embarrassment": "others",
+    "excitement": "others",
+    "fear": "others",
+    "gratitude": "others",
+    "grief": "others",
+    "love": "others",
+    "nervousness": "others",
+    "optimism": "others",
+    "pride": "others",
+    "realization": "others",
+    "relief": "others",
+    "remorse": "others",
+    "surprise": "others",
+    "neutral": "others"
+}

labels.txt

@@ -0,0 +1,28 @@
+admiration
+amusement
+anger
+annoyance
+approval
+caring
+confusion
+curiosity
+desire
+disappointment
+disapproval
+disgust
+embarrassment
+excitement
+fear
+gratitude
+grief
+joy
+love
+nervousness
+optimism
+pride
+realization
+relief
+remorse
+sadness
+surprise
+neutral

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d96256a9cb2f0522a96ec69197482b5629a5dbe880de6df0dd04c61b3ab01db5
+size 1421602016

special_tokens_map.json

@@ -0,0 +1,51 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "cls_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "mask_token": {
+    "content": "<mask>",
+    "lstrip": true,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<pad>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "sep_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer_config.json

@@ -0,0 +1,64 @@
+{
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<s>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<pad>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "</s>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "3": {
+      "content": "<unk>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50264": {
+      "content": "<mask>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "<s>",
+  "clean_up_tokenization_spaces": true,
+  "cls_token": "<s>",
+  "eos_token": "</s>",
+  "errors": "replace",
+  "mask_token": "<mask>",
+  "max_length": 512,
+  "model_max_length": 512,
+  "pad_to_multiple_of": null,
+  "pad_token": "<pad>",
+  "pad_token_type_id": 0,
+  "padding_side": "right",
+  "sep_token": "</s>",
+  "stride": 0,
+  "tokenizer_class": "RobertaTokenizer",
+  "trim_offsets": true,
+  "truncation_side": "left",
+  "truncation_strategy": "longest_first",
+  "unk_token": "<unk>"
+}

trainer_state.json

@@ -0,0 +1,119 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 3.0,
+  "eval_steps": 500,
+  "global_step": 5655,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.26525198938992045,
+      "grad_norm": 1.645719051361084,
+      "learning_rate": 1.8231653404067196e-05,
+      "loss": 0.057,
+      "step": 500
+    },
+    {
+      "epoch": 0.5305039787798409,
+      "grad_norm": 1.2083187103271484,
+      "learning_rate": 1.6463306808134398e-05,
+      "loss": 0.0443,
+      "step": 1000
+    },
+    {
+      "epoch": 0.7957559681697612,
+      "grad_norm": 0.7489728331565857,
+      "learning_rate": 1.4694960212201592e-05,
+      "loss": 0.0428,
+      "step": 1500
+    },
+    {
+      "epoch": 1.0610079575596818,
+      "grad_norm": 1.094058871269226,
+      "learning_rate": 1.292661361626879e-05,
+      "loss": 0.04,
+      "step": 2000
+    },
+    {
+      "epoch": 1.3262599469496021,
+      "grad_norm": 1.3806736469268799,
+      "learning_rate": 1.1158267020335986e-05,
+      "loss": 0.0294,
+      "step": 2500
+    },
+    {
+      "epoch": 1.5915119363395225,
+      "grad_norm": 0.7832322120666504,
+      "learning_rate": 9.389920424403184e-06,
+      "loss": 0.0297,
+      "step": 3000
+    },
+    {
+      "epoch": 1.8567639257294428,
+      "grad_norm": 1.1327711343765259,
+      "learning_rate": 7.6215738284703815e-06,
+      "loss": 0.0273,
+      "step": 3500
+    },
+    {
+      "epoch": 2.1220159151193636,
+      "grad_norm": 1.6918989419937134,
+      "learning_rate": 5.853227232537579e-06,
+      "loss": 0.0229,
+      "step": 4000
+    },
+    {
+      "epoch": 2.387267904509284,
+      "grad_norm": 1.099871039390564,
+      "learning_rate": 4.084880636604775e-06,
+      "loss": 0.0155,
+      "step": 4500
+    },
+    {
+      "epoch": 2.6525198938992043,
+      "grad_norm": 1.586369276046753,
+      "learning_rate": 2.316534040671972e-06,
+      "loss": 0.0158,
+      "step": 5000
+    },
+    {
+      "epoch": 2.9177718832891246,
+      "grad_norm": 0.8191050291061401,
+      "learning_rate": 5.481874447391689e-07,
+      "loss": 0.0161,
+      "step": 5500
+    },
+    {
+      "epoch": 3.0,
+      "step": 5655,
+      "total_flos": 8.432016912385229e+16,
+      "train_loss": 0.030565504348984238,
+      "train_runtime": 3113.0357,
+      "train_samples_per_second": 29.062,
+      "train_steps_per_second": 1.817
+    }
+  ],
+  "logging_steps": 500,
+  "max_steps": 5655,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 3,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": true
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 8.432016912385229e+16,
+  "train_batch_size": 16,
+  "trial_name": null,
+  "trial_params": null
+}

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:851e02b88c71b052af89553746aa72f33f9edcf5091ee3d635e840987b950631
+size 5304