---
license: apache-2.0
language:
- en
tags:
- propaganda-detection
- social-media
- multi-label-classification
- bertweet
- nci-protocol
datasets:
- synapti/nci-social-v3
metrics:
- f1
pipeline_tag: text-classification
base_model: vinai/bertweet-base
---

# NCI Social Media Classifier v3

A multi-label classifier for detecting propaganda techniques in social media content (tweets, posts, short-form text).

## Model Description

This model is fine-tuned from [vinai/bertweet-base](https://huggingface.co/vinai/bertweet-base) for multi-label classification of 18 propaganda techniques defined in the SemEval-2020 Task 11.

**Key improvements in v3:**
- **4x better F1 Macro** (0.55 vs 0.14 in v2) through synthetic data augmentation
- **15/18 techniques** with good coverage (F1 ≥ 0.3), up from 6/18
- Better detection of rare techniques: Repetition, Flag-Waving, Slogans, Black-and-White Fallacy

## Performance

| Metric | Score |
|--------|-------|
| F1 Micro | 0.796 |
| F1 Macro | 0.551 |
| Techniques with F1 ≥ 0.3 | 15/18 |

### Per-Technique Performance

| Technique | F1 | Support |
|-----------|-----|---------|
| Name_Calling,Labeling | 0.960 | 724 |
| Appeal_to_Authority | 0.897 | 16 |
| Red_Herring | 0.880 | 14 |
| Obfuscation,Intentional_Vagueness,Confusion | 0.800 | 6 |
| Loaded_Language | 0.796 | 480 |
| Whataboutism,Straw_Men,Red_Herring | 0.700 | 26 |
| Slogans | 0.667 | 4 |
| Appeal_to_fear-prejudice | 0.662 | 246 |
| Reductio_ad_hitlerum | 0.612 | 68 |
| Doubt | 0.564 | 23 |
| Exaggeration,Minimisation | 0.524 | 146 |
| Flag-Waving | 0.467 | 23 |
| Repetition | 0.444 | 7 |
| Bandwagon | 0.400 | 8 |
| Black-and-White_Fallacy | 0.300 | 17 |

## Intended Use

- Analyzing social media posts for propaganda techniques
- Content moderation assistance
- Media literacy tools
- Research on information manipulation

## Usage

```python
from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

# Load model
tokenizer = AutoTokenizer.from_pretrained("vinai/bertweet-base", use_fast=False)
model = AutoModelForSequenceClassification.from_pretrained("synapti/nci-social-classifier-v3")

# Technique labels
TECHNIQUES = [
    "Loaded_Language", "Appeal_to_fear-prejudice", "Exaggeration,Minimisation",
    "Repetition", "Flag-Waving", "Name_Calling,Labeling", "Reductio_ad_hitlerum",
    "Black-and-White_Fallacy", "Causal_Oversimplification", "Whataboutism,Straw_Men,Red_Herring",
    "Straw_Man", "Red_Herring", "Doubt", "Appeal_to_Authority", "Thought-terminating_Cliches",
    "Bandwagon", "Slogans", "Obfuscation,Intentional_Vagueness,Confusion",
]

# Predict
text = "Build the wall! Build the wall! Build the wall! #MAGA"
inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=128)

with torch.no_grad():
    outputs = model(**inputs)
    probs = torch.sigmoid(outputs.logits)[0]

# Get detected techniques
detected = [(TECHNIQUES[i], float(probs[i])) for i in range(18) if probs[i] > 0.3]
print(detected)
# [('Slogans', 0.51), ('Repetition', 0.39), ...]
```

## Training Data

Trained on [synapti/nci-social-v3](https://huggingface.co/datasets/synapti/nci-social-v3):
- 9,069 training samples
- 1,134 validation samples  
- 1,134 test samples

Data sources:
- LLM-labeled HateXplain samples
- SemEval span_propaganda short samples
- Synthetic examples for underrepresented techniques
- Balanced negative samples

## Training Procedure

- **Base model**: vinai/bertweet-base
- **Loss**: Focal Loss (γ=2.0) for class imbalance
- **Optimizer**: AdamW
- **Learning rate**: 2e-5
- **Batch size**: 32
- **Epochs**: 8
- **Hardware**: NVIDIA L4 GPU

## Limitations

- Optimized for English social media content
- May not generalize well to formal news articles (use nci-technique-classifier for those)
- Some rare techniques (Straw_Man, Thought-terminating_Cliches) still have limited coverage

## Citation

```bibtex
@misc{nci-social-classifier-v3,
  author = {Synapti},
  title = {NCI Social Media Classifier v3},
  year = {2024},
  publisher = {HuggingFace},
  url = {https://huggingface.co/synapti/nci-social-classifier-v3}
}
```

## License

Apache 2.0