README.md

---
tags:
- image-classification
- timm
- transformers
- animetimm
- dghs-imgutils
library_name: timm
license: gpl-3.0
datasets:
- animetimm/danbooru-wdtagger-v4a-w640-ws-full
base_model:
- timm/swinv2_base_window8_256.ms_in1k
---

# Anime Tagger swinv2_base_window8_256.dbv4a-full

## Model Details

- **Model Type:** Multilabel Image classification / feature backbone
- **Model Stats:**
  - Params: 87.6M
  - FLOPs / MACs: 121.6G / 60.7G
  - Image size: train = 448 x 448, test = 448 x 448
- **Dataset:** [animetimm/danbooru-wdtagger-v4a-w640-ws-full](https://huggingface.co/datasets/animetimm/danbooru-wdtagger-v4a-w640-ws-full)
  - Tags Count: 650
    - Artist (#1) Tags Count: 650

## Results

|     #      |    [email protected] (F1/MCC/P/R)    |    [email protected] (F1/MCC/P/R)    |  Macro@Best (F1/P/R)  |
|:----------:|:-----------------------------:|:-----------------------------:|:---------------------:|
| Validation | 0.889 / 0.895 / 0.930 / 0.873 | 0.887 / 0.886 / 0.894 / 0.879 |          ---          |
|    Test    | 0.891 / 0.897 / 0.932 / 0.875 | 0.890 / 0.890 / 0.896 / 0.883 | 0.929 / 0.961 / 0.903 |

* `Macro/[email protected]` means the metrics on the threshold 0.40.
* `Macro@Best` means the mean metrics on the tag-level thresholds on each tags, which should have the best F1 scores.

## Thresholds

|  Category  |  Name  |  Alpha  |  Threshold  |  Micro@Thr (F1/P/R)   |  [email protected] (F1/P/R)  |  Macro@Best (F1/P/R)  |
|:----------:|:------:|:-------:|:-----------:|:---------------------:|:---------------------:|:---------------------:|
|     1      | artist |    1    |    0.67     | 0.894 / 0.925 / 0.865 | 0.891 / 0.932 / 0.875 | 0.929 / 0.961 / 0.903 |

* `Micro@Thr` means the metrics on the category-level suggested thresholds, which are listed in the table above.
* `[email protected]` means the metrics on the threshold 0.40.
* `Macro@Best` means the metrics on the tag-level thresholds on each tags, which should have the best F1 scores.

For tag-level thresholds, you can find them in [selected_tags.csv](https://huggingface.co/animetimm/swinv2_base_window8_256.dbv4a-full/resolve/main/selected_tags.csv).

## How to Use

We provided a sample image for our code samples, you can find it [here](https://huggingface.co/animetimm/swinv2_base_window8_256.dbv4a-full/blob/main/sample.webp).

### Use TIMM And Torch

Install [dghs-imgutils](https://github.com/deepghs/imgutils), [timm](https://github.com/huggingface/pytorch-image-models) and other necessary requirements with the following command

```shell
pip install 'dghs-imgutils>=0.17.0' torch huggingface_hub timm pillow pandas
```

After that you can load this model with timm library, and use it for train, validation and test, with the following code

```python
import json

import pandas as pd
import torch
from huggingface_hub import hf_hub_download
from imgutils.data import load_image
from imgutils.preprocess import create_torchvision_transforms
from timm import create_model

repo_id = 'animetimm/swinv2_base_window8_256.dbv4a-full'
model = create_model(f'hf-hub:{repo_id}', pretrained=True)
model.eval()

with open(hf_hub_download(repo_id=repo_id, repo_type='model', filename='preprocess.json'), 'r') as f:
    preprocessor = create_torchvision_transforms(json.load(f)['test'])
# Compose(
#     PadToSize(size=(512, 512), interpolation=bilinear, background_color=white)
#     Resize(size=448, interpolation=bicubic, max_size=None, antialias=True)
#     CenterCrop(size=[448, 448])
#     MaybeToTensor()
#     Normalize(mean=tensor([0.4850, 0.4560, 0.4060]), std=tensor([0.2290, 0.2240, 0.2250]))
# )

image = load_image('https://huggingface.co/animetimm/swinv2_base_window8_256.dbv4a-full/resolve/main/sample.webp')
input_ = preprocessor(image).unsqueeze(0)
# input_, shape: torch.Size([1, 3, 448, 448]), dtype: torch.float32
with torch.no_grad():
    output = model(input_)
    prediction = torch.sigmoid(output)[0]
# output, shape: torch.Size([1, 650]), dtype: torch.float32
# prediction, shape: torch.Size([650]), dtype: torch.float32

df_tags = pd.read_csv(
    hf_hub_download(repo_id=repo_id, repo_type='model', filename='selected_tags.csv'),
    keep_default_na=False
)
tags = df_tags['name']
mask = prediction.numpy() >= df_tags['best_threshold']
print(dict(zip(tags[mask].tolist(), prediction[mask].tolist())))
# {'maru_(marg0613)': 0.9999892711639404}
```
### Use ONNX Model For Inference

Install [dghs-imgutils](https://github.com/deepghs/imgutils) with the following command

```shell
pip install 'dghs-imgutils>=0.17.0'
```

Use `multilabel_timm_predict` function with the following code

```python
from imgutils.generic import multilabel_timm_predict

artist = multilabel_timm_predict(
    'https://huggingface.co/animetimm/swinv2_base_window8_256.dbv4a-full/resolve/main/sample.webp',
    repo_id='animetimm/swinv2_base_window8_256.dbv4a-full',
    fmt='artist',
)

print(artist)
# {'maru_(marg0613)': 0.9999892711639404}
```

For further information, see [documentation of function multilabel_timm_predict](https://dghs-imgutils.deepghs.org/main/api_doc/generic/multilabel_timm.html#multilabel-timm-predict).