This is LFM2.5-1.2B-Instruct quantized with AutoRound. The model is compatible with vLLM (tested: v0.13.0). Tested with an RTX 4090.

Safetensors

Model size

0.4B params

Tensor type

I32

BF16

F16

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for kaitchup/LFM2.5-1.2B-Instruct-autoround-W4A16-G128

Base model

Finetuned

Quantized

(35)

this model

Collection including kaitchup/LFM2.5-1.2B-Instruct-autoround-W4A16-G128