Models for local deployment - a yjernite Collection

yjernite 's Collections

Text embedding models I use

super-smol to fine-tune

Models for local deployment

Inference-supported production models

Document processing

Privacy

Models for local deployment

updated 27 days ago

Qwen/Qwen3-4B-Instruct-2507

Text Generation • 4B • Updated Sep 17 • 3.88M • • 605
unsloth/Qwen3-4B-Instruct-2507-GGUF

4B • Updated Aug 20 • 71.3k • 123
HuggingFaceTB/SmolLM3-3B

Text Generation • 3B • Updated Sep 10 • 98.9k • • 858
unsloth/SmolLM3-3B-GGUF

3B • Updated Jul 8 • 4.67k • 17
Qwen/Qwen3-Embedding-4B-GGUF

4B • Updated Aug 3 • 14.4k • 78

Note ./build/bin/llama-server -hf Qwen/Qwen3-Embedding-4B-GGUF:Q6_K --port 8080 --pooling last --verbose-prompt --parallel 4 --batch-size 8 --embedding --ctx-size 8000 --cache-type-k q4_0 --gpu-layers 36
Qwen/Qwen3-Embedding-0.6B

Feature Extraction • 0.6B • Updated Jun 20 • 3.16M • • 803
google/gemma-3-270m

Text Generation • 0.3B • Updated Aug 14 • 50.6k • 940
ByteDance/Ouro-1.4B

Text Generation • Updated Nov 16 • 14.2k • 57