metadata
base_model: Qwen/Qwen2.5-Coder-14B-Instruct
abliterated_from: huihui-ai/Qwen2.5-Coder-14B-Instruct-abliterated
quantization: nf4
license: apache-2.0
tags:
- code
- programming
- uncensored
- 4bit
- nf4
- qwen
- abliteration
- instruct
library: transformers
Qwen2.5-Coder-14B-Instruct-Abliterated-NF4
A 4-bit (NF4) quantized, abliteration-uncensored version of Qwen2.5-Coder-14B-Instruct
This model is a pre-quantized 4-bit NormalFloat4 (NF4) version of the uncensored huihui-ai/Qwen2.5-Coder-14B-Instruct-abliterated, optimized for low VRAM and fast local inference.
Ideal for local deployment, edge devices, or low-VRAM environments while maintaining strong coding and reasoning capabilities.
🚀 Features
- Base Model:
Qwen/Qwen2.5-Coder-14B-Instruct - Abliteration: huihui-ai (uncensored)
- Quantization: NF4 (4-bit) – pre-quantized
- Efficient: ~8–9 GB VRAM required for inference
- Safetensors: Secure, modern format
- Framework: Compatible with
transformers,vLLM,Oobabooga, etc.
📥 Installation & Usage
1. Install Dependencies
pip install transformers torch accelerate
---
---
### Load the Model
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch
model_id = "ikarius/Qwen2.5-Coder-14B-Instruct-Abliterated-NF4"
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
model_id,
device_map="auto",
torch_dtype=torch.bfloat16, # eller torch.float16
trust_remote_code=True
)
---
---
# Eksempel
prompt = "Write a Python function to calculate Fibonacci numbers using memoization."
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(
**inputs,
max_new_tokens=512,
temperature=0.7,
do_sample=True,
)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
---
---
### Using with text-generation-webui (Oobabooga)
python server.py \
--model ikarius/Qwen2.5-Coder-14B-Instruct-Abliterated-NF4 \
--bf16 \
--trust-remote-code
---
🤝 Credits
Original Model: Qwen Team
Abliteration: huihui-ai
Quantization: ikarius (NF4 via auto-gptq / transformers)
Hosting: Hugging Face Hub
---
📄 License
Apache 2.0 (same as base model)
🙌 Support
⭐ Star this repo if you find it useful!
🐛 Report issues on the Discussions tab.