---
license: llama3.1
language:
- en
tags:
- llama
- llama-3
- gguf
- quantized
- tax
- indian-law
- finance
base_model: meta-llama/Meta-Llama-3.1-8B-Instruct
datasets:
- custom
pipeline_tag: text-generation
---

# Llama 3.1 8B - Indian Income Tax Act 1961 (GGUF)

Fine-tuned **Llama 3.1 8B Instruct** model specialized in the **Indian Income Tax Act 1961**. Optimized for tax law queries, compliance questions, and section references.

## 🎯 Model Details

- **Base Model**: [meta-llama/Meta-Llama-3.1-8B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct)
- **Fine-tuning Method**: LoRA (r=64, alpha=64) with Unsloth
- **Training Date**: 20251201
- **Context Length**: 4096 tokens
- **Format**: GGUF (ready for llama.cpp, Ollama, LM Studio, Jan, etc.)
- **Specialization**: Indian Income Tax Act 1961

## 📦 Available Quantizations

| File | Size | Use Case | RAM Required | Quality |
|------|------|----------|--------------|---------|
| `q4_k_m` | ~4.5GB | **Recommended** - Best balance | 6-8GB | ⭐⭐⭐⭐ |
| `q5_k_m` | ~5.5GB | High quality responses | 8-10GB | ⭐⭐⭐⭐⭐ |
| `q8_0` | ~8GB | Near-original quality | 10-12GB | ⭐⭐⭐⭐⭐ |
| `f16` | ~15GB | Maximum quality (if available) | 18-20GB | ⭐⭐⭐⭐⭐ |

### Quantization Guide
- **q4_k_m**: Best for most users - good quality, reasonable size
- **q5_k_m**: Better quality with slight size increase
- **q8_0**: Minimal quality loss, larger file
- **f16**: Full precision, largest file

## 🚀 Quick Start

### Using Ollama

1. Download the model file (e.g., q4_k_m)
wget https://huggingface.co/teclabs/Llama-3.1-8b-Instruct-Ind-Tax-Act-1961_Optimal/resolve/main/llama-tax-act-q4_k_m.gguf

2. Create Modelfile
cat > Modelfile << 'EOF'
FROM ./llama-tax-act-q4_k_m.gguf

PARAMETER temperature 0.7
PARAMETER top_p 0.9
PARAMETER top_k 40
PARAMETER repeat_penalty 1.1

SYSTEM """You are an expert on the Indian Income Tax Act 1961.
Provide accurate, detailed information about tax regulations, exemptions,
deductions, and compliance requirements. Always cite relevant sections
when applicable."""
EOF

3. Create the model
ollama create llama-tax-act -f Modelfile

4. Run it
ollama run llama-tax-act

### Using llama.cpp

Download model
wget https://huggingface.co/teclabs/Llama-3.1-8b-Instruct-Ind-Tax-Act-1961_Optimal/resolve/main/llama-tax-act-q4_k_m.gguf

Run inference
./llama-cli -m llama-tax-act-q4_k_m.gguf -p "Explain Section 80C deductions:" -n 512

Download model
wget https://huggingface.co/teclabs/Llama-3.1-8b-Instruct-Ind-Tax-Act-1961_Optimal/resolve/main/llama-tax-act-q4_k_m.gguf

Run inference
./llama-cli -m llama-tax-act-q4_k_m.gguf -p "Explain Section 80C deductions:" -n 512

from llama_cpp import Llama

Load model
llm = Llama(
model_path="./llama-tax-act-q4_k_m.gguf",
n_ctx=4096,
n_threads=8,
n_gpu_layers=35 # Adjust based on your GPU
)

Generate response
output = llm(
"What are the tax implications under Section 54?",
max_tokens=512,
temperature=0.7,
top_p=0.9,
)

print(output['choices']['text'])


## 💡 Example Queries

Q: What are the deductions available under Chapter VI-A?
Q: Explain Section 80C and its limit for FY 2023-24
Q: What is the difference between Section 80C and 80D?
Q: How is capital gains tax calculated under Section 112A?
Q: What are the exemptions available under Section 10?


## 📊 Training Details

- **Training Hardware**: NVIDIA A100 80GB
- **Training Time**: ~8 minutes (including quantization)
- **Dataset**: Custom corpus from Income Tax Act 1961
- **Epochs**: Optimized for convergence
- **Learning Rate**: 2e-4 with cosine schedule
- **Precision**: BF16 training, quantized for deployment

## ⚙️ Technical Specifications

- **Architecture**: Llama 3.1 (8B parameters)
- **Vocabulary**: 128,256 tokens
- **Max Context**: 4096 tokens
- **Attention**: Grouped-Query Attention (GQA)
- **Activation**: SwiGLU
- **LoRA Rank**: 64 (higher than standard for better quality)

## 📈 Performance

- **Inference Speed** (q4_k_m on RTX 3090): ~40-50 tokens/sec
- **Inference Speed** (q4_k_m on M1 Max): ~25-35 tokens/sec
- **Quality**: Specialized responses with section references

## ⚠️ Limitations

- Trained on Income Tax Act 1961 as of training date (20251201)
- May not reflect latest amendments after this date
- Should be used as reference only, not legal advice
- Always verify with official sources

## 📜 License

This model inherits the [Llama 3.1 Community License](https://github.com/meta-llama/llama-models/blob/main/models/llama3_1/LICENSE).

## 🙏 Acknowledgments

- **Meta AI** for Llama 3.1 base model
- **Unsloth AI** for efficient fine-tuning framework
- **ggerganov** for llama.cpp and GGUF format

## 📧 Contact

For questions or issues, please open an issue on the [repository](https://huggingface.co/teclabs/Llama-3.1-8b-Instruct-Ind-Tax-Act-1961_Optimal/discussions).

## 🔄 Updates

**2025-12-01**: Initial release with q4_k_m, q5_k_m, q8_0 quantizations

---

**Disclaimer**: This model is for educational and research purposes. Tax laws are complex and subject to change. Always consult qualified tax professionals for advice.