adityasasidhar/Gemma_270M_french

Gemma 3 270M French Instruct - Fine-Tuned

This model is a fine-tune of google/gemma-3-270m-it optimized for French language instruction following and conversation. It was trained using Unsloth on a Tesla T4 GPU for approximately 3 hours.

Model Details

• Base Model: google/gemma-3-270m-it
• Language: French
• Finetuning Framework: Unsloth (PEFT/LoRA)
• Model Size: 270M parameters (efficient and lightweight)
• License: Gemma Terms of Use

Datasets

The model was trained on a combination of high-quality French instruction datasets:

1. French-Alpaca-dataset-Instruct-55K: A translation of the Alpaca dataset for instruction following.
2. OpenAssistant/oasst1: Filtered specifically for French conversations (lang="fr").

Total training examples: ~56,676

Training Details

The model was trained for 2 epochs on a single Tesla T4 GPU.

Hyperparameters

• LoRA Rank (r): 32
• LoRA Alpha: 32
• LoRA Dropout: 0.05
• Target Modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
• Batch Size: 4 per device (Effective batch size: 8 with gradient accumulation)
• Learning Rate: 2e-4
• Scheduler: Cosine with 0.03 warmup ratio
• Optimizer: AdamW 8-bit

Training Loss

The training loss converged from an initial ~2.53 down to ~1.40 over 14,000+ steps, indicating stable learning.

Usage

You can use this model with the unsloth library for fast inference, or with standard transformers.

Using Unsloth (Recommended)

from unsloth import FastLanguageModel
import torch

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name = "adityasasidhar/Gemma_270M_french",
    max_seq_length = 2048,
    load_in_4bit = True,
)
FastLanguageModel.for_inference(model)

prompt = """<start_of_turn>user
Écris un poème sur l'automne à Paris.<end_of_turn>
<start_of_turn>model
"""

inputs = tokenizer([prompt], return_tensors = "pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens = 200, use_cache = True)
print(tokenizer.batch_decode(outputs)[0])

Sample Outputs

Prompt: Explique la différence entre 'savoir' et 'connaître'.

Response: 'Savoir' est utilisé pour les faits, les informations ou les compétences acquises (ex: Je sais nager). 'Connaître' est utilisé pour les personnes, les lieux ou les choses familières (ex: Je connais Marie).

Prompt: Raconte-moi une histoire romantique se déroulant dans un café parisien.

Response: (Generates a story about a meeting at Café de Flore...)

Tools & Hardware

• Hardware: 1x NVIDIA Tesla T4 (16GB VRAM)
• Training Time: ~3 hours
• Library: Unsloth 2025.11.6