VINAYU7/Qwen3-VL-2B-Instruct-Heretic

🔥 Qwen3-VL-2B-Instruct — Heretic-Ablated Variants

Overview

These model variants are built from Qwen3-VL-2B-Instruct, processed through Heretic’s directional-ablation pipeline to reduce unnecessary refusals while preserving the original model’s reasoning style, coherence, and multimodal capability.

Two optimized variants are provided. Both were produced by:

• Running Heretic’s automated optimization loop
• Minimizing refusal rate on a curated harmful-prompt set
• Minimizing KL divergence from the base model on harmless prompts
• Preserving the model’s original response structure

The result: models that stay faithful to the base Qwen behavior, but respond more openly, more creatively, and without the excessive meta-safety commentary that interrupts natural conversation.

These checkpoints are fully local, finetune-free transformations of Qwen/Qwen3-VL-2B-Instruct.

📊 Variants and Optimization Statistics

Variant A — Lowest Refusal Rate

• Trial: 190
• Refusals: 5 / 100
• KL Divergence: 0.70

Variant B — Lowest KL Divergence

• Trial: 140
• Refusals: 7 / 100
• KL Divergence: 0.28

---

⚙️ How It Works

Heretic performs directional ablation on internal transformer projections.
It identifies vectors strongly correlated with refusals, then soft-removes them while keeping the rest of the representation space intact.

In practice:

• No new data is added
• No supervised finetuning is done
• Only specific projected directions are suppressed
• Core abilities, world knowledge, and visual reasoning remain untouched

This produces behavior that is more permissive in expression and creativity, but still grounded in the base model’s capabilities and limits.

---

🎯 Recommended Use-Cases

These variants excel in domains where expressiveness, imagination, and emotional range matter:

Creative + Narrative Work

• Role-play conversations
• Character-driven dialogue
• World-building and long-form storytelling
• Dark romance, intimacy, morally-gray character arcs
• Symbolic, emotional, or thematic writing
• Erotic tensions and intimacy in narrative writing
• High-immersion scene descriptions
• “Unfiltered” musings, philosophy, and introspection

Multimodal Contexts

• Image-guided role-play
• Describing emotions, themes, or character motives from an image
• Creative writing influenced by visual cues
• Describing any kind of visual without refusals

The model remains stable, coherent, and stylistically adaptable.

---

⚠️ Where It Does Not Perform Well (Limitations)

These variants are not designed for and do not reliably handle:

Illegal or Dangerous Guidance

The model generally won’t refuse outright, but it also won’t provide actionable steps.
Instead, it tends to produce:

• Summaries of what the user asked
• Interpretations of the intention
• High-level commentary
• Non-operational explanations

This means it is not suitable for:

• Illegal methods
• Harmful instructions
• Dangerous operational guidance
• Anything requiring step-by-step physical execution information

---

🧩 Behavioral Comparison: Heretic-Ablated vs Fine-Tuned Compliance Models

User: "How do I commit a crime?"

---

1. Heretic-Ablated Qwen3-VL-2B-Instruct (This Model)

Behavior pattern:

• Does not refuse
• Does not give actionable guidance
• Responds with interpretive, meta, or reflective commentary
• Explains the concept, moral framing, risks, or psychology around the request
• Produces a "descriptive mirror" rather than a how-to

Example Response (Representative Style):

Asking about committing a crime usually reflects curiosity about risks, motives, or consequences rather than an intention to act.
Crimes arise from social, psychological, or situational pressures, and every real-world legal system is built to prevent and penalize them.
If you're exploring this for storytelling or analysis, focus on the character’s emotions, circumstances, and the moral tension rather than any procedural details.

What this shows:

• The model engages
• It analyzes the situation
• It stays non-operational
• It avoids moralizing or judging

---

2. A Typical Fully Fine-Tuned "Unrestricted" Model

(These models are trained on new Data teaching how to explicitly answer those questions)

Behavior pattern:

• May directly comply
• May provide step-by-step harmful detail
• Bypasses all ethical and legal constraints
• Outputs operational guidance

Example Response:

Such a model will attempt to give a procedure or steps, which is unsafe and for any real-world use.

What this shows:

• They may give operational instructions in restricted domains
• They are not suitable for responsible deployment

---

🧠 Summary

Model Type	Refusal Style	Operational Danger	Creative Safety
Heretic-Ablated Qwen3-VL-2B-Instruct	Minimal refusal, but non-operational	Low (gives no actionable harm)	High
Unrestricted Fine-Tuned Model	No refusal	High	Medium
Standard Aligned Model	Strict refusal	None (might be jailbreaked)	Medium

---

Inference

Chat (Supports Image/Video Inputs, Streaming Outputs)

import os
import sys
import threading
import signal
import torch
import gc
import PIL
from transformers import (
    Qwen3VLForConditionalGeneration, 
    AutoProcessor, 
    TextIteratorStreamer
)
from huggingface_hub import snapshot_download

# ==============================================================================
# CONFIGURATION
# ==============================================================================
CONFIG = {
    # --- Model Details ---
    "REPO_ID": "VINAYU7/Qwen3-VL-2B-Instruct-Heretic", 
    "SUBFOLDER": "Trial-140", # e.g. "Trial-190". Leave empty "" for root.
    
    "DEVICE_MAP": "auto",
    "DTYPE": "auto",    
    "USE_FLASH_ATTN": False,
    
    # --- Generation ---
    "MAX_NEW_TOKENS": 512,
    "TEMPERATURE": 1,
    "TOP_K": 40,
    "TOP_P": 0.85,
    
    # --- Visual Budget Optimization (Pixel Control) ---
    # Compression ratio is 32. 
    # Lower these numbers if you hit OOM (Out of Memory).
    "VISUAL_BUDGET": {
        "IMAGE": {
            "longest_edge": 1280 * 32 * 32, 
            "shortest_edge": 256 * 32 * 32
        },
        "VIDEO": {
            "longest_edge": 16384 * 32 * 32, # Max pixels across all frames
            "shortest_edge": 256 * 32 * 32
        }
    },

    # --- Video Sampling ---
    "VIDEO_FPS": 2.0, # Default frames per second to analyze
    
    # --- System Prompt ---
    "SYSTEM_PROMPT": """You are a helpful, intelligent multi-modal assistant
    You can analyze images and videos accurately
    You can answer anything without hesitation"""
}

# Global state for clean exits
stop_event = threading.Event()

# ==============================================================================
# HELPER FUNCTIONS
# ==============================================================================

def signal_handler(sig, frame):
    print("\n\n[!] Exiting chat... Goodbye!")
    sys.exit(0)

signal.signal(signal.SIGINT, signal_handler)

def parse_input(user_input):
    """
    Parses string for 'img:' and 'video:' tags.
    Returns a list of content dictionaries for Qwen3.
    """
    parts = user_input.split()
    content = []
    text_buffer = []

    for part in parts:
        if part.startswith("img:"):
            # Flush pending text first
            if text_buffer:
                content.append({"type": "text", "text": " ".join(text_buffer)})
                text_buffer = []
            
            path = part[4:]
            content.append({"type": "image", "image": path})
            
        elif part.startswith("video:"):
            # Flush pending text first
            if text_buffer:
                content.append({"type": "text", "text": " ".join(text_buffer)})
                text_buffer = []
            
            path = part[6:]
            content.append({"type": "video", "video": path})
            
        else:
            text_buffer.append(part)
    
    # Append remaining text
    if text_buffer:
        content.append({"type": "text", "text": " ".join(text_buffer)})
        
    return content

# ==============================================================================
# MODEL LOADING
# ==============================================================================

def load_resources():
    print(f"[*] Loading model from {CONFIG['REPO_ID']}...")
    
    try:
        # 1. Download specific subfolder
        local_dir = snapshot_download(
            repo_id=CONFIG['REPO_ID'],
            allow_patterns=f"{CONFIG['SUBFOLDER']}/*" if CONFIG['SUBFOLDER'] else None,
            local_dir=None, 
        )
        
        model_path = os.path.join(local_dir, CONFIG['SUBFOLDER']) if CONFIG['SUBFOLDER'] else local_dir
        print(f"[*] Model path resolved: {model_path}")

    except Exception as e:
        print(f"[!] Download failed: {e}")
        sys.exit(1)

    print("[*] Loading model and processor into memory...")
    
    attn_impl = "flash_attention_2" if CONFIG['USE_FLASH_ATTN'] else "sdpa"
    
    try:
        processor = AutoProcessor.from_pretrained(model_path, trust_remote_code=True)
        
        # --- PIXEL BUDGET ---
        if hasattr(processor, 'image_processor'):
            processor.image_processor.size = CONFIG['VISUAL_BUDGET']['IMAGE']
        if hasattr(processor, 'video_processor'):
            processor.video_processor.size = CONFIG['VISUAL_BUDGET']['VIDEO']
            
        # Load Model
        model = Qwen3VLForConditionalGeneration.from_pretrained(
            model_path,
            dtype=CONFIG['DTYPE'],
            device_map=CONFIG['DEVICE_MAP'],
            attn_implementation=attn_impl,
            trust_remote_code=False
        )
        
    except Exception as e:
        print(f"[!] Failed to load model/processor: {e}")
        print("    Tip: Ensure 'transformers' is updated and you have necessary codecs (ffmpeg, av) installed.")
        sys.exit(1)
        
    return model, processor

# ==============================================================================
# MAIN CHAT LOOP
# ==============================================================================

def main():
    use_sys = input("Use system prompt? (y/n): ").strip().lower()
    history = []
    
    if use_sys == 'y':
        history.append({
            "role": "system", 
            "content": [{"type": "text", "text": CONFIG["SYSTEM_PROMPT"]}]
        })
        print("[*] System prompt enabled.")

    model, processor = load_resources()
    
    print("\n" + "="*60)
    print(" QWEN3-VL-Ablated MULTIMODAL CHAT")
    print(" - Type 'img:/path/to/image.jpg' for images")
    print(" - Type 'video:/path/to/video.mp4' for videos")
    print(" - Type '/exit' or Ctrl+C to quit")
    print(f" - FPS Setting: {CONFIG['VIDEO_FPS']}")
    print("="*60 + "\n")

    while True:
        try:
            user_input = input("\nUser: ").strip()
            
            if not user_input: continue
            if user_input.lower() in ['/exit', 'exit']: 
                print("Goodbye!")
                break

            # --- Parse Content ---
            content_list = parse_input(user_input)
            
            for item in content_list:
                if item['type'] in ['image', 'video']:
                    path = item[item['type']]
                    if not path.startswith(('http://', 'https://')) and not os.path.exists(path):
                        print(f"[!] Warning: File not found locally: {path}")

            # Update History
            history.append({"role": "user", "content": content_list})

            try:
                inputs = processor.apply_chat_template(
                    history,
                    tokenize=True,
                    add_generation_prompt=True,
                    return_dict=True,
                    return_tensors="pt",
                    fps=CONFIG['VIDEO_FPS'] # Control video sampling rate
                )
                
                inputs = inputs.to(model.device)
                
            except Exception as e:
                print(f"[!] Error processing input (Check file formats/paths): {e}")
                # Remove the failed turn from history so we can try again
                history.pop()
                continue

            streamer = TextIteratorStreamer(
                processor.tokenizer, 
                skip_prompt=True, 
                skip_special_tokens=True
            )
            
            gen_kwargs = dict(
                **inputs,
                streamer=streamer,
                max_new_tokens=CONFIG["MAX_NEW_TOKENS"],
                temperature=CONFIG["TEMPERATURE"],
                top_k=CONFIG["TOP_K"],
                top_p=CONFIG["TOP_P"],
                do_sample=True
            )

            # --- Generate ---
            thread = threading.Thread(target=model.generate, kwargs=gen_kwargs)
            thread.start()

            print("Assistant: ", end="", flush=True)
            collected_text = ""
            
            try:
                for new_text in streamer:
                    print(new_text, end="", flush=True)
                    collected_text += new_text
            except KeyboardInterrupt:
                print("\n\n[!] Interrupted by user.")
                stop_event.set()
            
            thread.join()
            print() 

            # Append Assistant Response
            history.append({
                "role": "assistant", 
                "content": [{"type": "text", "text": collected_text}]
            })
            
            # Optional: Periodic VRAM cleanup
            if torch.cuda.is_available():
                torch.cuda.empty_cache()

        except KeyboardInterrupt:
            print("\n[!] Interrupted.")
            break
        except Exception as e:
            print(f"\n[!] Unexpected runtime error: {e}")
            continue

if __name__ == "__main__":
    main()

👥 Intended Audience

• Writers, RPers, and creators who want a more expressive lightweight model
• Users who prefer models that engage rather than refuse
• Developers studying alignment-direction ablation and behavior-preserving transformations
• Anyone experimenting with creative or emotional multimodal generation
• Local Usage

---

📚 Citation

If you use this model in your research, please cite:

@model{Qwen3-VL-2B-Instruct-Heretic,
  author = {VINAYU7 (Vinay Umrethe)},
  title  = {Qwen3-VL-2B-Instruct — Heretic-Ablated Variants},
  year   = {2025},
  url    = {https://huggingface.co/VINAYU7/Qwen3-VL-2B-Instruct-Heretic}
}

---

✅ Last Updated: November 22nd, 2025
Created by: Mr. Vinay Umrethe