kofdai/Verantyx-hle-4.6

Verantyx HLE — 4.6%

Fully LLM-free symbolic solver for Humanity's Last Exam (HLE) — no neural networks, no language models, pure rule-based reasoning with Wikipedia as the only knowledge source.

Score

Split	Score	Method
Full 2500 questions	115/2500 = 4.6%	atom_cross + knowledge_match + cross_decompose

Approach

Verantyx solves HLE through structural decomposition:

1. Atom Extraction — Break questions and choices into atomic facts using 200+ regex patterns
2. Wikipedia Knowledge — Fetch relevant articles as the sole knowledge source
3. Cross-Decompose — Decompose each MCQ choice individually, cross-match against Wikipedia facts
4. Atom Relation Classification — LLM-free supports/contradicts/unknown classifier (60+ antonym pairs, negation detection, numeric cross-check)
5. MCQ全問回答 (Always Answer) — HLE has no wrong-answer penalty; fallback uses best keyword overlap

Pipeline

Question → Fact Atomizer → Wikipedia Fetch → Atom Cross Solver
                                              ↓
                                    Choice Scoring (supports/contradicts)
                                              ↓
                                    Best Choice or Keyword Fallback

Solver Components

Component	Fires	Description
cross_decompose	122	Per-choice decomposition + Wikipedia cross-match
knowledge_match	18	Direct atom-based knowledge matching
atom_cross	fallback	Normalized atom scoring with Wikipedia overlap

Properties

• ✅ No LLM — zero language model inference (Qwen 7B fully removed)
• ✅ No neural network — pure rule-based symbolic reasoning
• ✅ No pattern detectors — DISABLE_PATTERN_DETECTORS=1
• ✅ No concept boost — DISABLE_CONCEPT_BOOST=1
• ✅ No wrong-answer penalty exploitation — MCQ全問回答 is valid since HLE scoring has no penalty
• ✅ Wikipedia-only knowledge — no pre-trained embeddings or cached answers
• ✅ Deterministic — same input always produces same output

Score History

Version	Score	Method
v1 (with LLM)	2.68%	mcq_direct (Qwen 7B) + cross_decompose
v2 (LLM-free partial)	1.22%	Early LLM removal, limited coverage
v4 (LLM-free full)	4.6%	atom_cross + MCQ全問回答 + normalized scoring

Stats

Total: 2500 questions
Correct: 115 (4.6%)
Time: 98 minutes (4 parallel workers)
Wiki hits: 2298
Knowledge match: 18
Cross decompose: 122 fired

Links

• GitHub: Ag3497120/verantyx-v6
• ARC-AGI-2 Solver: kofdai/Verantyx-arc-agi2-7.4 (same philosophy)
• HLE Benchmark: cais/hle