search-reranker-broad-policy-v1

Policy-tuned derivative of temsa/search-reranker-broad-v1 for Irish government search.

This release targets the product policy gap identified in live usage:

• explicit news or year intent should favor the relevant latest or target-year news item
• current-status queries should prefer service or guide pages over news
• newline-separated multi-turn chat history should weight the latest turn most heavily
• optional Published: and Category: metadata should help ranking when present, but not be required

Training data

• preservation mix from the existing broad reranker line
• policy dataset: temsa/reranker-broad-policy-v2

The policy dataset includes happy-path, sad-path, and red-team grouped rows in English and Irish Gaelic.

Recommended deployment

• backend: ONNX q8
• max_length: 224
• threads: 8

Key q8 results

Suite	This model	search-reranker-broad-v1	l5-k10-v2	l6-fast-v1
Policy all MRR@10	0.9205	0.7305	0.7936	0.8035
Policy red-team MRR@10	0.8530	0.6941	0.6728	0.6845
Office bilingual MRR@10	0.9537	0.7963	n/a	n/a
Finephrase gov MRR@10	1.0000	1.0000	n/a	n/a
Hard K10 MRR@10	0.9815	1.0000	n/a	n/a
Holdout A04 MRR@10	0.9049	0.9211	n/a	n/a

CPU runtime

Proxy K10 runtime, q8, threads=8, max_length=224:

• this model: 15.22 qps
• broad-v1 baseline: 15.15 qps

So this release is primarily a quality upgrade on the identified policy issues, with no material runtime penalty on the same CPU path.

Files

• raw checkpoint in the root of the repo
• q8 ONNX artifact in onnx/
• benchmark summary in eval/benchmark_summary.json

Notes

• This is not a general replacement for every reranker line in the repository.
• It is the broad-policy derivative intended for cases where the broad reranker family is still the right serving choice.
• The smaller l5-k10 and l6-fast lines remain better speed-first options for stricter K=10 deployments.

Portfolio comparison

Updated 2026-03-19 from local reranker reports only.

Use this section as the side-by-side public temsa reranker view. Cells are intentionally left out or summarized when the local report set does not contain a trustworthy non-quantized benchmark for the same public path.

General bilingual rerankers

Repo	Primary role	Non-quantized path	Quantized path	Extra trustworthy signals
temsa/search-reranker-broad-v1	Broad final-stage reranker	ONNX fp32 l256: proxy 0.9490 / 2.08 qps	Sibling q8 l160: proxy 0.9458 / 8.41 qps	office 0.8056; hard-k10 0.9815; holdout-a03 0.8759
temsa/search-reranker-broad-v1-qint8	Broad CPU q8 sibling	See temsa/search-reranker-broad-v1	ONNX q8 l160: proxy 0.9458 / 8.41 qps	office 0.8056; hard-k10 0.9815; holdout-a03 0.8759
temsa/search-reranker-irishgov-l6-fast-v1	Fast stage-1 / cascade prefilter	PT l192: proxy 0.9192 / 7.99 qps	ONNX q8 l128: proxy 0.9218 / 27.25 qps	office 1.0000; holdout-a03 0.9259
temsa/search-reranker-irishgov-l5-k10-v1	Fast K=10 direct reranker	PT l160: proxy 0.8800 / 1.32 qps	ONNX q8 l140: proxy 0.8872 / 28.72 qps	office 1.0000; hard-k10 0.9815; holdout-a03 0.9630
temsa/search-reranker-irishgov-l5-k10-v2	Current K=10 successor	PT l128: office 1.0000, finephrase 0.9405	ONNX q8 l140: proxy 0.8853 / 28.27 qps	office 1.0000; holdout-a03 1.0000

Policy rerankers

Repo	Primary role	Non-quantized path	Quantized path	Extra trustworthy signals
temsa/search-reranker-broad-policy-v1	Broad policy-tuned reranker	PT l224: policy-all 0.9270 / 7.94 qps	ONNX q8 l224: policy-all 0.9205 / 27.55 qps	office 0.9537; holdout-a04 0.9049
temsa/search-reranker-broad-policy-v3	Current broad policy successor	ONNX fp32 l224: policy-all 0.9259 / 15.75 qps	ONNX q8 reduce_range l224: policy-all 0.9268 / 30.12 qps	office 0.9676; holdout-v3 0.9286
temsa/search-reranker-broad-policy-v4	Current broad policy serving release	Same raw checkpoint family as v3; q8 gov_broad_v1 is the recommended path	ONNX q8 reduce_range + gov_broad_v1 l224: policy-all 0.9257 / 31.64 qps	office 0.9676; holdout-v3 0.9271; holdout-v4 0.9583

Intentional gaps:

• search-reranker-broad-v1: the local reports include strong fp32 ONNX proxy and office data, but not a matching fp32 in-domain finephrase / hard-K10 / holdout-A03 set, so those are not claimed here.
• search-reranker-irishgov-l5-k10-v2: the local non-quantized reports are trustworthy for office / finephrase / hard-K10, but not for the same proxy runtime shape as the shipped q8 path.
• search-reranker-irishgov-l6-fast-v1: the local non-quantized reports cover proxy and office, but not the fresh holdout-A03 slice used for the q8 K=10 comparison.