One pair where the labels and your definitions still disagree
The sixteen umbrella commitments carry definitions you confirmed. The pair-level gold set was labelled beforethose definitions existed, under a looser reading — “does this article bear on the commitment”. So when the verifier applies your definition and rejects a pair the old label calls valid, the disagreement can sit in three different places, and the score cannot tell them apart.
Previous round — decided and applied
10 pairs adjudicated: 7 definition too narrow, 3 stale label, 0 verifier error. The verifier was not misapplying the rules — the rules were too narrow. Five items were amended and three labels corrected, so those pairs no longer appear below.
Pairs still to adjudicate
1
Commitments involved
1
False positives already suppressed
16
The three answers
- Old label is wrong. The article genuinely does not concern this commitment. The verifier was right; the annotation is stale and is corrected. This pair leaves the valid set, which lowers the recall denominator.
- Verifier was wrong. It does match, and your definition already covers it — the verification stage misapplied the rule. A real recall loss that no re-wording fixes.
- Definition too narrow. It does match, but the confirmed definition does not admit it. This sends the item back to the decision sheet for new wording, so it is a change to the definition rather than to the pipeline.
Each pair below shows the article itself — its text and a link to the original, so you are reading the source rather than my summary — together with the commitment’s approved text, the definition the pair failed, and the other labelled pairs for the same commitment so the boundary is visible. Two valid pairs are deliberately absent: the matcher never proposed them, so no verifier rule can reach them, and they are a separate defect from anything decided here.
Showing the 10 adjudications on record; your edits layer on top. Press 1 2 3 to decide, ↑ ↓ to move. Export and hand back the file.
What your answers do
0 stale labels
Valid pairs fall 10 → 10; the verifier’s scoped recall on the corrected set is 8/10 = 80%.
0 verifier errors
Real losses the definition already covers. These need the verification stage fixed — no re-wording helps.
0 too narrow
Items to take back to the decision sheet and reword.
Not in question here: 16 false positives the verifier correctly suppressed, and 1 valid pairs the matcher never proposed at all — no verifier rule can reach those.
- #081Consolidating Scattered Public Funds for Investmentne undecided
The article
आर्थिक वर्ष २०८३/८४ बजेट : यथार्थ, चुनौती र आत्मनिर्भर अर्थतन्त्रको बाटो
Nagarik · 2026-05-30 · open original
यद्यपि बजेटको आकार, स्रोत र खर्चको संरचनालाई नजिकबाट विश्लेषण गर्दा नेपालको अर्थतन्त्र अझै पनि राजस्वभन्दा ऋणमा बढी निर्भर रहेको तथ्य स्पष्ट देखिन्छ।...
The confirmed definition this pair fails
- Counts only if it bears on:
- merging the 139+ scattered or duplicated public funds and investing them in high-return projects
- And NOT merely:
- general budget analysis, revenue collection, provincial budgets, development-spending commentary
Approved commitment text: To solve the problem of poor utilization of public resources due to more than 139 funds being scattered, duplicated, or operated less effectively, to prepare a framework within 60 days to merge funds of a similar nature and invest the money in high-return projects.
1 pairs across 1 commitments, derived from the cached measurement — regenerating costs no API calls.