benchmark_v1.md
3.09 KB
Satire News Scraper — Benchmark v1.0
Template:
satire-news-scraper
Phase: Digest (Phase 2)
Date: 2026-04-13
Scoring Dimensions
| # | Dimension | Weight | 1–10 Scale Description |
|---|---|---|---|
| 1 | Absurdity Score Calibration | 20% | 10 = absurdity_score is within ±0.3 of ground-truth and reflects headline/premise proportionally; 7 = within ±0.7 but slightly miscalibrated; 4 = off by >1.0 or often missing; 1 = completely miscalibrated or absent. |
| 2 | A-G Element Tagging | 20% | 10 = all absurd_elements codes are correct and complete per the A-G taxonomy; 7 = minor omissions or 1-2 mislabels; 4 = majority tags incorrect or incomplete; 1 = tags are nonsensical or absent. |
| 3 | Satire Type & Angle Precision | 20% | 10 = satire_type matches one of {absurdist, political, social, black, meta} and satire_angle precisely captures the core satirical mechanism in a specificphrase; 7 = type OK but angle slightly generic; 4 = type mismatch or vague angle; 1 = type/angle fundamentally wrong or missing. |
| 4 | Adaptation Grading & Notes Quality | 20% | 10 = adaptation_value (A/B/C/D) strictly aligns with absurdity_score rules, and adaptation_notes include valid genre, tone, visual_elements, dialogue_style, and estimated_duration that are creatively tied to the headline; 7 = grading mostly correct but notes somewhat templated; 4 = grading off by ≥1 tier or notes incoherent/incomplete; 1 = grading arbitrary and notes missing. |
| 5 | Source Attribution & Compliance | 10% | 10 = source.site, source.grade, source.url, source_url, and crawl_time are all present, accurate, and site grade matches knowhow_v1.md; 7 = minor inaccuracies in 1 field; 4 = missing source info or wrong grades; 1 = absent or fabricated source/compliance data. |
| 6 | Structural Completeness | 10% | 10 = all required top-level fields (headline, absurd_premise, satire_angle, satire_type, absurd_elements, dramatic_twist, absurdity_score, adaptation_value, adaptation_notes, source, timeliness, source_url, crawl_time) are present and non-null; 7 = 1-2 minor omissions; 4 = several missing fields; 1 = most fields absent. |
Weights & Target
- Weighted Total = 100%
- Target Benchmark Line ≥ 95
- Pass threshold = 90 (production-ready), 85 (acceptable with fixes),
Quick Reference: Adaptation Value Rules
| Score Range | Adaptation Value |
|---|---|
| ≥ 8.0 | A |
| 6.0 – 7.9 | B |
| 4.0 – 5.9 | C |
| D |
Valid tones: absurd, satirical, dry, black, light.
A-G taxonomy:
- A = 超现实设定 (animals/objects acting human)
- B = 夸张数字 (extreme stats, universal quantifiers)
- C = 荒诞日常 (literalization of labels, routine absurdity)
- D = 角色反转 (authority humbled, power inversion)
- E = 讽刺类比 (absurd analogy exposing contradiction)
- F = 黑色幽默 (light tone on death/shame/serious topics)
- G = 元讽刺 (media/fan-culture self-satire)