benchmark_v1.md 3.09 KB

Satire News Scraper — Benchmark v1.0

Template: satire-news-scraper
Phase: Digest (Phase 2)
Date: 2026-04-13


Scoring Dimensions

# Dimension Weight 1–10 Scale Description
1 Absurdity Score Calibration 20% 10 = absurdity_score is within ±0.3 of ground-truth and reflects headline/premise proportionally; 7 = within ±0.7 but slightly miscalibrated; 4 = off by >1.0 or often missing; 1 = completely miscalibrated or absent.
2 A-G Element Tagging 20% 10 = all absurd_elements codes are correct and complete per the A-G taxonomy; 7 = minor omissions or 1-2 mislabels; 4 = majority tags incorrect or incomplete; 1 = tags are nonsensical or absent.
3 Satire Type & Angle Precision 20% 10 = satire_type matches one of {absurdist, political, social, black, meta} and satire_angle precisely captures the core satirical mechanism in a specificphrase; 7 = type OK but angle slightly generic; 4 = type mismatch or vague angle; 1 = type/angle fundamentally wrong or missing.
4 Adaptation Grading & Notes Quality 20% 10 = adaptation_value (A/B/C/D) strictly aligns with absurdity_score rules, and adaptation_notes include valid genre, tone, visual_elements, dialogue_style, and estimated_duration that are creatively tied to the headline; 7 = grading mostly correct but notes somewhat templated; 4 = grading off by ≥1 tier or notes incoherent/incomplete; 1 = grading arbitrary and notes missing.
5 Source Attribution & Compliance 10% 10 = source.site, source.grade, source.url, source_url, and crawl_time are all present, accurate, and site grade matches knowhow_v1.md; 7 = minor inaccuracies in 1 field; 4 = missing source info or wrong grades; 1 = absent or fabricated source/compliance data.
6 Structural Completeness 10% 10 = all required top-level fields (headline, absurd_premise, satire_angle, satire_type, absurd_elements, dramatic_twist, absurdity_score, adaptation_value, adaptation_notes, source, timeliness, source_url, crawl_time) are present and non-null; 7 = 1-2 minor omissions; 4 = several missing fields; 1 = most fields absent.

Weights & Target

  • Weighted Total = 100%
  • Target Benchmark Line ≥ 95
  • Pass threshold = 90 (production-ready), 85 (acceptable with fixes),

Quick Reference: Adaptation Value Rules

Score Range Adaptation Value
≥ 8.0 A
6.0 – 7.9 B
4.0 – 5.9 C
D

Valid tones: absurd, satirical, dry, black, light.

A-G taxonomy:

  • A = 超现实设定 (animals/objects acting human)
  • B = 夸张数字 (extreme stats, universal quantifiers)
  • C = 荒诞日常 (literalization of labels, routine absurdity)
  • D = 角色反转 (authority humbled, power inversion)
  • E = 讽刺类比 (absurd analogy exposing contradiction)
  • F = 黑色幽默 (light tone on death/shame/serious topics)
  • G = 元讽刺 (media/fan-culture self-satire)