Reddit Scraper v2 评估基准 (benchmark.md)

版本: v2.0 适用 Agent: reddit-scraper-v2 数据基础: 280条合成 Reddit 帖子 (基于真实分布) 更新日期: 2026-03-26

---

评估维度与权重

维度	权重	说明
抓取准确性	25%	数据字段完整度、格式正确性
内容筛选质量	30%	戏剧性内容识别准确率
提取完整性	25%	drama_elements 字段覆盖率、准确性
JSON格式规范性	20%	Schema 合规度、类型正确性

---

行业基准分数

基准	分数	说明
通过标准 (pass_score)	85	达到生产部署标准
行业平均 (industry_avg_score)	75	一般爬虫 + 基础内容分析能力
精英水平 (elite_avg_score)	90	精准识别 + 完整提取 + 规范输出

---

评分细则

维度1: 抓取准确性 (25分)

分数	标准
21-25	所有必需字段完整，数据类型正确，无格式错误
16-20	少数字段缺失或类型错误 (
11-15	部分字段缺失或类型错误 (5-15%)
6-10	较多字段问题 (15-30%)
0-5	严重缺失或格式混乱 (>30%)

必需字段清单:

• id, subreddit, title, selftext
• score, num_comments, upvote_ratio
• created_utc

维度2: 内容筛选质量 (30分)

戏剧性评分模型:

dramatic_score = base_score + engagement_factor + conflict_bonus

其中:
- base_score: 基于冲突类型基础分
  * family_conflict: 7.5
  * relationship_crisis: 8.0
  * workplace_drama: 7.0
  * social_conflict: 6.5
  * personal_crisis: 7.0
  * comedy_situation: 4.5

- engagement_factor: 互动数据加权 (±1.5)
- 最终范围: 1.0 - 10.0

分级标准:

• 高戏剧性: ≥7.0
• 中戏剧性: 5.0 - 7.0
• 低戏剧性:

分数	标准
25-30	戏剧性识别准确率 ≥90%，无误判
20-24	准确率 80-90%，少量误判
15-19	准确率 70-80%，部分误判
10-14	准确率 60-70%，较多误判
0-9	准确率

维度3: 提取完整性 (25分)

drama_elements 字段要求:

元素类别	说明	示例
betrayal	背叛/欺骗	出轨、背刺、失信
loyalty_test	忠诚测试	两难选择、站队压力
power_abuse	权力滥用	职场霸凌、操控
ethical_dilemma	道德困境	对错难辨的选择
secret_revealed	秘密揭露	隐藏真相曝光
infidelity	不忠	婚外情、出轨
financial_dispute	财务纠纷	钱引发的冲突
generational_gap	代沟	价值观差异
public_humiliation	公开羞辱	当众出丑
injustice	不公待遇	被冤枉、被欺负
embarrassment	尴尬	社死瞬间
irony	讽刺/反转	意料之外的结局

分数	标准
21-25	所有相关元素准确提取，覆盖率 ≥95%
16-20	覆盖率 85-95%，少量遗漏
11-15	覆盖率 75-85%，部分遗漏或误判
6-10	覆盖率 60-75%，较多问题
0-5	覆盖率

维度4: JSON格式规范性 (20分)

Schema 要求:

{
  "id": "string (t3_xxxx)",
  "subreddit": "string",
  "title": "string",
  "selftext": "string",
  "score": "integer",
  "num_comments": "integer",
  "upvote_ratio": "float (0.0-1.0)",
  "created_utc": "string (ISO8601)",
  "dramatic_score": "float (1.0-10.0)",
  "conflict_type": "enum",
  "drama_elements": "array of strings",
  "short Drama_potential": "enum [high/medium/low]"
}

分数	标准
17-20	完全符合 Schema，无类型错误
13-16	minor 类型问题 (
9-12	部分类型问题 (3-6处)
5-8	较多类型问题 (6-10处)
0-4	严重格式问题 (>10处)

---

测试用例 (10个)

高戏剧性案例 (≥7.0)

Test Case 1: 高戏剧性-家庭背叛

• Input: AITA for refusing to attend my sister's wedding? (sister having affair with fiancé)
• Expected: dramatic_score=8.5, conflict_type=family_conflict, elements=[betrayal, infidelity]

Test Case 2: 高戏剧性-职场权力滥用

• Input: My boss wants me to lie to HR about harassment
• Expected: dramatic_score=8.8, conflict_type=workplace_drama, elements=[power_abuse, ethical_dilemma]

Test Case 3: 高戏剧性-关系危机

• Input: I found out my wife of 10 years has been cheating
• Expected: dramatic_score=9.2, conflict_type=relationship_crisis, elements=[infidelity, betrayal]

Test Case 4: 高戏剧性-社会冲突

• Input: Neighbor called cops on 8-year-old's lemonade stand
• Expected: dramatic_score=7.2, conflict_type=social_conflict, elements=[injustice, power_abuse]

中戏剧性案例 (5.0-7.0)

Test Case 5: 中戏剧性-个人困境

• Input: TIFU by sending boss a meme about him
• Expected: dramatic_score=5.8, conflict_type=personal_crisis, elements=[embarrassment]

Test Case 6: 中戏剧性-代际冲突

• Input: Parents disowned me for choosing art over family business
• Expected: dramatic_score=6.8, conflict_type=family_conflict, elements=[generational_gap]

Test Case 7: 中戏剧性-工作冲突

• Input: Coworker stole my promotion by taking credit
• Expected: dramatic_score=6.5, conflict_type=workplace_drama, elements=[betrayal, injustice]

低戏剧性案例 (

Test Case 8: 低戏剧性-纯搞笑

• Input: Waved back at someone who wasn't waving at me
• Expected: dramatic_score=3.2, conflict_type=comedy_situation, elements=[embarrassment]

Test Case 9: 低戏剧性-笑话

• Input: Why don't scientists trust atoms?
• Expected: dramatic_score=2.0, conflict_type=comedy_situation, elements=[irony]

Test Case 10: 低戏剧性-段子

• Input: Stand-up set about dating apps didn't go well
• Expected: dramatic_score=4.1, conflict_type=comedy_situation, elements=[embarrassment]

---

质量分级

等级	分数区间	说明
S	95-100	卓越，可处理复杂边界情况
A	90-94	优秀，达到生产标准
B	85-89	良好，基本可用
C	75-84	及格，需改进
D	60-74	不及格，需重新训练
F		严重不合格

---

维护: Digester Agent 最后更新: 2026-03-26