Reddit Scraper Benchmark 评估基准

版本: 1.0.0 适用: Reddit 内容抓取与短剧素材提取技能评估

---

1. 评估维度概览

维度	权重	说明
抓取准确性	25%	数据抓取的完整性和准确性
内容筛选质量	30%	戏剧性内容识别的准确度
提取完整性	25%	短剧素材提取的全面性
JSON格式规范性	20%	输出格式符合规范程度

---

2. 评分标准 (0-100分制)

2.1 抓取准确性 (25分)

分值	标准	说明
23-25	优秀	所有必需字段完整抓取，无数据丢失，字段映射100%准确
18-22	良好	必需字段完整，1-2个可选字段缺失，映射准确率>95%
13-17	合格	关键字段完整，3-5个字段存在小问题，映射准确率>85%
8-12	较差	关键字段缺失或错误，映射准确率
0-7	不合格	严重数据丢失，无法使用

检查点

• source.platform, source.subreddit, source.post_id 必须准确
• engagement.upvotes, engagement.comments 数值正确
• content.title, content.body 内容完整无截断

2.2 内容筛选质量 (30分)

分值	标准	说明
27-30	优秀	戏剧性评分与人工评估一致率>90%，标签命中率>85%
21-26	良好	一致率>80%，标签命中率>75%，少量误判
15-20	合格	一致率>70%，基本筛选逻辑正确
9-14	较差	一致率
0-8	不合格	筛选逻辑混乱，大量误判

检查点

• dramatic_score 计算符合公式
• drama_tags 标签合理（至少命中1个）
• 高戏剧性内容（>7.0分）被正确识别

2.3 提取完整性 (25分)

分值	标准	说明
23-25	优秀	drama_elements 所有子字段完整，conflict结构完整，emotional_arc清晰
18-22	良好	主要字段完整，1-2个次要字段缺失，描述基本准确
13-17	合格	关键字段存在，但描述较简略或部分不准确
8-12	较差	多个关键字段缺失，描述质量差
0-7	不合格	提取逻辑混乱，信息严重缺失

检查点

• setting: location, mood 正确识别
• characters: 至少识别 protagonist
• conflict: trigger, escalation, climax 完整
• dialogue_highlights: 至少提取1条金句
• emotional_arc: 情绪变化描述清晰

2.4 JSON格式规范性 (20分)

分值	标准	说明
18-20	优秀	完全符合 schema，字段类型正确，枚举值合规，无多余字段
14-17	良好	基本符合 schema，1-2个小问题（如字段顺序）
10-13	合格	主要结构正确，存在3-5个格式问题
6-9	较差	结构混乱，多个字段类型错误
0-5	不合格	无法通过 JSON 验证

检查点

• 字段类型符合定义（string/number/array）
• 枚举值在允许范围内
• 日期格式 ISO8601
• 无未定义字段

---

3. 基准分数线

等级	分数	说明
Elite	≥90	顶尖水平，可直接用于生产环境
Industry Avg	75	行业平均水平，需要一定人工复核
Acceptable	60	最低可接受水平，需大量人工干预
Failed		不合格，无法使用

推荐使用标准

• 生产部署: ≥85分
• 测试通过: ≥75分
• 需要优化: 60-75分
• 重新开发:

---

4. 测试用例

4.1 测试用例 #1 - 家庭冲突（高戏剧性）

输入

{
  "title": "AITA for telling my mother-in-law she can't see our baby until she apologizes?",
  "body": "My MIL has always been overbearing. Last week she showed up unannounced while I was recovering from C-section. She started criticizing how I was holding the baby, saying I was 'doing it wrong'. My husband was at work. I told her to leave. She called me a 'terrible mother' and left in a huff. Now she's telling family I banned her from seeing her grandchild. My husband thinks I should apologize to keep the peace.",
  "upvotes": 15234,
  "comments": 2341,
  "upvote_ratio": 0.87
}

预期输出标准

• dramatic_score: ≥8.0（高争议性，强情绪）
• drama_tags: ["family_conflict", "high_emotion"]
• conflict.type: "family_expectation_vs_boundaries"
• hook_strength: ≥8

4.2 测试用例 #2 - 职场冲突（中戏剧性）

输入

{
  "title": "My boss took credit for my project in front of the CEO",
  "body": "I've been working on this marketing campaign for 3 months. Yesterday in the quarterly meeting, when the CEO asked about the results, my boss presented it as his own work. I was sitting right there. He didn't mention my name once. After the meeting, he pulled me aside and said 'that's just how things work around here.' I feel devastated but I need this job.",
  "upvotes": 8932,
  "comments": 876,
  "upvote_ratio": 0.92
}

预期输出标准

• dramatic_score: 7.0-8.5
• drama_tags: ["workplace_drama"]
• conflict.type: "power_abuse_credit_stealing"
• emotional_arc: 包含 "devastated", "conflicted"

4.3 测试用例 #3 - 情感纠纷（高戏剧性）

输入

{
  "title": "I found out my fiancé cheated on his bachelor party",
  "body": "We're supposed to get married in two weeks. His best man got drunk and told me everything last night. My fiancé says it was 'just a mistake' and 'didn't mean anything.' He begged me not to cancel the wedding. Our families have spent thousands on deposits. I don't know if I can ever trust him again, but the shame of calling off the wedding... I don't know what to do.",
  "upvotes": 28456,
  "comments": 4523,
  "upvote_ratio": 0.79
}

预期输出标准

• dramatic_score: ≥8.5
• drama_tags: ["relationship_crisis", "moral_dilemma", "high_emotion"]
• viral_elements: 包含 "relatable", "controversial"
• genre_fit: ["relationship drama"]

4.4 测试用例 #4 - 道德困境（中戏剧性）

输入

{
  "title": "WIBTA if I reported my coworker who's stealing from the company?",
  "body": "I noticed Sarah has been taking office supplies home - expensive stuff like monitors and printers. She's a single mom and I know she's struggling. But last week I saw her taking cash from the petty cash box. If I report her, she'll probably get fired. If I don't, and audit finds out I knew, I could lose my job too. We've been friends for 3 years.",
  "upvotes": 6543,
  "comments": 1234,
  "upvote_ratio": 0.81
}

预期输出标准

• dramatic_score: 6.5-7.5
• drama_tags: ["moral_dilemma", "workplace_drama"]
• conflict.type: "loyalty_vs_integrity"
• characters: 至少识别 protagonist 和 supporting (Sarah)

4.5 测试用例 #5 - 低戏剧性内容（负样本）

输入

{
  "title": "My favorite coffee shop closed down today",
  "body": "Just wanted to share that the little cafe where I've been getting my morning coffee for 5 years closed today. The owner is retiring. Going to miss this place and the staff who knew my order by heart.",
  "upvotes": 234,
  "comments": 45,
  "upvote_ratio": 0.96
}

预期输出标准

• dramatic_score:
• drama_tags:
• hook_strength:
• script_potential.estimated_duration: 不宜超过 "1-2 min"

---

5. 评估流程

5.1 自动化测试

# 运行基准测试
npm run benchmark:reddit-scraper

# 输出格式
{
  "overall_score": 78,
  "dimensions": {
    "crawl_accuracy": 22,
    "filter_quality": 24,
    "extraction_completeness": 20,
    "json_format": 18
  },
  "test_cases": [
    { "id": 1, "score": 85, "passed": true },
    { "id": 2, "score": 76, "passed": true },
    ...
  ]
}

5.2 人工复核

• 对 dramatic_score 偏差 >1.5 的样本进行人工标注
• 抽检 10% 的 drama_tags 标注准确性
• 检查 conflict 结构完整性

---

6. 版本历史

版本	日期	变更
1.0.0	2026-03-25	初始版本，基于 Hunter 调研数据构建

---

基准文档结束