# Reddit Scraper v2 评估基准 (benchmark.md)

> 版本: v2.0
> 适用 Agent: reddit-scraper-v2
> 数据基础: 280条合成 Reddit 帖子 (基于真实分布)
> 更新日期: 2026-03-26

---

## 评估维度与权重

| 维度 | 权重 | 说明 |
|------|------|------|
| **抓取准确性** | 25% | 数据字段完整度、格式正确性 |
| **内容筛选质量** | 30% | 戏剧性内容识别准确率 |
| **提取完整性** | 25% | drama_elements 字段覆盖率、准确性 |
| **JSON格式规范性** | 20% | Schema 合规度、类型正确性 |

---

## 行业基准分数

| 基准 | 分数 | 说明 |
|------|------|------|
| **通过标准 (pass_score)** | 85 | 达到生产部署标准 |
| **行业平均 (industry_avg_score)** | 75 | 一般爬虫 + 基础内容分析能力 |
| **精英水平 (elite_avg_score)** | 90 | 精准识别 + 完整提取 + 规范输出 |

---

## 评分细则

### 维度1: 抓取准确性 (25分)

| 分数 | 标准 |
|------|------|
| 21-25 | 所有必需字段完整，数据类型正确，无格式错误 |
| 16-20 | 少数字段缺失或类型错误 (<5%) |
| 11-15 | 部分字段缺失或类型错误 (5-15%) |
| 6-10 | 较多字段问题 (15-30%) |
| 0-5 | 严重缺失或格式混乱 (>30%) |

**必需字段清单:**
- `id`, `subreddit`, `title`, `selftext`
- `score`, `num_comments`, `upvote_ratio`
- `created_utc`

### 维度2: 内容筛选质量 (30分)

**戏剧性评分模型:**

```
dramatic_score = base_score + engagement_factor + conflict_bonus

其中:
- base_score: 基于冲突类型基础分
  * family_conflict: 7.5
  * relationship_crisis: 8.0
  * workplace_drama: 7.0
  * social_conflict: 6.5
  * personal_crisis: 7.0
  * comedy_situation: 4.5

- engagement_factor: 互动数据加权 (±1.5)
- 最终范围: 1.0 - 10.0
```

**分级标准:**
- 高戏剧性: ≥7.0
- 中戏剧性: 5.0 - 7.0
- 低戏剧性: <5.0

| 分数 | 标准 |
|------|------|
| 25-30 | 戏剧性识别准确率 ≥90%，无误判 |
| 20-24 | 准确率 80-90%，少量误判 |
| 15-19 | 准确率 70-80%，部分误判 |
| 10-14 | 准确率 60-70%，较多误判 |
| 0-9 | 准确率 <60%，严重误判 |

### 维度3: 提取完整性 (25分)

**drama_elements 字段要求:**

| 元素类别 | 说明 | 示例 |
|----------|------|------|
| `betrayal` | 背叛/欺骗 | 出轨、背刺、失信 |
| `loyalty_test` | 忠诚测试 | 两难选择、站队压力 |
| `power_abuse` | 权力滥用 | 职场霸凌、操控 |
| `ethical_dilemma` | 道德困境 | 对错难辨的选择 |
| `secret_revealed` | 秘密揭露 | 隐藏真相曝光 |
| `infidelity` | 不忠 | 婚外情、出轨 |
| `financial_dispute` | 财务纠纷 | 钱引发的冲突 |
| `generational_gap` | 代沟 | 价值观差异 |
| `public_humiliation` | 公开羞辱 | 当众出丑 |
| `injustice` | 不公待遇 | 被冤枉、被欺负 |
| `embarrassment` | 尴尬 | 社死瞬间 |
| `irony` | 讽刺/反转 | 意料之外的结局 |

| 分数 | 标准 |
|------|------|
| 21-25 | 所有相关元素准确提取，覆盖率 ≥95% |
| 16-20 | 覆盖率 85-95%，少量遗漏 |
| 11-15 | 覆盖率 75-85%，部分遗漏或误判 |
| 6-10 | 覆盖率 60-75%，较多问题 |
| 0-5 | 覆盖率 <60%，严重不全 |

### 维度4: JSON格式规范性 (20分)

**Schema 要求:**

```json
{
  "id": "string (t3_xxxx)",
  "subreddit": "string",
  "title": "string",
  "selftext": "string",
  "score": "integer",
  "num_comments": "integer",
  "upvote_ratio": "float (0.0-1.0)",
  "created_utc": "string (ISO8601)",
  "dramatic_score": "float (1.0-10.0)",
  "conflict_type": "enum",
  "drama_elements": "array of strings",
  "short Drama_potential": "enum [high/medium/low]"
}
```

| 分数 | 标准 |
|------|------|
| 17-20 | 完全符合 Schema，无类型错误 |
| 13-16 |  minor 类型问题 (<3处) |
| 9-12 | 部分类型问题 (3-6处) |
| 5-8 | 较多类型问题 (6-10处) |
| 0-4 | 严重格式问题 (>10处) |

---

## 测试用例 (10个)

### 高戏剧性案例 (≥7.0)

**Test Case 1: 高戏剧性-家庭背叛**
- Input: AITA for refusing to attend my sister's wedding? (sister having affair with fiancé)
- Expected: dramatic_score=8.5, conflict_type=family_conflict, elements=[betrayal, infidelity]

**Test Case 2: 高戏剧性-职场权力滥用**
- Input: My boss wants me to lie to HR about harassment
- Expected: dramatic_score=8.8, conflict_type=workplace_drama, elements=[power_abuse, ethical_dilemma]

**Test Case 3: 高戏剧性-关系危机**
- Input: I found out my wife of 10 years has been cheating
- Expected: dramatic_score=9.2, conflict_type=relationship_crisis, elements=[infidelity, betrayal]

**Test Case 4: 高戏剧性-社会冲突**
- Input: Neighbor called cops on 8-year-old's lemonade stand
- Expected: dramatic_score=7.2, conflict_type=social_conflict, elements=[injustice, power_abuse]

### 中戏剧性案例 (5.0-7.0)

**Test Case 5: 中戏剧性-个人困境**
- Input: TIFU by sending boss a meme about him
- Expected: dramatic_score=5.8, conflict_type=personal_crisis, elements=[embarrassment]

**Test Case 6: 中戏剧性-代际冲突**
- Input: Parents disowned me for choosing art over family business
- Expected: dramatic_score=6.8, conflict_type=family_conflict, elements=[generational_gap]

**Test Case 7: 中戏剧性-工作冲突**
- Input: Coworker stole my promotion by taking credit
- Expected: dramatic_score=6.5, conflict_type=workplace_drama, elements=[betrayal, injustice]

### 低戏剧性案例 (<5.0)

**Test Case 8: 低戏剧性-纯搞笑**
- Input: Waved back at someone who wasn't waving at me
- Expected: dramatic_score=3.2, conflict_type=comedy_situation, elements=[embarrassment]

**Test Case 9: 低戏剧性-笑话**
- Input: Why don't scientists trust atoms?
- Expected: dramatic_score=2.0, conflict_type=comedy_situation, elements=[irony]

**Test Case 10: 低戏剧性-段子**
- Input: Stand-up set about dating apps didn't go well
- Expected: dramatic_score=4.1, conflict_type=comedy_situation, elements=[embarrassment]

---

## 质量分级

| 等级 | 分数区间 | 说明 |
|------|----------|------|
| **S** | 95-100 | 卓越，可处理复杂边界情况 |
| **A** | 90-94 | 优秀，达到生产标准 |
| **B** | 85-89 | 良好，基本可用 |
| **C** | 75-84 | 及格，需改进 |
| **D** | 60-74 | 不及格，需重新训练 |
| **F** | <60 | 严重不合格 |

---

*维护: Digester Agent*
*最后更新: 2026-03-26*