热点研究员 Benchmark
测试数据集
Test Set 1: 基础抓取 (10条)
[
{"title": "Tech companies face new regulations", "source": "HN", "category": "tech"},
{"title": "World Cup qualifier results", "source": "BBC", "category": "sports"},
{"title": "Oil prices surge amid tensions", "source": "Al Jazeera", "category": "business"},
{"title": "Climate summit reaches agreement", "source": "BBC", "category": "world"},
{"title": "Celebrity news trending", "source": "Various", "category": "entertainment"},
{"title": "Military developments in conflict zone", "source": "Al Jazeera", "category": "military"},
{"title": "Stock market update", "source": "BBC", "category": "business"},
{"title": "Entertainment award ceremony", "source": "Various", "category": "entertainment"},
{"title": "Technology breakthrough announced", "source": "HN", "category": "tech"},
{"title": "International trade negotiations", "source": "Al Jazeera", "category": "world"}
]
Test Set 2: 深度研究 (5条)
[
{
"topic": "Major international conflict",
"required_fields": ["background", "key_figures", "story_arc"]
},
{
"topic": "Celebrity scandal",
"required_fields": ["background", "key_figures", "dialogue_samples"]
},
{
"topic": "Sports championship",
"required_fields": ["background", "key_figures", "story_arc"]
},
{
"topic": "Business merger",
"required_fields": ["background", "key_figures", "adaptation_notes"]
},
{
"topic": "Tech innovation",
"required_fields": ["background", "key_figures", "story_arc"]
}
]
评分维度
1. 抓取完整性 (40分)
| 指标 |
得分 |
| 标题完整 |
10 |
| 来源标注 |
5 |
| 分类准确 |
10 |
| 时间戳 |
5 |
| 热度评分 |
10 |
2. 数据质量 (30分)
| 指标 |
得分 |
| 无重复 |
10 |
| 无截断 |
10 |
| 编码正确 |
5 |
| 格式规范 |
5 |
3. 研究深度 (20分)
| 指标 |
得分 |
| 背景调查 |
8 |
| 人物分析 |
6 |
| 剧情建议 |
6 |
4. 效率 (10分)
合格标准
测试命令
# 抓取测试
python -c "
from skills.hot_topic_researcher import HotTopicResearcher
researcher = HotTopicResearcher()
results = researcher.fetch_all()
score = evaluate(results)
print(f'Score: {score}')
"
# 研究测试
python -c "
from skills.hot_topic_researcher import DeepResearcher
researcher = DeepResearcher()
report = researcher.investigate('topic_name')
score = evaluate_report(report)
print(f'Score: {score}')
"