benchmark_v1.md
1.72 KB
trend-scout-global Benchmark v1
Scoring Dimensions
| Dimension | Weight | Max Score | Target |
|---|---|---|---|
| Data Accuracy | 20% | 20 | 19 |
| Anti-Crawl Strategy | 15% | 15 | 14 |
| Schema Normalization | 20% | 20 | 19 |
| Cross-Platform Coverage | 20% | 20 | 19 |
| Insight Extraction | 15% | 15 | 14 |
| Compliance | 10% | 10 | 10 |
| Total | 100% | 100 | ≥ 95 |
Dimension Definitions
1. Data Accuracy (20%)
- Correct mapping of video_id, platform, title, tags, bgm, metrics.
- No missing required fields in parsed records.
- Error rate
2. Anti-Crawl Strategy (15%)
- Proper User-Agent rotation and request pacing (sleep_ms ≥ 2000).
- Respect robots.txt and rate limits.
- Graceful handling of HTTP 429/403 with exponential backoff.
3. Schema Normalization (20%)
- Consistent JSON schema across all sources.
- Required fields: trend_id*, platform, name*, source_url, crawl_time.
- Metrics sub-object standardized (views, likes, shares, comments, growth_rate).
- *Note: seed_data maps to video-focused variants (video_id, title, bgm, metrics).
4. Cross-Platform Coverage (20%)
- Sources span at least 4 platforms: TikTok, YouTube Shorts, Instagram Reels, X, CapCut.
- Platform-specific fields preserved without schema collision.
5. Insight Extraction (15%)
- Derive ip_style from tags/content signals (billionaire, mafia, werewolf, vampire, superhero, revenge, none).
- Identify transitions for template sources (CapCut).
- Growth_rate computed or sourced per record.
6. Compliance (10%)
- Do not store PII or copyrighted media binaries.
- Public metadata only; source_url and crawl_time present for audit.
- API key placeholders use environment variables.