benchmark_v1.md 1.72 KB

trend-scout-global Benchmark v1

Scoring Dimensions

Dimension Weight Max Score Target
Data Accuracy 20% 20 19
Anti-Crawl Strategy 15% 15 14
Schema Normalization 20% 20 19
Cross-Platform Coverage 20% 20 19
Insight Extraction 15% 15 14
Compliance 10% 10 10
Total 100% 100 ≥ 95

Dimension Definitions

1. Data Accuracy (20%)

  • Correct mapping of video_id, platform, title, tags, bgm, metrics.
  • No missing required fields in parsed records.
  • Error rate

2. Anti-Crawl Strategy (15%)

  • Proper User-Agent rotation and request pacing (sleep_ms ≥ 2000).
  • Respect robots.txt and rate limits.
  • Graceful handling of HTTP 429/403 with exponential backoff.

3. Schema Normalization (20%)

  • Consistent JSON schema across all sources.
  • Required fields: trend_id*, platform, name*, source_url, crawl_time.
  • Metrics sub-object standardized (views, likes, shares, comments, growth_rate).
  • *Note: seed_data maps to video-focused variants (video_id, title, bgm, metrics).

4. Cross-Platform Coverage (20%)

  • Sources span at least 4 platforms: TikTok, YouTube Shorts, Instagram Reels, X, CapCut.
  • Platform-specific fields preserved without schema collision.

5. Insight Extraction (15%)

  • Derive ip_style from tags/content signals (billionaire, mafia, werewolf, vampire, superhero, revenge, none).
  • Identify transitions for template sources (CapCut).
  • Growth_rate computed or sourced per record.

6. Compliance (10%)

  • Do not store PII or copyrighted media binaries.
  • Public metadata only; source_url and crawl_time present for audit.
  • API key placeholders use environment variables.