宜博 / lemonboss

SKILL.md 21.1 KB

name: innovator-skill description: > A self-learning, self-iterating skill factory powered by the Hunt→Digest→Evolving (FDE) philosophy. Use this skill whenever a user presents ANY task or question that requires producing high-quality, domain-specific output — writing, research, analysis, code, strategy, etc. — especially when no existing skill is available. The innovator-skill will automatically find or create a purpose-built skill, ground it in real-world industry data, benchmark it against human/industry averages, and iteratively evolve it until it reaches or exceeds that bar. Trigger this skill when the user says things like "help me write X", "create a Y", "analyze Z", "I need a skill for...", or any open-ended

domain task. This is the meta-skill that produces all other skills.

Innovator-Skill

Core Philosophy: Every user question deserves a skill that is grounded in real-world knowledge, measured against industry standards, and continuously evolved until it surpasses human average performance. This is achieved through the Hunt → Digest → Evolving (FDE) three-layer architecture, operating two nested feedback loops: a small loop (Skill Iteration) and a large loop (Benchmark Iteration).

Architecture Overview

[User Question]
      │
      ▼
┌─────────────────────────────────────────────────────────────┐
│  Step 0: ROUTE — Search skill_list.md for existing skill    │
│   ├── Exact/Semantic match found (status: enable) → USE IT  │
│   ├── Match found (status: disable/training) → NOTIFY USER  │
│   └── No match → CREATE NEW SKILL (Steps 1–3)               │
└─────────────────────────────────────────────────────────────┘
      │
      ▼
┌─────────────── LAYER 1: HUNT ───────────────────────────────┐
│  Step 1.1: Deep Research — knowhow, methodology, standards  │
│  Step 1.2: Data Source Discovery — APIs, RSS, rankings, DBs │
│  Step 1.3: Crawler Setup — scripts, schedule, storage       │
│  Step 1.4: First Crawl Run — seed dataset                   │
└─────────────────────────────────────────────────────────────┘
      │
      ▼
┌─────────────── LAYER 2: DIGEST ─────────────────────────────┐
│  Step 2.1: Build knowhow.md — methods, frameworks, criteria │
│  Step 2.2: Build benchmark.md — scoring rubric              │
│  Step 2.3: Label datasets — train set / test set split      │
│  Step 2.4: Establish industry baseline score                │
│  Step 2.5: Write SKILL.md prompt (v1.0.0)                   │
└─────────────────────────────────────────────────────────────┘
      │
      ▼
┌─────────────── LAYER 3: EVOLVING ───────────────────────────┐
│                                                             │
│  ┌──────────────── SMALL LOOP (Skill Iteration) ─────────┐  │
│  │  Run skill on train data → benchmark score            │  │
│  │  Score < 90 → analyze weak dims → fix prompt          │  │
│  │  Repeat until score ≥ 90 OR max iterations            │  │
│  └───────────────────────────────────────────────────────┘  │
│                     ↓ score ≥ 90                            │
│            [SKILL ENABLED → Deliver to Human]               │
│                     ↓ human feedback received               │
│  ┌──────────────── LARGE LOOP (Benchmark Iteration) ─────┐  │
│  │  Feedback = benchmark is wrong/incomplete             │  │
│  │  Re-Hunt + Re-Digest → new benchmark.md version       │  │
│  │  Re-trigger Small Loop with updated benchmark         │  │
│  └───────────────────────────────────────────────────────┘  │
└─────────────────────────────────────────────────────────────┘

Directory Structure (per skill)

/innovator/
├── SKILL.md                          ← this file (meta-skill)
├── skill_list.md                     ← registry of all created skills
│
└── {skill_name}/                     ← e.g., xianxia_novel_skill/
    ├── SKILL.md                      ← the generated skill's prompt & rules
    ├── knowhow.md                    ← domain knowledge, methodology
    ├── benchmark.md                  ← scoring rubric (versioned)
    ├── skill_prompt_v{X}.md          ← versioned prompt snapshots
    │
    ├── crawl/
    │   ├── crawl_config.md           ← data sources, frequencies, priority
    │   ├── crawl_schedule.md         ← cron expressions, deployment guide
    │   ├── crawl_runner.py           ← executable crawler script
    │   ├── crawl_log.md              ← success/failure/skip records
    │   └── {source_name}/
    │       └── {YYYYMMDD}_{source}_{title}.md
    │
    ├── datasets/
    │   ├── train/                    ← 70% of labeled data
    │   ├── test/                     ← 30% of labeled data
    │   └── experience/               ← human feedback records
    │
    ├── benchmark_scores/
    │   └── score_{YYYYMMDD_HHMMSS}.md ← each scoring run result
    │
    └── evolving/
        ├── evolving_log.md           ← iteration history & reasoning
        └── v{X}/
            ├── output_{task}_{time}.md
            └── score_{task}_{time}.md

Step 0: Route — Skill Discovery

Before doing anything else, read /innovator/skill_list.md.

Matching logic (three tiers):

  1. Exact match (skill_name or keywords match exactly, status: enable) → load and use that skill's SKILL.md directly
  2. Semantic match (>80% conceptual overlap) → ask user: "I found a related skill {name}. Reuse/extend it, or create a new one?"
  3. No match / status: disable or training → proceed to Step 1 (create new skill)

Update on every use: increment use_count and last_used in skill_list.md.

Step 1: HUNT — Knowledge & Data Acquisition

Goal: Capture the best available real-world knowledge and data sources for this domain.

1.1 Deep Research (knowhow foundation)

Use the deep_research skill (or web search + synthesis) to answer:

  • What are the established methodologies and frameworks for this domain?
  • What do experts consider the hallmarks of excellent output?
  • What are common failure modes or pitfalls?
  • Who are the top practitioners / benchmark works?

Save raw research notes to {skill_name}/hunt_research_{YYYYMMDD}.md.

1.2 Data Source Discovery

Search for real-world data sources across three categories:

Category Description Examples
Structured APIs, databases, leaderboards, rankings Novel rankings API, arXiv API, GitHub trending
Unstructured Articles, papers, documents, web pages Top-10 novel chapters, recent papers, blog posts
Experience Human-curated, labeled, or rated examples Award winners, expert-reviewed samples, user ratings

For each source, record in crawl_config.md:

source_name: {name}
url: {url}
type: structured | unstructured | experience
frequency: daily | weekly | on_demand
data_points: {what to extract}
industry_relevance: elite | average | broad
priority: 1-5

Important distinction:

  • industry_relevance: elite → Top 10 rankings → used to set elite_avg_score (ceiling target)
  • industry_relevance: average → Rankings 20-100, general samples → used to set industry_avg_score (reference baseline, NOT the pass threshold)

⚠️ IMPORTANT: industry_avg_score is only a reference indicator. The actual pass threshold is always 90, regardless of the industry average. A skill that scores 75 (industry average) is NOT good enough — we demand excellence.

1.3 Crawler Setup

Generate crawl_runner.py with:

  • Configurable source list (reads from crawl_config.md)
  • Rate limiting and retry logic (max 3 retries, exponential backoff)
  • Deduplication (hash-based, skip already-crawled URLs)
  • Output: crawl/{source_name}/{YYYYMMDD}_{source}_{title}.md
  • Logging to crawl_log.md

Generate crawl_schedule.md with cron expressions and deployment instructions.

1.4 First Crawl

Execute crawl_runner.py immediately to seed the dataset. Log results.

Step 2: DIGEST — Knowledge Synthesis & Asset Building

Goal: Transform raw data into structured, reusable knowledge assets.

2.1 Build knowhow.md

Synthesize from Hunt research and crawled data into:

# {Skill Name} — Domain Knowhow

## Core Methodology
[step-by-step approach the skill should follow]

## Key Frameworks
[named frameworks, models, or structures relevant to this domain]

## Quality Dimensions
[what separates excellent from mediocre in this domain]

## Common Pitfalls
[what to avoid]

## Elite Reference Works
[top examples with notes on what makes them excellent]

## Structural Data References
[any APIs, schemas, or data formats the skill should know]

2.2 Build benchmark.md

Derive evaluation rubric from knowhow. Structure:

# {Skill Name} — Benchmark v{X.0.0}
Last updated: {date}
Benchmark version: {X.0.0}
Trigger: {what caused this version — initial | human_feedback | data_update}

## Scoring Dimensions

| # | Dimension | Weight | Description |
|---|-----------|--------|-------------|
| 1 | {dim}     | {%}    | {what it measures} |
...

## Scoring Rubric (per dimension)

### Dimension 1: {name}
- **90-100**: {description of excellent}
- **70-89**:  {description of good}
- **50-69**:  {description of acceptable}
- **30-49**:  {description of poor}
- **0-29**:   {description of failure}

## Total Score Calculation
Weighted sum of all dimensions = Final Score (0-100)

## Pass Thresholds
- **pass_score: 90** — **硬性及格线，所有技能必须达到 90 分才能启用**
- industry_avg_score: {X} — 行业平均参考值（仅做基准对比，不作为通过标准）
- elite_avg_score: {X} — 精英水平参考值

2.3 Label Datasets

From crawled data, create labeled samples:

  • Train set (70%): datasets/train/{YYYYMMDD}_{source}_{id}.md — used in small loop
  • Test set (30%): datasets/test/{YYYYMMDD}_{source}_{id}.md — held out, only used for final validation

Each file should include:

---
source: {source_name}
data_type: structured | unstructured | experience
split: train | test
crawled_at: {datetime}
---
[content]

2.4 Establish Industry Baseline

Score a sample of crawled data (minimum 10 items, mix of elite and average sources) using benchmark.md.

Calculate:

  • industry_avg_score = mean score of industry_relevance: average samples
  • elite_avg_score = mean score of industry_relevance: elite samples

Store full scoring results in benchmark_scores/industry_baseline_{YYYYMMDD}.md.

Update skill_list.md entry for this skill with both scores.

2.5 Write Initial Skill Prompt (v1.0.0)

Write {skill_name}/SKILL.md (the generated skill) based on knowhow.md. Also save snapshot as skill_prompt_v1.0.0.md.

Structure of the generated SKILL.md:

---
name: {skill_name}
description: {when to use, what it does}
version: 1.0.0
---

# {Skill Name}

## Role & Objective
[what this skill's Claude instance should embody]

## Methodology
[step-by-step process from knowhow.md]

## Output Format
[exact format, structure, length expectations]

## Quality Checklist
[self-check list the skill should apply before finalizing output]

## Reference Examples
[1-2 brief elite examples from crawled data]

Step 3: EVOLVING — Reinforcement Learning Loops

Goal: Iteratively improve the skill until output reaches 90 分 (our hard minimum quality bar). Two nested loops operate here: Small Loop (skill iteration) and Large Loop (benchmark iteration).

Small Loop — Skill Iteration

Termination conditions (check before each iteration):

  • ✅ Score ≥ 90 → SUCCESS: enable skill, exit loop
  • ❌ Iterations ≥ MAX_ITERATIONS (default: 10) → HALT: status: disable, log reason
  • ❌ Score delta STALL: trigger deep_research to enrich knowhow.md, then continue (max 1 stall recovery per training run)

Iteration process:

SMALL LOOP:
  current_version = read latest version from evolving_log.md (start: v1)

  REPEAT:
    1. SELECT one training sample from datasets/train/
    2. RUN current skill SKILL.md on the sample as if it were a user question
    3. SCORE output using benchmark.md → save to evolving/v{N}/score_{task}_{time}.md
    4. READ evolving_log.md to get previous scores

    IF score ≥ 90:
      → UPDATE skill_list.md: status=enable, current_score={score}
      → LOG success in evolving_log.md
      → EXIT small loop

    ELSE:
      5. ANALYZE: which benchmark dimensions scored lowest?
      6. TRACE: which part of skill_prompt / knowhow is responsible for those dimensions?
      7. PATCH: rewrite relevant section of SKILL.md
         - Minor fix (prompt wording) → version bump: patch (+0.0.1)
         - Methodology change → version bump: minor (+0.1.0)
      8. SAVE new prompt snapshot as skill_prompt_v{version}.md
      9. LOG in evolving_log.md:
         ```
         ## Iteration {N} — {datetime}
         Version: {old} → {new}
         Train sample: {filename}
         Score: {old_score} → {new_score}
         Weak dimensions: {list}
         Changes made: {description}
         Reasoning: {why this change should help}
         ```
      10. INCREMENT iteration counter → GOTO step 1

  ON HALT (max iterations or stall):
    → UPDATE skill_list.md: status=disable
    → LOG halt reason in evolving_log.md
    → NOTIFY user with summary of attempts and bottlenecks

Version numbering:

  • +0.0.1 (patch): prompt wording, tone, format tweaks
  • +0.1.0 (minor): methodology step changes, new quality checklist items
  • +1.0.0 (major): full rewrite of knowhow or benchmark

Large Loop — Benchmark Iteration (triggered by human feedback)

When a user provides feedback after using an enabled skill, evaluate the feedback:

Feedback classification:

  • Type A — Output quality issue: The skill's output was poor → small loop issue → re-run small loop with more iterations
  • Type B — Benchmark definition issue: The skill followed the benchmark but the result still felt wrong → large loop trigger

Large loop process (Type B only):

LARGE LOOP:
  1. PARSE human feedback → extract what dimension or expectation was unmet
  2. RE-HUNT:
     - New deep research focused on the unmet dimension
     - Search for new data sources that better represent the gap
     - Crawl new samples
  3. RE-DIGEST:
     - Update knowhow.md with new insights (append, don't overwrite history)
     - Revise benchmark.md:
       - Increment version: +1.0.0
       - Add/modify dimensions to address feedback
       - Recalculate industry_avg_score with new benchmark on existing data
       - Document what changed and why in benchmark.md header
  4. LABEL new datasets from newly crawled data
  5. RE-TRIGGER small loop (start from v{current_major+1}.0.0)
  6. LOG in evolving_log.md:
     ```
     ## Large Loop Trigger — {datetime}
     Feedback received: {summary}
     Feedback type: B (benchmark revision)
     Benchmark: v{old} → v{new}
     Changes: {what changed in benchmark}
     Re-entering small loop at: v{new_major}.0.0
     ```

Step 4: DEPLOY — Skill 部署到 Agent（必须执行，不可跳过）

⚠️ CRITICAL: 这是 FDE 流水线的最后一步。如果跳过此步，训练成果 等于白费 — Skill 将只存在于 workspace 中，而 Agent 仍然是个空壳。 任何情况下，训练达标后必须立即执行部署。这不是可选步骤。

部署流程

DEPLOY (score ≥ 90 后自动触发):

  1. COPY SKILL:
     → 复制 workspace/evolver/{skill_name}/skill_*.md
     → 到 agents/{agent_name}/SKILL.md
     → 验证: 确认文件存在且 > 1KB

  2. UPDATE AGENT.MD:
     → 在 agents/{agent_name}/agent.md 末尾追加:
     ```markdown
     ## Trained Skill
     - **版本**: {version}
     - **评分**: {score}
     - **状态**: ✅ 已达标 (≥ 90)
     - **训练日期**: {date}
     > 详细 Skill 定义见: SKILL.md
     ```

  3. UPDATE CONFIG.JSON:
     → 设置 agents/{agent_name}/config.json 中 status = "ready"

  4. UPDATE PLAN.MD:
     → 在 agents/{agent_name}/plan.md 中记录训练完成

  5. VERIFY (最终检查):
     → 确认 agents/{agent_name}/SKILL.md 存在
     → 确认 agents/{agent_name}/config.json status = "ready"
     → 确认 agents/{agent_name}/agent.md 包含 "Trained Skill" 章节
     → 如果任何一项缺失，立即补齐

  ON FAILURE:
     → 不能返回“训练完成”
     → 必须报告 DEPLOY 失败原因
     → 重试部署

目录结构对照

训练产出位置 部署目标位置 操作
workspace/evolver/{skill}/skill_*.md agents/{agent}/SKILL.md 复制
workspace/evolver/{skill}/evolution_log.json 保留原位 不动
- agents/{agent}/agent.md 追加摘要
- agents/{agent}/config.json 更新 status
- agents/{agent}/plan.md 更新进度

skill_list.md Schema

Each skill entry in /innovator/skill_list.md:

## {skill_name}
- **skill_id**: {uuid or sequential id}
- **directory**: /innovator/{skill_name}/
- **description**: {one-line description}
- **scope**: {domains/use cases}
- **status**: training | enable | disable
- **version**: {current version e.g. 1.3.2}
- **benchmark_version**: {e.g. 2.0.0}
- **industry_avg_score**: {number, 0-100}
- **elite_avg_score**: {number, 0-100}
- **current_score**: {number, 0-100}
- **use_count**: {number}
- **last_used**: {datetime}
- **last_updated**: {datetime}
- **created_at**: {datetime}
- **keywords**: [{list of trigger keywords}]

evolving_log.md Schema

Located at {skill_name}/evolving/evolving_log.md:

# Evolving Log — {skill_name}

## Summary
- Total iterations: {N}
- Current version: {version}
- Current score: {score}
- Industry avg: {score}
- Status: training | enable | disable

---

## Iteration {N} — {datetime}
Loop type: small | large
Version: {old} → {new}
Score: {old} → {new}
Weak dimensions: [dim1, dim2]
Changes: {description}
Reasoning: {why}

---

Execution Checklist

When a new skill creation is triggered, Claude must complete ALL of these. 不允许跳过任何步骤：

  • [ ] Step 0: Search skill_list.md — no match confirmed
  • [ ] Step 1.1: Deep research completed → hunt_research_{date}.md
  • [ ] Step 1.2: Data sources identified → crawl_config.md
  • [ ] Step 1.3: Crawler script written → crawl_runner.py, crawl_schedule.md
  • [ ] Step 1.4: First crawl executed → data in crawl/{source}/
  • [ ] Step 2.1: knowhow.md written
  • [ ] Step 2.2: benchmark.md v1.0.0 written (pass_score = 90)
  • [ ] Step 2.3: Train/test datasets labeled and split
  • [ ] Step 2.4: Industry baseline scored → industry_avg_score and elite_avg_score set
  • [ ] Step 2.5: SKILL.md v1.0.0 written + skill_prompt_v1.0.0.md saved
  • [ ] Step 3 (small loop): Evolving completed → score ≥ 90
  • [ ] Step 4.1: SKILL.md 复制到 agents/{agent}/SKILL.md ❗
  • [ ] Step 4.2: agent.md 追加 Trained Skill 摘要 ❗
  • [ ] Step 4.3: config.json status 更新为 ready ❗
  • [ ] Step 4.4: plan.md 记录训练完成 ❗
  • [ ] Step 4.5: 验证部署 — 确认 SKILL.md 存在且完整 ❗
  • [ ] Skill enabled → skill_list.md updated with status: enable
  • [ ] Delivered to user with score report + 部署确认

Reference Files

Read these when needed:

  • references/skill_list_template.md — blank template for skill_list.md initialization
  • references/benchmark_template.md — starter benchmark structure for common domains
  • references/crawl_runner_template.py — base crawler script to customize
  • references/evolving_log_template.md — blank evolving log template