U-011 fix: deterministically enforce the canonical BRD->PRD->SDD->Benchmark chain

The 6th real self-hosting attempt (first to reach the full 4-node doc-writing graph after the previous five bugs) hit a genuinely new failure: this real graph_manifest had no edges at all, not just wrong field names -- worse than the case _normalize_graph_manifest already handled. With zero dependency info to normalize from, topological_order() scheduled 'benchmark' before 'sdd' (alphabetically first among zero-in-degree nodes), and build_prompt() crashed with KeyError: 'sdd_content' trying to format a template needing sdd's not-yet-produced output. _PROMPT_TEMPLATE always asks for exactly the same fixed 4-node linear chain -- that's a constant architectural fact, not something that should be re-inferred from the LLM's output every call. _normalize_graph_manifest now deterministically enforces brd->prd->sdd->benchmark whenever all four canonical artifact_types are present, regardless of what depends_on/edges the LLM did or didn't provide. Same "机制用代码" principle as ensure_worktree_committed and the earlier normalize logic. Three new regression tests: zero dependency info still gets the chain enforced, node_id != artifact_type still resolves to the real node_id (not a guessed string), and a partial set (missing one of the four types) is correctly left alone rather than guessed at. 560 tests green (440 src + 120 supervisor). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

U-011 fix: deterministically enforce the canonical BRD->PRD->SDD->Benchmark chain
The 6th real self-hosting attempt (first to reach the full 4-node doc-writing graph after the previous five bugs) hit a genuinely new failure: this real graph_manifest had no edges at all, not just wrong field names -- worse than the case _normalize_graph_manifest already handled. With zero dependency info to normalize from, topological_order() scheduled 'benchmark' before 'sdd' (alphabetically first among zero-in-degree nodes), and build_prompt() crashed with KeyError: 'sdd_content' trying to format a template needing sdd's not-yet-produced output. _PROMPT_TEMPLATE always asks for exactly the same fixed 4-node linear chain -- that's a constant architectural fact, not something that should be re-inferred from the LLM's output every call. _normalize_graph_manifest now deterministically enforces brd->prd->sdd->benchmark whenever all four canonical artifact_types are present, regardless of what depends_on/edges the LLM did or didn't provide. Same "机制用代码" principle as ensure_worktree_committed and the earlier normalize logic. Three new regression tests: zero dependency info still gets the chain enforced, node_id != artifact_type still resolves to the real node_id (not a guessed string), and a partial set (missing one of the four types) is correctly left alone rather than guessed at. 560 tests green (440 src + 120 supervisor). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
bboonnee
1 parent 2aa68435
Showing 4 changed files with 90 additions and 1 deletions
docs/development-plan.md
docs/sdd/unit_matrix.json
src/test_work_graph_plan.py
src/work_graph_plan.py
@@ -121,6 +121,7 @@
   - Done（**2026-07-31 订正**：原来写"REQ-02 happy 用例通过（Planner 产出 BRD/PRD/SDD/Benchmark 四类 artifact 且血缘完整）"，但 03-C §3.4 里 `c_work_graph_plan` 自己的契约明确写着"不执行，只规划"——它只产出 `graph_manifest`（结构/计划），不产出真实 Artifact 内容；REQ-02 happy 用例要求的"四类 artifact 且血缘完整"需要真正执行（U-012/U-013）加上 `c_artifact_write`（见下方缺口），单靠 U-011 做不到，也不该做，这是同一类 Done 标准过头的问题，U-009/U-010 已经出现过两次）：`c_work_graph_plan` 真实调用 `claude --agent planner -p` 产出一个通过结构校验的 `graph_manifest`（四个节点对应 brd/prd/sdd/benchmark，依赖链 sdd→prd→brd 正确），落一条 `work_graph_run(status='planning')` 记录；REQ-02 happy 用例整体随 U-012/U-013 落地后累计满足
   - **已知缺口（2026-07-31，实现时发现）**：`c_artifact_write`（03-C §3.7）是 REQ-02 happy/failure 两个用例都直接依赖的控制器（failure 用例原话："`c_artifact_write` 应拒绝写入"），但翻遍 development-plan.md 全部 43 个 Unit，**没有任何一个 Unit 的"覆盖"字段提到 `c_artifact_write`**——不是分散在别处，是真的没人认领。等做到 U-012/U-013（或不管哪个 Unit 第一个需要"把 Agent 写的文件登记成 artifact 行"）时需要显式补上，不要假设它已经存在
   - **技术发现（2026-07-31）**：`--agent` 和 `--json-schema` 同时使用时，`--json-schema` 会被静默忽略（`structured_output` 完全不出现），已记录到 05 节 §5.4.1.1，`plan_work_graph` 改成从 `result` 里解析 ```` ```json ```` 围栏代码块
+  - **真实 bug（2026-08-03，U-042 第六次真实自举压测发现）**：前五次真实压测各自卡在别的更早环节，这是第一次真正走到"用真实 `graph_manifest` 跑完整个 4 节点文档链"这一步就出问题。这次真实调用返回的 `graph_manifest` 里，四个节点连 `edges` 都没给（比 2026-08-01 那次"字段名不对但至少有 edges"更彻底），`_normalize_graph_manifest` 原来的"从 edges 反推 depends_on"逻辑无从反推——`topological_order()` 在完全没有依赖信息时按 node_id 字母序处理零入度节点，"benchmark"排在"sdd"前面，于是 `e2e_pipeline.build_prompt()` 试图给 benchmark 的 prompt 模板填 `{sdd_content}` 时，`sdd` 节点根本还没跑，直接 `KeyError`。**这次没有再指望 LLM 每次都能把这个结构给对**：`_PROMPT_TEMPLATE` 本来就是让 Planner 每次都产出完全固定的 brd→prd→sdd→benchmark 线性四节点，这是个不会变的架构事实，不是每次都要重新推断的东西——`_normalize_graph_manifest` 现在多一步：只要四个规范 `artifact_type`（brd/prd/sdd/benchmark）齐全，不管 LLM 给没给 `depends_on`/`edges`，直接按已知的固定线性顺序强制补上依赖链，和 `ensure_worktree_committed` 一样是"机制用代码"而不是祈祷 Agent 每次都按契约给结构。补了三个回归测试：完全没有依赖信息时依然能正确补链、`node_id` 和 `artifact_type` 不同名时按真实 `node_id` 补（不是瞎猜字符串）、四个规范类型不全时不强行拼凑。
   - 覆盖：01-M `artifact`/`work_graph_run`；03-C `c_work_graph_plan`；REQ-02（happy 用例的"计划"部分，非全部）
 - **U-012 [Value Path]** `c_work_graph_execute` 完整实现（`auto` 权限模式、`USAGE_LIMIT_REACHED` 退避重试 `p_usage_limit_wait`、`CLASSIFIER_BLOCKED` 处理）
@@ -23,7 +23,7 @@
     {"id": "U-009", "phase": "PH1", "sub_phase": "PH1-S1", "name": "c_task_create + task 状态机（c_project_create 已在 U-002 完成）", "classification": "Value Path", "depends_on": ["U-001", "U-002"], "coverage": {"model": ["task"], "view": [], "controller": ["c_task_create"], "process": ["p_org_task_lifecycle"], "requirement": ["REQ-01"]}, "done": "REQ-01 boundary/failure 通过；happy 用例需要 c_org_node_trigger 完整实现，随 U-010 累计满足（2026-07-31 订正）"},
     {"id": "U-010", "phase": "PH1", "sub_phase": "PH1-S1", "name": "c_org_node_trigger 完整实现（三段前向状态迁移，见 development-plan.md 2026-07-31 范围说明）", "classification": "Value Path", "depends_on": ["U-005", "U-009"], "coverage": {"model": [], "view": [], "controller": ["c_org_node_trigger"], "process": [], "requirement": []}, "done": "三个 Org Agent 分别可被触发（真实 claude --agent 调用），task.status 按 queued→planning_brd_prd_sdd→coding→validating 正确流转；重试/路由/blocked_pending_human 不在本 Unit 范围"},
-    {"id": "U-011", "phase": "PH1", "sub_phase": "PH1-S2", "name": "c_work_graph_plan 实现（只规划不执行，见 development-plan.md 2026-07-31 订正）", "classification": "Value Path", "depends_on": ["U-004", "U-010"], "coverage": {"model": ["artifact", "work_graph_run"], "view": [], "controller": ["c_work_graph_plan"], "process": [], "requirement": ["REQ-02"]}, "done": "graph_manifest 通过结构校验（brd/prd/sdd/benchmark 四节点+依赖链），work_graph_run(status=planning) 落地；REQ-02 happy 用例整体随 U-012/U-013 累计满足。c_artifact_write（03-C §3.7）目前没有任何 Unit 认领，是已知缺口"},
+    {"id": "U-011", "phase": "PH1", "sub_phase": "PH1-S2", "name": "c_work_graph_plan 实现（只规划不执行，见 development-plan.md 2026-07-31/2026-08-03 订正）", "classification": "Value Path", "depends_on": ["U-004", "U-010"], "coverage": {"model": ["artifact", "work_graph_run"], "view": [], "controller": ["c_work_graph_plan"], "process": [], "requirement": ["REQ-02"]}, "done": "graph_manifest 通过结构校验（brd/prd/sdd/benchmark 四节点+依赖链），work_graph_run(status=planning) 落地；REQ-02 happy 用例整体随 U-012/U-013 累计满足。c_artifact_write（03-C §3.7）目前没有任何 Unit 认领，是已知缺口。2026-08-03：U-042 第六次真实自举压测发现真实 LLM 输出可能完全不给 depends_on/edges（比此前已知的\"字段名不对\"更彻底），_normalize_graph_manifest 现在对 brd/prd/sdd/benchmark 四个规范 artifact_type 齐全时强制按已知的固定线性链补 depends_on，不再完全依赖 LLM 自己给对"},
     {"id": "U-012", "phase": "PH1", "sub_phase": "PH1-S2", "name": "c_work_graph_execute 完整实现（见 development-plan.md 2026-07-31 范围说明）", "classification": "Value Path", "depends_on": ["U-006", "U-011"], "coverage": {"model": [], "view": [], "controller": ["c_work_graph_execute"], "process": ["p_work_graph_execution", "p_usage_limit_wait"], "requirement": ["REQ-07", "REQ-14"]}, "done": "REQ-07 boundary/failure 通过（happy 需 PH2 REQ-04/05/06 就绪后累计满足）；REQ-14 happy 通过。退避重试是整图级别，非逐节点；CLASSIFIER_BLOCKED 用单次调用 permission_denials≥3 近似判定"},
     {"id": "U-013", "phase": "PH1", "sub_phase": "PH1-S2", "name": "c_work_node_run + git worktree 隔离（见 development-plan.md 2026-07-31 订正、2026-08-02/03 真实 bug 修复）", "classification": "Value Path", "depends_on": ["U-012"], "coverage": {"model": ["work_node_execution"], "view": [], "controller": ["c_work_node_run"], "process": [], "requirement": ["REQ-03"]}, "done": "REQ-03 boundary 通过（真实 git worktree 隔离+merge，无冲突）；happy 需 c_artifact_write 落地后累计满足。2026-08-02/03：U-042 第一次真实自举压测发现 Coder 编辑 worktree 后从不 commit、merge_worktree 空操作、真实改动被 remove_worktree 静默吞掉的真实 bug，新增 ensure_worktree_committed() 兜底自动提交，已修复并补测试"},
     {"id": "U-014", "phase": "PH1", "sub_phase": "PH1-S3", "name": "贪吃蛇场景端到端验收（含补建 c_artifact_write，见 development-plan.md 2026-07-31 实现补充）", "classification": "Value Path", "depends_on": ["U-011", "U-012", "U-013"], "coverage": {"model": ["artifact"], "view": [], "controller": ["c_artifact_write"], "process": [], "requirement": []}, "done": "架构文档 §11 验收标准第 1-3 条全部达成；plan_documents（Planner 四件套+血缘）真实验证过，Coder/Validator 全流程真实跑通待用户确认后执行"},
@@ -128,6 +128,64 @@ class NormalizeGraphManifestTest(unittest.TestCase):
         result = work_graph_plan._normalize_graph_manifest(manifest)
         self.assertEqual(result["nodes"][0]["depends_on"], [])
+    def test_canonical_doc_chain_enforced_even_with_zero_dependency_info(self):
+        # 2026-08-03 regression: a real call omitted depends_on AND edges
+        # entirely -- worse than test_depends_on_derived_from_edges_when_absent,
+        # which at least had edges to derive from. With nothing to
+        # normalize, topological_order() scheduled 'benchmark' before 'sdd'
+        # (alphabetically first among zero-in-degree nodes), and
+        # e2e_pipeline's build_prompt() crashed formatting a template that
+        # needed sdd's not-yet-produced output.
+        manifest = {
+            "nodes": [
+                {"node_id": "brd", "artifact_type": "brd"},
+                {"node_id": "prd", "artifact_type": "prd"},
+                {"node_id": "sdd", "artifact_type": "sdd"},
+                {"node_id": "benchmark", "artifact_type": "benchmark"},
+            ],
+            "edges": [],
+        }
+        result = work_graph_plan._normalize_graph_manifest(manifest)
+        by_type = {n["artifact_type"]: n for n in result["nodes"]}
+        self.assertEqual(by_type["brd"]["depends_on"], [])
+        self.assertEqual(by_type["prd"]["depends_on"], ["brd"])
+        self.assertEqual(by_type["sdd"]["depends_on"], ["prd"])
+        self.assertEqual(by_type["benchmark"]["depends_on"], ["sdd"])
+
+    def test_canonical_chain_enforcement_uses_real_node_ids_not_artifact_types(self):
+        # node_id and artifact_type can differ (e.g. real LLM output naming
+        # nodes "n1"/"n2"/... while still setting artifact_type correctly) --
+        # the enforced dependency must reference the real node_id.
+        manifest = {
+            "nodes": [
+                {"node_id": "step-1", "artifact_type": "brd"},
+                {"node_id": "step-2", "artifact_type": "prd"},
+                {"node_id": "step-3", "artifact_type": "sdd"},
+                {"node_id": "step-4", "artifact_type": "benchmark"},
+            ],
+            "edges": [],
+        }
+        result = work_graph_plan._normalize_graph_manifest(manifest)
+        by_type = {n["artifact_type"]: n for n in result["nodes"]}
+        self.assertEqual(by_type["prd"]["depends_on"], ["step-1"])
+        self.assertEqual(by_type["sdd"]["depends_on"], ["step-2"])
+        self.assertEqual(by_type["benchmark"]["depends_on"], ["step-3"])
+
+    def test_partial_canonical_set_does_not_trigger_enforcement(self):
+        # only brd/prd/sdd present, no benchmark -- not a recognized
+        # canonical 4-node doc chain, must not guess at a shape.
+        manifest = {
+            "nodes": [
+                {"node_id": "brd", "artifact_type": "brd"},
+                {"node_id": "prd", "artifact_type": "prd"},
+                {"node_id": "sdd", "artifact_type": "sdd"},
+            ],
+            "edges": [],
+        }
+        result = work_graph_plan._normalize_graph_manifest(manifest)
+        for node in result["nodes"]:
+            self.assertEqual(node["depends_on"], [])
+
 class HappyPathTest(PlanWorkGraphTestBase):
     def test_returns_run_id_and_graph_manifest(self):
@@ -102,6 +102,9 @@ def _extract_graph_manifest(text):
     return None
+_CANONICAL_DOC_CHAIN = ("brd", "prd", "sdd", "benchmark")
+
+
 def _normalize_graph_manifest(manifest):
     """Real LLM output has been observed (2026-08-01, U-031 integration
     validation) to use `id` instead of the documented `node_id` key, and to
@@ -114,6 +117,24 @@ def _normalize_graph_manifest(manifest):
     instead of each one needing its own defensive "id or node_id,
     depends_on or edges" fallback logic repeated independently. Mutates and
     returns `manifest`.
+
+    **Canonical doc-chain enforcement (2026-08-03, real self-hosting run)**:
+    a real call was observed to omit the brd->prd->sdd->benchmark
+    dependency entirely -- no `depends_on`, no `edges` connecting them at
+    all, not just the wrong key names the edge-derived backfill above can
+    already recover from. With no dependency info to normalize from,
+    `topological_order()` scheduled `benchmark` before `sdd` even ran
+    (alphabetically first among nodes with an empty in-degree), and
+    `e2e_pipeline.build_prompt()` crashed with `KeyError: 'sdd_content'`
+    trying to format a template that needed sdd's not-yet-produced output.
+    `_PROMPT_TEMPLATE` always asks for exactly this fixed 4-node linear
+    chain -- it's a constant architectural fact about what the Planner's
+    doc-writing graph looks like, not something that genuinely varies call
+    to call, so don't leave it to chance: whenever all four canonical
+    artifact_types are present, deterministically enforce the known linear
+    order regardless of what edges/depends_on the LLM did or didn't
+    provide. Same "机制用代码" principle already applied elsewhere
+    (`work_node_run.ensure_worktree_committed`, etc.).
     """
     nodes = manifest.get("nodes", [])
     for node in nodes:
@@ -128,6 +149,15 @@ def _normalize_graph_manifest(manifest):
             for node in nodes:
                 if node.get("node_id") == to_id and from_id not in node["depends_on"]:
                     node["depends_on"].append(from_id)
+
+    by_artifact_type = {n.get("artifact_type"): n for n in nodes}
+    if all(t in by_artifact_type for t in _CANONICAL_DOC_CHAIN):
+        for prev_type, cur_type in zip(_CANONICAL_DOC_CHAIN, _CANONICAL_DOC_CHAIN[1:]):
+            prev_id = by_artifact_type[prev_type]["node_id"]
+            cur_node = by_artifact_type[cur_type]
+            if prev_id not in cur_node["depends_on"]:
+                cur_node["depends_on"].append(prev_id)
+
     return manifest