Commit a33aa8cf0d37aca7ddd92fe67e84ca91b738e1b2
1 parent
8712aef4
F: three-perspective jury as a real Work Graph fan-out
Implements docs/todo/verifier-fan-out-jury-design.md. root_cause_trace.py:
_tally_perspectives() extracts the existing >=2/3 rule out of
judge_root_cause so the closure-based version and the new
judge_root_cause_via_work_graph() share one implementation, not two that
could drift apart. The fan-in vote stays plain Python (zero_token
aggregation per the design-dynamic-work-graph Skill's own edge_type
distinction) rather than becoming a fourth Work Node -- no reason to pay
for an Agent call just to count votes.
judge_root_cause_via_work_graph(): each perspective becomes an
independent Work Node with real schema-constrained structured output
({"category", "detail"} directly), not real_validator_perspective's
manual "CATEGORY:"/"DETAIL:" text-line parsing. judge_root_cause /
real_validator_perspective are kept, not removed -- documented as the
lightweight fallback for scenarios that don't need worktree/Work Graph
overhead.
org_level_loop.run_fix_loop (the one real production caller) now calls
the Work Graph version; the fixer/verify calls are unchanged flat
trigger_org_node calls.
Real bug found and fixed along the way: this is the first production code
path that actually triggers a run_graph() multi-node CONCURRENT layer
(every graph_manifest before F was single-node or a strict serial chain,
so G's ThreadPoolExecutor path had never really run in production).
scheduler.py::run_goal_to_completion and
task_execution.py::start_task_execution_in_background both created their
conn without check_same_thread=False -- fixed both, since their conn can
now reach run_graph()'s worker threads via run_fix_loop.
Test boundary migrated in test_org_level_loop.py/test_scheduler.py: the
3 perspective calls are now mocked at work_graph_execute.write_workflow_script/
execute_work_graph (matching test_work_graph_orchestrate.py's pattern)
instead of org_node_trigger.trigger_org_node. New tests cover: three real
independent Work Node executions, judgment-rule equivalence with the
closure version, clean per-node input_payload, and concurrent wall-clock
overlap across the three perspectives.
Full src suite (586 tests) + supervisor suite (125 tests) pass with zero
regressions.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Showing
9 changed files
with
472 additions
and
47 deletions
docs/todo/graph-engineering-open-items.md
| @@ -90,7 +90,7 @@ | @@ -90,7 +90,7 @@ | ||
| 90 | 90 | ||
| 91 | ## F. 三视角评审团机制要不要推广到其他验证点 | 91 | ## F. 三视角评审团机制要不要推广到其他验证点 |
| 92 | 92 | ||
| 93 | -**状态**:✅ 已确认,且明确了实现方式 | 93 | +**状态**:✅ 已实现(2026-08-06)→ 需求文档 `docs/todo/verifier-fan-out-jury-requirements.md` + 设计文档 `docs/todo/verifier-fan-out-jury-design.md` |
| 94 | 94 | ||
| 95 | **来源**:综述版对照 §7 开放问题 4 + 14 步版对照 §3 开放问题 4(同一主题) | 95 | **来源**:综述版对照 §7 开放问题 4 + 14 步版对照 §3 开放问题 4(同一主题) |
| 96 | 96 | ||
| @@ -98,7 +98,7 @@ | @@ -98,7 +98,7 @@ | ||
| 98 | 98 | ||
| 99 | **确认结果(2026-08-05)**:要做,而且明确了实现方式——**不是**继续用现在 `root_cause_trace.judge_root_cause` 那种 Python 层直接顺序调用三个 `perspective_fn` 的写法,而是把三视角评审团**做成一张真正的 Work Graph fan-out/fan-in 菱形拓扑**:fan-out 出 N 个独立的验证节点(每个节点是一次独立的 `claude -p` 调用,带自己的视角提示词),fan-in 节点做投票/汇总。用户原话:"三视角评审团可以做,做成 work graph 的 fan out 图来实现吧"。 | 99 | **确认结果(2026-08-05)**:要做,而且明确了实现方式——**不是**继续用现在 `root_cause_trace.judge_root_cause` 那种 Python 层直接顺序调用三个 `perspective_fn` 的写法,而是把三视角评审团**做成一张真正的 Work Graph fan-out/fan-in 菱形拓扑**:fan-out 出 N 个独立的验证节点(每个节点是一次独立的 `claude -p` 调用,带自己的视角提示词),fan-in 节点做投票/汇总。用户原话:"三视角评审团可以做,做成 work graph 的 fan out 图来实现吧"。 |
| 100 | 100 | ||
| 101 | -**待办**:✅ 需求文档已产出 → `docs/verifier-fan-out-jury-requirements.md`。明确标注**强依赖 G**——没有并行执行能力,把 `judge_root_cause` 改成 Work Graph fan-out 反而更慢(多了 Work Node 调度开销,没有并行收益),建议排在 G 之后。 | 101 | +**待办**:✅ 已实现(2026-08-06)。汇总节点没有做成第四个 Work Node——投票是纯 `zero_token` 聚合(跟 E 里 `edge_type` 的区分直接对应),不需要 Agent 推理,做成普通 Python 函数在 `run_graph()` 返回后直接对 `outputs` 计数,不产生额外 `claude -p` 调用/token 成本,设计文档第 1 节记录了这个取舍。实现内容:`root_cause_trace.py` 新增 `_tally_perspectives()`(>=2/3 判定规则从 `judge_root_cause` 抽出来,闭包版和 Work Graph 版共用同一份规则,不是两份容易失配的实现)+ `judge_root_cause_via_work_graph()`(N 个视角节点用真实 schema 约束的结构化输出,不再像 `real_validator_perspective` 那样手工解析"CATEGORY:/DETAIL:"文本行)。闭包版 `judge_root_cause`/`real_validator_perspective` **保留**,不废弃(轻量备选,见设计文档第 4 节)。`org_level_loop.run_fix_loop`(唯一真实生产调用方)已迁移到 Work Graph 版本,fixer/verify 两次调用保持不变的 flat `trigger_org_node` 调用。**顺带修了一个真实的潜在 bug**:`judge_root_cause_via_work_graph` 是第一个真正触发 `run_graph()` 多节点并发层的生产代码路径(之前所有真实 `graph_manifest` 要么单节点要么严格串行链,G 的并发调度从未在生产里真正跑过),暴露出 `scheduler.py::run_goal_to_completion`/`task_execution.py::start_task_execution_in_background` 的 `schema.connect(db_path)` 都没传 `check_same_thread=False`——已一并修复。测试改造范围(设计文档第 5 节标注的最高风险点):`test_org_level_loop.py`/`test_scheduler.py` 的 3 个测试把 mock 边界从 `org_node_trigger.trigger_org_node`(拦截"CATEGORY:"提示词)改成 `work_graph_execute.write_workflow_script`/`execute_work_graph`。新增测试证明验收标准 1-3(三次真实独立 Work Node 执行、判定规则与闭包版等价、每个视角节点的 `input_payload` 不含其他视角内容)+ 一个直接验证并发重叠的测试(`ConcurrentPerspectiveExecutionTest`)。`python3 -m unittest discover -s src`(586 tests)+ `discover -s supervisor`(125 tests)全绿,零回归。FR-4(`verify_fn` 通用对接其他验证点)本轮不实现,标注"应该"优先级,留待后续。 |
| 102 | 102 | ||
| 103 | --- | 103 | --- |
| 104 | 104 | ||
| @@ -134,7 +134,7 @@ | @@ -134,7 +134,7 @@ | ||
| 134 | |---|---|---|---|---|---| | 134 | |---|---|---|---|---|---| |
| 135 | | **G** 并行执行 | Work Graph 层内并发、层间屏障 | ✅ **已实现**(`bc0eaa5`,575 测试全绿) | M | 已完成 | 无 | | 135 | | **G** 并行执行 | Work Graph 层内并发、层间屏障 | ✅ **已实现**(`bc0eaa5`,575 测试全绿) | M | 已完成 | 无 | |
| 136 | | **E** 动态 Work Graph 设计能力 | 让 Coder/Validator 也能规划自己的图,Planning 提示词识别可并行节点 | ✅ **已实现**(2026-08-06,580 测试全绿) | L | 已完成 | G(已满足) | | 136 | | **E** 动态 Work Graph 设计能力 | 让 Coder/Validator 也能规划自己的图,Planning 提示词识别可并行节点 | ✅ **已实现**(2026-08-06,580 测试全绿) | L | 已完成 | G(已满足) | |
| 137 | -| **F** 三视角评审团 fan-out | `judge_root_cause` 从 Python 函数改成 Work Graph 菱形拓扑 | 设计已产出 → `docs/todo/verifier-fan-out-jury-design.md`,实现中 | M | P0(G/E 都已落地,正在推进) | 无(G 已满足) | | 137 | +| **F** 三视角评审团 fan-out | `judge_root_cause` 从 Python 函数改成 Work Graph 菱形拓扑 | ✅ **已实现**(2026-08-06,586 测试全绿) | M | 已完成 | 无(G 已满足) | |
| 138 | | **B** loop-until-dry Skill | Validator 未知规模扫描收敛能力 | 仅需求 | S/M | P2 | 无 | | 138 | | **B** loop-until-dry Skill | Validator 未知规模扫描收敛能力 | 仅需求 | S/M | P2 | 无 | |
| 139 | | **H** Role Package 抽象层 | 统一 role/context/skills/applications/workflows/permissions/acceptance 打包格式 | 仅需求 | XL | P2(战略级,不阻塞其他项) | workflows 字段依赖 E 的产出 | | 139 | | **H** Role Package 抽象层 | 统一 role/context/skills/applications/workflows/permissions/acceptance 打包格式 | 仅需求 | XL | P2(战略级,不阻塞其他项) | workflows 字段依赖 E 的产出 | |
| 140 | | **D** Time-travel 调试 | 回到历史节点分叉重跑 | 仅需求 | L | P3 | 无强依赖,但价值随图变复杂(E 落地后)才会显现 | | 140 | | **D** Time-travel 调试 | 回到历史节点分叉重跑 | 仅需求 | L | P3 | 无强依赖,但价值随图变复杂(E 落地后)才会显现 | |
docs/todo/verifier-fan-out-jury-design.md
0 → 100644
| 1 | +# 三视角评审团——Work Graph Fan-out 实现——详细设计(SDD) | ||
| 2 | + | ||
| 3 | +**日期**: 2026-08-06 | ||
| 4 | +**性质**: 详细设计,对应需求文档 `docs/todo/verifier-fan-out-jury-requirements.md`(条目 F,已确认)。回答该需求文档第 8 节的开放问题。 | ||
| 5 | +**依赖代码**:`src/root_cause_trace.py`(`judge_root_cause`/`_PERSPECTIVE_FRAMINGS`)、`src/org_level_loop.py`(`run_fix_loop`,唯一真实生产调用方)、`src/work_graph_orchestrate.py`(G 已实现的 `run_graph()`/`topological_layers()`)。 | ||
| 6 | +**状态**: 设计草案,待评审后实施。 | ||
| 7 | + | ||
| 8 | +## 1. 汇总节点不是 Work Node——这是"zero_token 边"原则的直接应用 | ||
| 9 | + | ||
| 10 | +需求文档背景里"1 个汇总节点"的表述容易被误读成"汇总也要是一次 `claude -p` 调用"。实际不需要:`>=2/3` 投票本身是纯粹的计数聚合,不需要 Agent 推理——这正是 E 设计文档里强调的 `edge_type: zero_token` 场景本身。汇总逻辑保持为一个普通 Python 函数,在 `run_graph()` 返回之后,在调用方所在的进程里直接对 `outputs` 做计数,不产生额外的 `claude -p` 调用、不产生额外 token 成本(呼应需求文档第 7 节风险"token 成本上升"的顾虑,此处直接不引入这块新增成本)。 | ||
| 11 | + | ||
| 12 | +验收标准 1 要求"能在 execution_event/日志里看到三次独立的节点执行记录"——这只要求**三个视角节点**是真 Work Node(`run_graph()`/`run_work_node()` 本来就会为每个节点写 `node_start`/`node_completed` 日志),不要求汇总也是。 | ||
| 13 | + | ||
| 14 | +## 2. 判定规则复用,不重写(FR-3) | ||
| 15 | + | ||
| 16 | +`root_cause_trace.judge_root_cause` 现有的 `>=2/3` 计数逻辑被抽成一个共享私有函数 `_tally_perspectives(judgments)`,`judge_root_cause`(保留,闭包版本)和新的 `judge_root_cause_via_work_graph`(Work Graph 版本)都调用它——两套判定规则物理上是同一份代码,不是两份逻辑各自实现、容易失配。`judgments` 的 dict 形状(`{"category", "detail"}`)两个版本完全一致,`_tally_perspectives` 不需要知道调用方是闭包还是 Work Node 输出。 | ||
| 17 | + | ||
| 18 | +## 3. `judge_root_cause_via_work_graph`:新的 Work Graph 版本 | ||
| 19 | + | ||
| 20 | +```python | ||
| 21 | +def judge_root_cause_via_work_graph( | ||
| 22 | + conn, task_id, target_folder_path, bug_description, context, | ||
| 23 | + org_node="validator", skill_id="qa-task-validation", timeout=300, | ||
| 24 | +): | ||
| 25 | +``` | ||
| 26 | + | ||
| 27 | +内部结构: | ||
| 28 | + | ||
| 29 | +1. `_PERSPECTIVE_FRAMINGS`(已存在,三个视角的 framing 文本)驱动构造每个视角的完整提示词(复用现有 framing 文本,不改视角内容本身)。 | ||
| 30 | +2. 构造一个菱形(严格说是"N 节点单层,无边")`graph_manifest`:`nodes = [{"node_id": "perspective-" + name, "depends_on": []} for name in perspectives]`,`edges = []`——三个节点互不依赖,`topological_layers()` 天然把它们分进同一层,G 自动并发调度,不需要额外的调度代码。 | ||
| 31 | +3. 像 `e2e_pipeline.py::implement_code`/`validate_implementation_with_own_graph` 一样,手动 `INSERT INTO work_graph_run (..., status='running')`(不经过 `plan_work_graph()`——这个图的形状是固定的、代码已知的,不需要一次额外的 LLM 规划调用,同样是"机制用代码"原则的应用)。 | ||
| 32 | +4. `run_graph()` 执行:每个视角节点用**真实 schema 约束的结构化输出**(`{"type": "object", "required": ["category", "detail"]}`),不再像现有 `real_validator_perspective` 那样手工解析"CATEGORY:/DETAIL:"文本行——`work_graph_orchestrate._make_run_fn` 走的是单节点 dynamic-workflow 脚本机制(`agent(prompt, {{ agentType, schema }})`),这条路径已经被 2026-08-01 的真实调用验证过 `structured_output` 真的能拿到(`work_graph_orchestrate.py` 模块 docstring 有记录),跟 `work_graph_plan.py` 那条"`--json-schema` 对 `--agent` 无效"的已知限制是两回事,不受它影响。 | ||
| 33 | +5. `_tally_perspectives([outputs["perspective-" + name] for name in perspectives])`,返回值形状跟现有 `judge_root_cause` 完全一致(`{"root_cause_category", "root_cause_detail", "judgments"}`),调用方不需要区分。 | ||
| 34 | + | ||
| 35 | +每个视角节点获得独立、干净的上下文(验收标准 3):`_ensure_node_row`/`run_work_node` 本身就是每个 node_exec_id 一次全新的 workflow 脚本渲染 + 独立子进程调用,`input_payload` 只含 `{"node_id": ...}`,视角提示词通过 `build_prompt` 的闭包参数传入,节点之间不共享对话历史——这是 `run_graph()` 现有机制的天然属性,不需要额外设计。 | ||
| 36 | + | ||
| 37 | +## 4. `org_level_loop.run_fix_loop` 迁移(FR-5,唯一真实生产场景) | ||
| 38 | + | ||
| 39 | +现状: | ||
| 40 | + | ||
| 41 | +```python | ||
| 42 | +perspective_fns = [root_cause_trace.real_validator_perspective(...) for name in (...)] | ||
| 43 | +judgment = root_cause_trace.judge_root_cause(perspective_fns) | ||
| 44 | +``` | ||
| 45 | + | ||
| 46 | +改为: | ||
| 47 | + | ||
| 48 | +```python | ||
| 49 | +judgment = root_cause_trace.judge_root_cause_via_work_graph( | ||
| 50 | + conn, task_id, target_folder_path, bug_description, context, | ||
| 51 | +) | ||
| 52 | +``` | ||
| 53 | + | ||
| 54 | +`real_validator_perspective`/闭包版 `judge_root_cause` **不删除**——需求文档第 8 节开放问题已问过"要不要保留作为轻量备选",答案是保留:`src/test_root_cause_trace.py` 现有单测(`JudgeRootCauseTest`/`RealValidatorPerspectiveParsingTest`)覆盖的是判定规则本身的正确性,这些测试不需要真实 `claude` 子进程调用就能跑(用纯 Python 闭包 mock),继续作为规则正确性的快速回归保障(`_tally_perspectives` 被两边共用,这些测试事实上也在验证 Work Graph 版本的判定逻辑);闭包版本本身作为"不需要 worktree/Work Graph 开销的场景"的备选实现继续存在,不强制废弃。 | ||
| 55 | + | ||
| 56 | +## 5. 测试改造范围(真实影响面,如实记录) | ||
| 57 | + | ||
| 58 | +`src/test_org_level_loop.py` 现有 3 个测试全部通过 mock `org_node_trigger.trigger_org_node`(拦截"CATEGORY:"提示词)来伪造三视角判断——这个 mock 边界在迁移后失效(新实现走 `work_graph_execute.execute_work_graph`,不再调用 `trigger_org_node` 做视角判断)。必须同步把这三个测试的 mock 边界改成 `work_graph_execute.write_workflow_script`/`execute_work_graph`(跟 `test_work_graph_orchestrate.py` 现有模式一致),`org_node_trigger.trigger_org_node` 的 mock 保留但只用于 fixer/verify 两次调用(迁移范围明确不包括这两处)。这是本次改动风险最高的一步——`run_fix_loop` 是真实生产闭环的一部分,测试改造必须逐条比对新旧断言等价,不能只是"能跑通"就算数。 | ||
| 59 | + | ||
| 60 | +## 6. 不实现的部分(诚实记录,对应需求文档非目标 + FR-4 的"应该"优先级) | ||
| 61 | + | ||
| 62 | +- **FR-4(`verify_fn` 通用对接)不在本轮实现**:把评审团机制包装成一个可以传给 `run_graph(verify_fn=...)` 的工厂函数,用于升级 `_check_prd_consistency` 这类现有单一验证器场景——需求文档标注"应该"而非"必须",且"哪些验证点该升级成评审团"本身是待定的判定标准(需求文档第 8 节开放问题未回答)。本轮只交付 FR-5(根因溯源场景本身)这一个"必须"级别的真实落地点,`judge_root_cause_via_work_graph` 的函数签名设计上保持通用(视角 framing/数量可参数化),为未来接 FR-4 留出空间,但不在本轮消费它。 | ||
| 63 | +- **视角节点并发时间收益的实测**(验收标准 4):G 已实现,`topological_layers()` 会自动把三个视角节点分进同一层走 `ThreadPoolExecutor`,机制上必然并发;但"在真实 `claude -p` 调用下测出时间重叠"需要 `RUN_REAL_AGENT_E2E=1` 真实调用,不在 mock 测试范围内验证,留给后续真实运行观察确认(跟 G 自己的 `RunGraphConcurrencyTest` 用 mock+`time.sleep` 验证并发的方式已经覆盖了"机制确实并发"这件事,不需要为 F 重新证明一遍 G 的能力)。 | ||
| 64 | + | ||
| 65 | +## 7. 回答需求文档第 8 节开放问题 | ||
| 66 | + | ||
| 67 | +- `judge_root_cause`(闭包版)保留作为轻量备选,不废弃(第 4 节)。 | ||
| 68 | +- FR-4 接口设计:本轮不实现,见第 6 节。 | ||
| 69 | +- 除根因溯源外还有哪些验证点该升级:本轮不判定,留待 FR-4 一起决定。 | ||
| 70 | +- 视角数量 N 是否可配置:`judge_root_cause_via_work_graph`/`_tally_perspectives` 设计上已经是对任意数量视角通用的(不硬编码 3),根因溯源场景固定传 3 个,未来场景可以传不同数量而不需要改动这两个函数本身。 |
src/org_level_loop.py
| @@ -15,15 +15,18 @@ def run_fix_loop(conn, task_id, target_folder_path, bug_description, context, fi | @@ -15,15 +15,18 @@ def run_fix_loop(conn, task_id, target_folder_path, bug_description, context, fi | ||
| 15 | "verify_response"}. Raises root_cause_trace.InconclusiveEscalateError if | 15 | "verify_response"}. Raises root_cause_trace.InconclusiveEscalateError if |
| 16 | the 3 perspectives don't reach 2/3 agreement (same as U-015 -- this | 16 | the 3 perspectives don't reach 2/3 agreement (same as U-015 -- this |
| 17 | function doesn't add its own fallback for that, matching 04-P's own | 17 | function doesn't add its own fallback for that, matching 04-P's own |
| 18 | - "升级人工" answer to disagreement).""" | ||
| 19 | - perspective_fns = [ | ||
| 20 | - root_cause_trace.real_validator_perspective( | ||
| 21 | - name, bug_description, context, target_folder_path, | ||
| 22 | - trigger_fn=org_node_trigger.trigger_org_node, timeout=timeout, | ||
| 23 | - ) | ||
| 24 | - for name in ("correctness", "requirement_completeness", "skill_defect") | ||
| 25 | - ] | ||
| 26 | - judgment = root_cause_trace.judge_root_cause(perspective_fns) | 18 | + "升级人工" answer to disagreement). |
| 19 | + | ||
| 20 | + **F (2026-08-06, docs/todo/verifier-fan-out-jury-design.md)**: root | ||
| 21 | + cause judgment now runs as a real Work Graph fan-out (each perspective | ||
| 22 | + is its own Work Node with its own clean context) instead of three | ||
| 23 | + sequential Python closures calling org_node_trigger.trigger_org_node -- | ||
| 24 | + see root_cause_trace.judge_root_cause_via_work_graph. The fixer/verify | ||
| 25 | + calls below are unchanged flat trigger_org_node calls; only root-cause | ||
| 26 | + judgment moved.""" | ||
| 27 | + judgment = root_cause_trace.judge_root_cause_via_work_graph( | ||
| 28 | + conn, task_id, target_folder_path, bug_description, context, timeout=timeout, | ||
| 29 | + ) | ||
| 27 | category = judgment["root_cause_category"] | 30 | category = judgment["root_cause_category"] |
| 28 | route = _ROUTE_FOR_CATEGORY.get(category, "full_replan") | 31 | route = _ROUTE_FOR_CATEGORY.get(category, "full_replan") |
| 29 | 32 |
src/root_cause_trace.py
| @@ -14,11 +14,27 @@ perspectives 04-P names (æ£ç¡®æ€§è§†è§’/需求完整性视角/Skill ç³»ç»Ÿæ€§ç¼ | @@ -14,11 +14,27 @@ perspectives 04-P names (æ£ç¡®æ€§è§†è§’/需求完整性视角/Skill ç³»ç»Ÿæ€§ç¼ | ||
| 14 | concrete question each perspective asks is a prompt-engineering concern for | 14 | concrete question each perspective asks is a prompt-engineering concern for |
| 15 | the caller (see real_validator_perspective below for the real-call version), | 15 | the caller (see real_validator_perspective below for the real-call version), |
| 16 | not this module's. | 16 | not this module's. |
| 17 | + | ||
| 18 | +**F (2026-08-06, docs/todo/verifier-fan-out-jury-design.md)**: judge_root_cause | ||
| 19 | +above is the closure-based implementation -- each perspective is a Python | ||
| 20 | +function call, sequentially, in the SAME process/context. Kept as-is (not | ||
| 21 | +replaced): it's the cheap path for scenarios that don't need worktree/Work | ||
| 22 | +Graph overhead, and its already-proven >=2/3 rule is now shared (via | ||
| 23 | +_tally_perspectives) with judge_root_cause_via_work_graph below, the real | ||
| 24 | +Work Graph fan-out version -- each perspective becomes an independent Work | ||
| 25 | +Node (own claude -p call, own clean context, own execution_event log | ||
| 26 | +entries), scheduled by G's topological_layers()/run_graph() same as any | ||
| 27 | +other graph_manifest. The fan-in vote itself stays plain Python (a | ||
| 28 | +zero_token aggregation per the design-dynamic-work-graph Skill's own | ||
| 29 | +edge_type distinction -- counting doesn't need an Agent call), not a | ||
| 30 | +fourth Work Node. | ||
| 17 | """ | 31 | """ |
| 18 | 32 | ||
| 33 | +import json | ||
| 34 | +import uuid | ||
| 19 | from collections import Counter | 35 | from collections import Counter |
| 20 | 36 | ||
| 21 | -from src import bug_record | 37 | +from src import bug_record, work_graph_orchestrate |
| 22 | 38 | ||
| 23 | 39 | ||
| 24 | class InconclusiveEscalateError(Exception): | 40 | class InconclusiveEscalateError(Exception): |
| @@ -31,6 +47,19 @@ class InconclusiveEscalateError(Exception): | @@ -31,6 +47,19 @@ class InconclusiveEscalateError(Exception): | ||
| 31 | super().__init__("perspectives did not reach 2/3 agreement: " + repr(judgments)) | 47 | super().__init__("perspectives did not reach 2/3 agreement: " + repr(judgments)) |
| 32 | 48 | ||
| 33 | 49 | ||
| 50 | +def _tally_perspectives(judgments): | ||
| 51 | + """>=2/3 agreement rule (04-P: "至少 2/3 视角一致æ‰å®šæ¡ˆ") -- shared by both | ||
| 52 | + judge_root_cause (closures) and judge_root_cause_via_work_graph (Work | ||
| 53 | + Graph fan-out), so the rule is one piece of code, not two that could | ||
| 54 | + drift apart. `judgments`: list of {"category", "detail"}.""" | ||
| 55 | + counts = Counter(j["category"] for j in judgments) | ||
| 56 | + winner, count = counts.most_common(1)[0] | ||
| 57 | + if count * 2 < len(judgments): # strictly "at least 2/3", not just plurality | ||
| 58 | + raise InconclusiveEscalateError(judgments) | ||
| 59 | + detail = " | ".join(j["detail"] for j in judgments if j["category"] == winner) | ||
| 60 | + return {"root_cause_category": winner, "root_cause_detail": detail, "judgments": judgments} | ||
| 61 | + | ||
| 62 | + | ||
| 34 | def judge_root_cause(perspective_fns): | 63 | def judge_root_cause(perspective_fns): |
| 35 | """Runs all perspective_fns, requires >=2/3 to agree on the same | 64 | """Runs all perspective_fns, requires >=2/3 to agree on the same |
| 36 | category (04-P guard: "至少 2/3 视角一致æ‰å®šæ¡ˆ"). Returns | 65 | category (04-P guard: "至少 2/3 视角一致æ‰å®šæ¡ˆ"). Returns |
| @@ -38,12 +67,7 @@ def judge_root_cause(perspective_fns): | @@ -38,12 +67,7 @@ def judge_root_cause(perspective_fns): | ||
| 38 | raises InconclusiveEscalateError otherwise. | 67 | raises InconclusiveEscalateError otherwise. |
| 39 | """ | 68 | """ |
| 40 | judgments = [fn() for fn in perspective_fns] | 69 | judgments = [fn() for fn in perspective_fns] |
| 41 | - counts = Counter(j["category"] for j in judgments) | ||
| 42 | - winner, count = counts.most_common(1)[0] | ||
| 43 | - if count * 2 < len(judgments): # strictly "at least 2/3", not just plurality | ||
| 44 | - raise InconclusiveEscalateError(judgments) | ||
| 45 | - detail = " | ".join(j["detail"] for j in judgments if j["category"] == winner) | ||
| 46 | - return {"root_cause_category": winner, "root_cause_detail": detail, "judgments": judgments} | 70 | + return _tally_perspectives(judgments) |
| 47 | 71 | ||
| 48 | 72 | ||
| 49 | def trace_root_cause_test_failed(conn, task_id, description, perspective_fns, suggested_route, related_artifact_ids=None): | 73 | def trace_root_cause_test_failed(conn, task_id, description, perspective_fns, suggested_route, related_artifact_ids=None): |
| @@ -126,3 +150,86 @@ def real_validator_perspective(perspective_name, bug_description, context, targe | @@ -126,3 +150,86 @@ def real_validator_perspective(perspective_name, bug_description, context, targe | ||
| 126 | return {"category": category, "detail": detail or response} | 150 | return {"category": category, "detail": detail or response} |
| 127 | 151 | ||
| 128 | return perspective_fn | 152 | return perspective_fn |
| 153 | + | ||
| 154 | + | ||
| 155 | +_PERSPECTIVE_INPUT_SCHEMA = {"type": "object", "required": ["node_id"]} | ||
| 156 | +_PERSPECTIVE_OUTPUT_SCHEMA = {"type": "object", "required": ["category", "detail"]} | ||
| 157 | + | ||
| 158 | + | ||
| 159 | +def _perspective_prompt(perspective_name, bug_description, context): | ||
| 160 | + framing = _PERSPECTIVE_FRAMINGS[perspective_name] | ||
| 161 | + return ( | ||
| 162 | + "{framing}\n\nBug: {bug_description}\n\nContext:\n{context}\n\n" | ||
| 163 | + "Reply with your judgment: category must be exactly one of 'planning', 'coder', or " | ||
| 164 | + "'skill_defect'; detail is one sentence citing the specific SDD section or code " | ||
| 165 | + "location that supports it." | ||
| 166 | + ).format(framing=framing, bug_description=bug_description, context=context) | ||
| 167 | + | ||
| 168 | + | ||
| 169 | +def _node_id_for_perspective(name): | ||
| 170 | + return "perspective-" + name | ||
| 171 | + | ||
| 172 | + | ||
| 173 | +def _build_jury_graph_manifest(perspective_names): | ||
| 174 | + """FR-1: a reusable fan-out graph_manifest shape -- N mutually | ||
| 175 | + independent perspective nodes, generic over any set of perspective | ||
| 176 | + names (not hardcoded to root-cause's three). No fan-in node: see this | ||
| 177 | + module's own docstring / design doc §1 for why the vote stays plain | ||
| 178 | + Python (zero_token aggregation, not worth an Agent call).""" | ||
| 179 | + return { | ||
| 180 | + "nodes": [{"node_id": _node_id_for_perspective(name), "depends_on": []} for name in perspective_names], | ||
| 181 | + "edges": [], | ||
| 182 | + } | ||
| 183 | + | ||
| 184 | + | ||
| 185 | +def _jury_prompt_builder(perspective_prompts): | ||
| 186 | + """FR-2: perspective content is parameterized -- this scheduling code | ||
| 187 | + has no idea what the perspectives actually ask, it just runs whatever | ||
| 188 | + prompt each node was given.""" | ||
| 189 | + def build_prompt(node, outputs): | ||
| 190 | + name = node["node_id"][len("perspective-"):] | ||
| 191 | + return perspective_prompts[name] | ||
| 192 | + return build_prompt | ||
| 193 | + | ||
| 194 | + | ||
| 195 | +def judge_root_cause_via_work_graph( | ||
| 196 | + conn, task_id, target_folder_path, bug_description, context, | ||
| 197 | + org_node="validator", skill_id="qa-task-validation", timeout=300, | ||
| 198 | +): | ||
| 199 | + """Work Graph fan-out version of judge_root_cause (FR-5's real landing | ||
| 200 | + scenario) -- each perspective is an independent Work Node (own | ||
| 201 | + `claude -p` call with real schema-constrained structured output, own | ||
| 202 | + clean context -- work_graph_orchestrate._make_run_fn's dynamic- | ||
| 203 | + workflow-script mechanism, validated for real 2026-08-01, see that | ||
| 204 | + module's docstring) instead of a Python closure called in a loop, so | ||
| 205 | + unlike real_validator_perspective there's no 'CATEGORY:'/'DETAIL:' | ||
| 206 | + text parsing needed -- the node's output_payload already IS | ||
| 207 | + {"category", "detail"}. Same return shape as judge_root_cause, and | ||
| 208 | + raises InconclusiveEscalateError on <2/3 agreement via the same shared | ||
| 209 | + _tally_perspectives call -- callers don't need to know which | ||
| 210 | + implementation they're getting. | ||
| 211 | + | ||
| 212 | + No plan_work_graph() call: this graph's shape (N independent nodes) is | ||
| 213 | + fixed and known ahead of time, same reasoning e2e_pipeline.py's | ||
| 214 | + implement_code()/validate_implementation_with_own_graph() already use | ||
| 215 | + for skipping an LLM planning step of their own.""" | ||
| 216 | + perspective_prompts = { | ||
| 217 | + name: _perspective_prompt(name, bug_description, context) for name in _PERSPECTIVE_FRAMINGS | ||
| 218 | + } | ||
| 219 | + manifest = _build_jury_graph_manifest(list(perspective_prompts)) | ||
| 220 | + | ||
| 221 | + run_id = str(uuid.uuid4()) | ||
| 222 | + conn.execute( | ||
| 223 | + "INSERT INTO work_graph_run (run_id, task_id, org_node, graph_manifest, status) " | ||
| 224 | + "VALUES (?, ?, ?, ?, 'running')", | ||
| 225 | + (run_id, task_id, org_node, json.dumps(manifest)), | ||
| 226 | + ) | ||
| 227 | + conn.commit() | ||
| 228 | + | ||
| 229 | + outputs = work_graph_orchestrate.run_graph( | ||
| 230 | + conn, run_id, org_node, skill_id, target_folder_path, manifest, | ||
| 231 | + _jury_prompt_builder(perspective_prompts), _PERSPECTIVE_INPUT_SCHEMA, _PERSPECTIVE_OUTPUT_SCHEMA, | ||
| 232 | + timeout=timeout, | ||
| 233 | + ) | ||
| 234 | + judgments = [outputs[_node_id_for_perspective(name)] for name in perspective_prompts] | ||
| 235 | + return _tally_perspectives(judgments) |
src/scheduler.py
| @@ -193,7 +193,17 @@ def run_goal_to_completion(db_path, target_folder_path, goal_text): | @@ -193,7 +193,17 @@ def run_goal_to_completion(db_path, target_folder_path, goal_text): | ||
| 193 | task always lands on a real terminal-ish status with a real error | 193 | task always lands on a real terminal-ish status with a real error |
| 194 | detail, never a caller-crashes-and-the-task-is-just-stuck outcome. | 194 | detail, never a caller-crashes-and-the-task-is-just-stuck outcome. |
| 195 | """ | 195 | """ |
| 196 | - conn = schema.connect(db_path) | 196 | + # check_same_thread=False (2026-08-06, F): this conn can reach |
| 197 | + # work_graph_orchestrate.run_graph() via finish_task_lifecycle -> | ||
| 198 | + # org_level_loop.run_fix_loop -> root_cause_trace. | ||
| 199 | + # judge_root_cause_via_work_graph, whose 3-perspective layer runs on | ||
| 200 | + # its OWN ThreadPoolExecutor worker threads (G), not this function's | ||
| 201 | + # thread -- a plain schema.connect() here would raise "SQLite objects | ||
| 202 | + # created in a thread can only be used in that same thread" the first | ||
| 203 | + # time a real multi-node concurrent layer actually runs (previously | ||
| 204 | + # never exercised in production: every graph_manifest before F was | ||
| 205 | + # either single-node or a strictly serial chain). | ||
| 206 | + conn = schema.connect(db_path, check_same_thread=False) | ||
| 197 | conn.execute("PRAGMA busy_timeout = {}".format(_BUSY_TIMEOUT_MS)) | 207 | conn.execute("PRAGMA busy_timeout = {}".format(_BUSY_TIMEOUT_MS)) |
| 198 | try: | 208 | try: |
| 199 | project_id = conn.execute( | 209 | project_id = conn.execute( |
src/task_execution.py
| @@ -157,7 +157,13 @@ def start_task_execution_in_background(db_path, task_id, target_folder_path, goa | @@ -157,7 +157,13 @@ def start_task_execution_in_background(db_path, task_id, target_folder_path, goa | ||
| 157 | started one reaches the same real terminal state. Returns the Thread | 157 | started one reaches the same real terminal state. Returns the Thread |
| 158 | (already started) so callers/tests can `.join()` it if needed.""" | 158 | (already started) so callers/tests can `.join()` it if needed.""" |
| 159 | def run(): | 159 | def run(): |
| 160 | - conn = schema.connect(db_path) | 160 | + # check_same_thread=False (2026-08-06, F): same reasoning as |
| 161 | + # scheduler.run_goal_to_completion's own conn -- this thread's conn | ||
| 162 | + # can reach work_graph_orchestrate.run_graph()'s multi-node | ||
| 163 | + # concurrent layers (G) via finish_task_lifecycle -> run_fix_loop -> | ||
| 164 | + # judge_root_cause_via_work_graph, whose worker threads aren't this | ||
| 165 | + # one. | ||
| 166 | + conn = schema.connect(db_path, check_same_thread=False) | ||
| 161 | try: | 167 | try: |
| 162 | result = run_or_resume_task_lifecycle(conn, task_id, target_folder_path, goal_text) | 168 | result = run_or_resume_task_lifecycle(conn, task_id, target_folder_path, goal_text) |
| 163 | scheduler.finish_task_lifecycle(conn, result, target_folder_path, goal_text) | 169 | scheduler.finish_task_lifecycle(conn, result, target_folder_path, goal_text) |
src/test_org_level_loop.py
| @@ -3,23 +3,35 @@ fix -> re-verify -> done), mocked at the subprocess boundary (CLAUDE.md | @@ -3,23 +3,35 @@ fix -> re-verify -> done), mocked at the subprocess boundary (CLAUDE.md | ||
| 3 | Testing constraint). Real coverage is in src/test_org_level_loop_e2e.py | 3 | Testing constraint). Real coverage is in src/test_org_level_loop_e2e.py |
| 4 | (RUN_REAL_AGENT_E2E=1). | 4 | (RUN_REAL_AGENT_E2E=1). |
| 5 | 5 | ||
| 6 | +**F (2026-08-06)**: root-cause judgment now runs through | ||
| 7 | +root_cause_trace.judge_root_cause_via_work_graph (a real Work Graph fan-out, | ||
| 8 | +see docs/todo/verifier-fan-out-jury-design.md), so the 3 perspective calls | ||
| 9 | +are mocked at work_graph_execute.write_workflow_script/execute_work_graph | ||
| 10 | +(same boundary as test_work_graph_orchestrate.py), not at | ||
| 11 | +org_node_trigger.trigger_org_node -- that mock is now only used for the | ||
| 12 | +fixer/verify calls, which are unchanged. | ||
| 13 | + | ||
| 6 | Run: python3 -m unittest src.test_org_level_loop -v | 14 | Run: python3 -m unittest src.test_org_level_loop -v |
| 7 | """ | 15 | """ |
| 8 | 16 | ||
| 9 | import unittest | 17 | import unittest |
| 10 | from unittest.mock import patch | 18 | from unittest.mock import patch |
| 11 | 19 | ||
| 12 | -from src import org_level_loop, schema | 20 | +from src import org_level_loop, root_cause_trace, schema, work_graph_execute |
| 21 | + | ||
| 13 | 22 | ||
| 23 | +def _make_perspective_execute(perspective_category): | ||
| 24 | + """Fakes work_graph_execute.execute_work_graph for the 3 perspective | ||
| 25 | + Work Nodes -- all agree on perspective_category, real schema-shaped | ||
| 26 | + output (no more 'CATEGORY:'/'DETAIL:' text parsing).""" | ||
| 27 | + def fake_execute(cwd, name, args, output_schema, timeout=300): | ||
| 28 | + return {"category": perspective_category, "detail": "because reasons"} | ||
| 29 | + return fake_execute | ||
| 14 | 30 | ||
| 15 | -def _make_trigger(perspective_category, fix_ok=True): | ||
| 16 | - """Fakes org_node_trigger.trigger_org_node for the whole loop: 3 | ||
| 17 | - perspective calls (all agreeing on perspective_category), one fixer | ||
| 18 | - call, one verify call.""" | ||
| 19 | 31 | ||
| 32 | +def _make_trigger(fix_ok=True): | ||
| 33 | + """Fakes org_node_trigger.trigger_org_node for the fixer/verify calls only.""" | ||
| 20 | def fake(org_node, prompt, target_folder_path, timeout=300): | 34 | def fake(org_node, prompt, target_folder_path, timeout=300): |
| 21 | - if "CATEGORY:" in prompt: | ||
| 22 | - return "CATEGORY: {}\nDETAIL: because reasons".format(perspective_category) | ||
| 23 | if org_node in ("coder", "planner"): | 35 | if org_node in ("coder", "planner"): |
| 24 | return "Fixed it." | 36 | return "Fixed it." |
| 25 | if org_node == "validator": | 37 | if org_node == "validator": |
| @@ -31,7 +43,9 @@ def _make_trigger(perspective_category, fix_ok=True): | @@ -31,7 +43,9 @@ def _make_trigger(perspective_category, fix_ok=True): | ||
| 31 | 43 | ||
| 32 | class RunFixLoopTestBase(unittest.TestCase): | 44 | class RunFixLoopTestBase(unittest.TestCase): |
| 33 | def setUp(self): | 45 | def setUp(self): |
| 34 | - self.conn = schema.connect(":memory:") | 46 | + # check_same_thread=False: the 3 root-cause perspective nodes form |
| 47 | + # one concurrent layer under G, run via ThreadPoolExecutor. | ||
| 48 | + self.conn = schema.connect(":memory:", check_same_thread=False) | ||
| 35 | schema.create_schema(self.conn) | 49 | schema.create_schema(self.conn) |
| 36 | self.conn.execute( | 50 | self.conn.execute( |
| 37 | "INSERT INTO project (project_id, name, target_folder_path, has_git_repo, status) " | 51 | "INSERT INTO project (project_id, name, target_folder_path, has_git_repo, status) " |
| @@ -46,13 +60,21 @@ class RunFixLoopTestBase(unittest.TestCase): | @@ -46,13 +60,21 @@ class RunFixLoopTestBase(unittest.TestCase): | ||
| 46 | def tearDown(self): | 60 | def tearDown(self): |
| 47 | self.conn.close() | 61 | self.conn.close() |
| 48 | 62 | ||
| 63 | + def _run_fix_loop(self, perspective_category, fix_ok=True): | ||
| 64 | + with patch.object(work_graph_execute, "write_workflow_script"), \ | ||
| 65 | + patch.object( | ||
| 66 | + work_graph_execute, "execute_work_graph", | ||
| 67 | + side_effect=_make_perspective_execute(perspective_category), | ||
| 68 | + ), \ | ||
| 69 | + patch("src.org_node_trigger.trigger_org_node", side_effect=_make_trigger(fix_ok=fix_ok)): | ||
| 70 | + return org_level_loop.run_fix_loop( | ||
| 71 | + self.conn, "task-1", "/tmp/proj-1", "bug desc", "context", "fix it", "verify it" | ||
| 72 | + ) | ||
| 73 | + | ||
| 49 | 74 | ||
| 50 | class CoderCategoryFastLaneTest(RunFixLoopTestBase): | 75 | class CoderCategoryFastLaneTest(RunFixLoopTestBase): |
| 51 | def test_coder_category_routes_fast_lane_and_reaches_done(self): | 76 | def test_coder_category_routes_fast_lane_and_reaches_done(self): |
| 52 | - with patch("src.org_node_trigger.trigger_org_node", side_effect=_make_trigger("coder", fix_ok=True)): | ||
| 53 | - result = org_level_loop.run_fix_loop( | ||
| 54 | - self.conn, "task-1", "/tmp/proj-1", "bug desc", "context", "fix it", "verify it" | ||
| 55 | - ) | 77 | + result = self._run_fix_loop("coder", fix_ok=True) |
| 56 | self.assertEqual(result["root_cause_category"], "coder") | 78 | self.assertEqual(result["root_cause_category"], "coder") |
| 57 | self.assertEqual(result["route"], "fast_lane") | 79 | self.assertEqual(result["route"], "fast_lane") |
| 58 | self.assertTrue(result["verified"]) | 80 | self.assertTrue(result["verified"]) |
| @@ -69,10 +91,7 @@ class CoderCategoryFastLaneTest(RunFixLoopTestBase): | @@ -69,10 +91,7 @@ class CoderCategoryFastLaneTest(RunFixLoopTestBase): | ||
| 69 | 91 | ||
| 70 | class PlanningCategoryFullReplanTest(RunFixLoopTestBase): | 92 | class PlanningCategoryFullReplanTest(RunFixLoopTestBase): |
| 71 | def test_planning_category_routes_full_replan(self): | 93 | def test_planning_category_routes_full_replan(self): |
| 72 | - with patch("src.org_node_trigger.trigger_org_node", side_effect=_make_trigger("planning", fix_ok=True)): | ||
| 73 | - result = org_level_loop.run_fix_loop( | ||
| 74 | - self.conn, "task-1", "/tmp/proj-1", "bug desc", "context", "fix it", "verify it" | ||
| 75 | - ) | 94 | + result = self._run_fix_loop("planning", fix_ok=True) |
| 76 | self.assertEqual(result["route"], "full_replan") | 95 | self.assertEqual(result["route"], "full_replan") |
| 77 | self.assertEqual(result["final_status"], "done") | 96 | self.assertEqual(result["final_status"], "done") |
| 78 | assigned = self.conn.execute("SELECT assigned_org_node FROM task WHERE task_id = 'task-1'").fetchone()[0] | 97 | assigned = self.conn.execute("SELECT assigned_org_node FROM task WHERE task_id = 'task-1'").fetchone()[0] |
| @@ -83,15 +102,36 @@ class PlanningCategoryFullReplanTest(RunFixLoopTestBase): | @@ -83,15 +102,36 @@ class PlanningCategoryFullReplanTest(RunFixLoopTestBase): | ||
| 83 | 102 | ||
| 84 | class VerificationFailsTest(RunFixLoopTestBase): | 103 | class VerificationFailsTest(RunFixLoopTestBase): |
| 85 | def test_fix_not_verified_leaves_task_out_of_done(self): | 104 | def test_fix_not_verified_leaves_task_out_of_done(self): |
| 86 | - with patch("src.org_node_trigger.trigger_org_node", side_effect=_make_trigger("coder", fix_ok=False)): | ||
| 87 | - result = org_level_loop.run_fix_loop( | ||
| 88 | - self.conn, "task-1", "/tmp/proj-1", "bug desc", "context", "fix it", "verify it" | ||
| 89 | - ) | 105 | + result = self._run_fix_loop("coder", fix_ok=False) |
| 90 | self.assertFalse(result["verified"]) | 106 | self.assertFalse(result["verified"]) |
| 91 | self.assertNotEqual(result["final_status"], "done") | 107 | self.assertNotEqual(result["final_status"], "done") |
| 92 | status = self.conn.execute("SELECT status FROM task WHERE task_id = 'task-1'").fetchone()[0] | 108 | status = self.conn.execute("SELECT status FROM task WHERE task_id = 'task-1'").fetchone()[0] |
| 93 | self.assertNotEqual(status, "done") | 109 | self.assertNotEqual(status, "done") |
| 94 | 110 | ||
| 95 | 111 | ||
| 112 | +class InconclusiveDisagreementTest(RunFixLoopTestBase): | ||
| 113 | + """Real Work Graph fan-out still surfaces InconclusiveEscalateError the | ||
| 114 | + same way the closure-based version did -- run_fix_loop adds no fallback | ||
| 115 | + of its own (04-P: disagreement escalates to a human, it isn't silently | ||
| 116 | + routed anywhere).""" | ||
| 117 | + | ||
| 118 | + def test_three_way_split_raises_and_creates_no_bug_record(self): | ||
| 119 | + categories = iter(["planning", "coder", "skill_defect"]) | ||
| 120 | + | ||
| 121 | + def fake_execute(cwd, name, args, output_schema, timeout=300): | ||
| 122 | + return {"category": next(categories), "detail": "d"} | ||
| 123 | + | ||
| 124 | + with patch.object(work_graph_execute, "write_workflow_script"), \ | ||
| 125 | + patch.object(work_graph_execute, "execute_work_graph", side_effect=fake_execute), \ | ||
| 126 | + patch("src.org_node_trigger.trigger_org_node", side_effect=_make_trigger(fix_ok=True)): | ||
| 127 | + with self.assertRaises(root_cause_trace.InconclusiveEscalateError): | ||
| 128 | + org_level_loop.run_fix_loop( | ||
| 129 | + self.conn, "task-1", "/tmp/proj-1", "bug desc", "context", "fix it", "verify it" | ||
| 130 | + ) | ||
| 131 | + | ||
| 132 | + count = self.conn.execute("SELECT COUNT(*) FROM bug_record").fetchone()[0] | ||
| 133 | + self.assertEqual(count, 0) | ||
| 134 | + | ||
| 135 | + | ||
| 96 | if __name__ == "__main__": | 136 | if __name__ == "__main__": |
| 97 | unittest.main() | 137 | unittest.main() |
src/test_root_cause_trace.py
| @@ -6,9 +6,15 @@ coverage is in src/test_root_cause_trace_e2e.py (RUN_REAL_AGENT_E2E=1). | @@ -6,9 +6,15 @@ coverage is in src/test_root_cause_trace_e2e.py (RUN_REAL_AGENT_E2E=1). | ||
| 6 | Run: python3 -m unittest src.test_root_cause_trace -v | 6 | Run: python3 -m unittest src.test_root_cause_trace -v |
| 7 | """ | 7 | """ |
| 8 | 8 | ||
| 9 | +import json | ||
| 10 | +import os | ||
| 11 | +import tempfile | ||
| 12 | +import threading | ||
| 13 | +import time | ||
| 9 | import unittest | 14 | import unittest |
| 15 | +from unittest.mock import patch | ||
| 10 | 16 | ||
| 11 | -from src import bug_record, root_cause_trace, schema | 17 | +from src import bug_record, root_cause_trace, schema, work_graph_execute |
| 12 | 18 | ||
| 13 | 19 | ||
| 14 | def _fixed(category, detail="detail"): | 20 | def _fixed(category, detail="detail"): |
| @@ -128,5 +134,180 @@ class RealValidatorPerspectiveParsingTest(unittest.TestCase): | @@ -128,5 +134,180 @@ class RealValidatorPerspectiveParsingTest(unittest.TestCase): | ||
| 128 | perspective_fn() | 134 | perspective_fn() |
| 129 | 135 | ||
| 130 | 136 | ||
| 137 | +# F (2026-08-06): Work Graph fan-out version -- each perspective becomes a | ||
| 138 | +# real Work Node (mocked at the claude subprocess boundary, same as | ||
| 139 | +# test_work_graph_orchestrate.py) instead of a Python closure. | ||
| 140 | + | ||
| 141 | +class JudgeRootCauseViaWorkGraphTestBase(unittest.TestCase): | ||
| 142 | + def setUp(self): | ||
| 143 | + self.home_tmp = tempfile.TemporaryDirectory() | ||
| 144 | + self.addCleanup(self.home_tmp.cleanup) | ||
| 145 | + self.old_graphlemon_home = os.environ.get("GRAPHLEMON_HOME") | ||
| 146 | + os.environ["GRAPHLEMON_HOME"] = self.home_tmp.name | ||
| 147 | + self.addCleanup(self._restore_graphlemon_home) | ||
| 148 | + | ||
| 149 | + # check_same_thread=False: the three perspective nodes form one | ||
| 150 | + # concurrent layer under G, run via ThreadPoolExecutor. | ||
| 151 | + self.conn = schema.connect(":memory:", check_same_thread=False) | ||
| 152 | + schema.create_schema(self.conn) | ||
| 153 | + self.conn.execute( | ||
| 154 | + "INSERT INTO project (project_id, name, target_folder_path, has_git_repo, status) " | ||
| 155 | + "VALUES ('proj-1', 'P', '/tmp/proj-1', 1, 'active')" | ||
| 156 | + ) | ||
| 157 | + self.conn.execute( | ||
| 158 | + "INSERT INTO task (task_id, project_id, goal_text, task_type, status, assigned_org_node) " | ||
| 159 | + "VALUES ('task-1', 'proj-1', 'goal', 'goal', 'validating', 'validator')" | ||
| 160 | + ) | ||
| 161 | + self.conn.commit() | ||
| 162 | + | ||
| 163 | + def _restore_graphlemon_home(self): | ||
| 164 | + if self.old_graphlemon_home is None: | ||
| 165 | + os.environ.pop("GRAPHLEMON_HOME", None) | ||
| 166 | + else: | ||
| 167 | + os.environ["GRAPHLEMON_HOME"] = self.old_graphlemon_home | ||
| 168 | + | ||
| 169 | + def tearDown(self): | ||
| 170 | + self.conn.close() | ||
| 171 | + | ||
| 172 | + def _perspective_for_script(self, prompt_by_script, script_name): | ||
| 173 | + content = prompt_by_script[script_name] | ||
| 174 | + for name, framing in root_cause_trace._PERSPECTIVE_FRAMINGS.items(): | ||
| 175 | + if framing in content: | ||
| 176 | + return name | ||
| 177 | + raise AssertionError("could not identify perspective for script " + script_name) | ||
| 178 | + | ||
| 179 | + | ||
| 180 | +class RealWorkNodesPerPerspectiveTest(JudgeRootCauseViaWorkGraphTestBase): | ||
| 181 | + """Acceptance criterion 1: three independent Work Node executions (real | ||
| 182 | + work_node_execution rows, real execution_event logs), not three closure | ||
| 183 | + calls inside one function.""" | ||
| 184 | + | ||
| 185 | + def test_three_perspectives_are_three_real_work_node_rows(self): | ||
| 186 | + prompt_by_script = {} | ||
| 187 | + | ||
| 188 | + def fake_write_script(cwd, script_name, script_content): | ||
| 189 | + prompt_by_script[script_name] = script_content | ||
| 190 | + | ||
| 191 | + def fake_execute(cwd, name, args, output_schema, timeout=600): | ||
| 192 | + perspective = self._perspective_for_script(prompt_by_script, name) | ||
| 193 | + return {"category": "coder", "detail": "agreed via " + perspective} | ||
| 194 | + | ||
| 195 | + with patch.object(work_graph_execute, "write_workflow_script", side_effect=fake_write_script), \ | ||
| 196 | + patch.object(work_graph_execute, "execute_work_graph", side_effect=fake_execute): | ||
| 197 | + result = root_cause_trace.judge_root_cause_via_work_graph( | ||
| 198 | + self.conn, "task-1", "/tmp/proj-1", "bug desc", "context", | ||
| 199 | + ) | ||
| 200 | + | ||
| 201 | + self.assertEqual(result["root_cause_category"], "coder") | ||
| 202 | + self.assertEqual(len(result["judgments"]), 3) | ||
| 203 | + | ||
| 204 | + rows = self.conn.execute( | ||
| 205 | + "SELECT node_id, status, output_payload FROM work_node_execution WHERE run_id = (" | ||
| 206 | + " SELECT run_id FROM work_graph_run WHERE task_id = 'task-1'" | ||
| 207 | + ")" | ||
| 208 | + ).fetchall() | ||
| 209 | + self.assertEqual( | ||
| 210 | + {r[0] for r in rows}, | ||
| 211 | + {"perspective-correctness", "perspective-requirement_completeness", "perspective-skill_defect"}, | ||
| 212 | + ) | ||
| 213 | + for node_id, status, output_payload in rows: | ||
| 214 | + self.assertEqual(status, "done") | ||
| 215 | + self.assertEqual(json.loads(output_payload)["category"], "coder") | ||
| 216 | + | ||
| 217 | + | ||
| 218 | +class JudgmentRuleEquivalenceTest(JudgeRootCauseViaWorkGraphTestBase): | ||
| 219 | + """Acceptance criterion 2: same >=2/3 rule, same results, as the | ||
| 220 | + closure-based judge_root_cause on equivalent inputs -- both call the | ||
| 221 | + same shared _tally_perspectives.""" | ||
| 222 | + | ||
| 223 | + def _run_with_categories(self, categories_by_perspective): | ||
| 224 | + prompt_by_script = {} | ||
| 225 | + | ||
| 226 | + def fake_write_script(cwd, script_name, script_content): | ||
| 227 | + prompt_by_script[script_name] = script_content | ||
| 228 | + | ||
| 229 | + def fake_execute(cwd, name, args, output_schema, timeout=600): | ||
| 230 | + perspective = self._perspective_for_script(prompt_by_script, name) | ||
| 231 | + return {"category": categories_by_perspective[perspective], "detail": "d-" + perspective} | ||
| 232 | + | ||
| 233 | + with patch.object(work_graph_execute, "write_workflow_script", side_effect=fake_write_script), \ | ||
| 234 | + patch.object(work_graph_execute, "execute_work_graph", side_effect=fake_execute): | ||
| 235 | + return root_cause_trace.judge_root_cause_via_work_graph( | ||
| 236 | + self.conn, "task-1", "/tmp/proj-1", "bug desc", "context", | ||
| 237 | + ) | ||
| 238 | + | ||
| 239 | + def test_two_of_three_agreement_wins(self): | ||
| 240 | + result = self._run_with_categories( | ||
| 241 | + {"correctness": "planning", "requirement_completeness": "planning", "skill_defect": "coder"} | ||
| 242 | + ) | ||
| 243 | + self.assertEqual(result["root_cause_category"], "planning") | ||
| 244 | + | ||
| 245 | + closure_result = root_cause_trace.judge_root_cause( | ||
| 246 | + [_fixed("planning"), _fixed("planning"), _fixed("coder")] | ||
| 247 | + ) | ||
| 248 | + self.assertEqual(result["root_cause_category"], closure_result["root_cause_category"]) | ||
| 249 | + | ||
| 250 | + def test_three_way_split_is_inconclusive(self): | ||
| 251 | + with self.assertRaises(root_cause_trace.InconclusiveEscalateError): | ||
| 252 | + self._run_with_categories( | ||
| 253 | + {"correctness": "planning", "requirement_completeness": "coder", "skill_defect": "skill_defect"} | ||
| 254 | + ) | ||
| 255 | + with self.assertRaises(root_cause_trace.InconclusiveEscalateError): | ||
| 256 | + root_cause_trace.judge_root_cause([_fixed("planning"), _fixed("coder"), _fixed("skill_defect")]) | ||
| 257 | + | ||
| 258 | + | ||
| 259 | +class IndependentCleanContextTest(JudgeRootCauseViaWorkGraphTestBase): | ||
| 260 | + """Acceptance criterion 3: each perspective node's input_payload only | ||
| 261 | + carries what it needs to judge -- not other perspectives' reasoning.""" | ||
| 262 | + | ||
| 263 | + def test_input_payload_has_no_cross_perspective_content(self): | ||
| 264 | + with patch.object(work_graph_execute, "write_workflow_script"), \ | ||
| 265 | + patch.object( | ||
| 266 | + work_graph_execute, "execute_work_graph", | ||
| 267 | + return_value={"category": "coder", "detail": "d"}, | ||
| 268 | + ): | ||
| 269 | + root_cause_trace.judge_root_cause_via_work_graph( | ||
| 270 | + self.conn, "task-1", "/tmp/proj-1", "bug desc", "context", | ||
| 271 | + ) | ||
| 272 | + | ||
| 273 | + rows = self.conn.execute("SELECT node_id, input_payload FROM work_node_execution").fetchall() | ||
| 274 | + self.assertEqual(len(rows), 3) | ||
| 275 | + for node_id, input_payload in rows: | ||
| 276 | + payload = json.loads(input_payload) | ||
| 277 | + self.assertEqual(payload, {"node_id": node_id}) | ||
| 278 | + | ||
| 279 | + | ||
| 280 | +class ConcurrentPerspectiveExecutionTest(JudgeRootCauseViaWorkGraphTestBase): | ||
| 281 | + """G is already in place -- three mutually-independent perspective | ||
| 282 | + nodes land in one topological layer, so they run concurrently, not | ||
| 283 | + queued one after another.""" | ||
| 284 | + | ||
| 285 | + def test_three_perspective_calls_overlap_in_wall_clock_time(self): | ||
| 286 | + events = [] | ||
| 287 | + events_lock = threading.Lock() | ||
| 288 | + | ||
| 289 | + def fake_execute(cwd, name, args, output_schema, timeout=600): | ||
| 290 | + start = time.monotonic() | ||
| 291 | + time.sleep(0.05) | ||
| 292 | + end = time.monotonic() | ||
| 293 | + with events_lock: | ||
| 294 | + events.append((start, end)) | ||
| 295 | + return {"category": "coder", "detail": "d"} | ||
| 296 | + | ||
| 297 | + with patch.object(work_graph_execute, "write_workflow_script"), \ | ||
| 298 | + patch.object(work_graph_execute, "execute_work_graph", side_effect=fake_execute): | ||
| 299 | + root_cause_trace.judge_root_cause_via_work_graph( | ||
| 300 | + self.conn, "task-1", "/tmp/proj-1", "bug desc", "context", | ||
| 301 | + ) | ||
| 302 | + | ||
| 303 | + self.assertEqual(len(events), 3) | ||
| 304 | + # at least two of the three calls must have overlapped | ||
| 305 | + overlaps = sum( | ||
| 306 | + 1 for i in range(len(events)) for j in range(i + 1, len(events)) | ||
| 307 | + if events[i][0] < events[j][1] and events[j][0] < events[i][1] | ||
| 308 | + ) | ||
| 309 | + self.assertGreater(overlaps, 0) | ||
| 310 | + | ||
| 311 | + | ||
| 131 | if __name__ == "__main__": | 312 | if __name__ == "__main__": |
| 132 | unittest.main() | 313 | unittest.main() |
src/test_scheduler.py
| @@ -377,10 +377,10 @@ class UnroutedExceptionStillBlocksTaskTest(RoutableExceptionTestBase): | @@ -377,10 +377,10 @@ class UnroutedExceptionStillBlocksTaskTest(RoutableExceptionTestBase): | ||
| 377 | 377 | ||
| 378 | def _fix_loop_trigger(org_node, prompt, target_folder_path, timeout=300): | 378 | def _fix_loop_trigger(org_node, prompt, target_folder_path, timeout=300): |
| 379 | """org_node_trigger.trigger_org_node fake for org_level_loop.run_fix_loop's | 379 | """org_node_trigger.trigger_org_node fake for org_level_loop.run_fix_loop's |
| 380 | - 3 perspective calls (all agree 'coder') + fixer + verify -- verify always | ||
| 381 | - fails, so route_org_loop keeps incrementing loop_round on retries.""" | ||
| 382 | - if "CATEGORY:" in prompt: | ||
| 383 | - return "CATEGORY: coder\nDETAIL: because reasons" | 380 | + fixer + verify calls -- verify always fails, so route_org_loop keeps |
| 381 | + incrementing loop_round on retries. Root-cause judgment itself (F, | ||
| 382 | + 2026-08-06) no longer goes through trigger_org_node -- see | ||
| 383 | + _fix_loop_perspective_execute below for that boundary.""" | ||
| 384 | if org_node == "coder": | 384 | if org_node == "coder": |
| 385 | return "Fixed it." | 385 | return "Fixed it." |
| 386 | if org_node == "validator": | 386 | if org_node == "validator": |
| @@ -388,6 +388,12 @@ def _fix_loop_trigger(org_node, prompt, target_folder_path, timeout=300): | @@ -388,6 +388,12 @@ def _fix_loop_trigger(org_node, prompt, target_folder_path, timeout=300): | ||
| 388 | raise AssertionError("unexpected call: " + org_node) | 388 | raise AssertionError("unexpected call: " + org_node) |
| 389 | 389 | ||
| 390 | 390 | ||
| 391 | +def _fix_loop_perspective_execute(cwd, name, args, output_schema, timeout=300): | ||
| 392 | + """work_graph_execute.execute_work_graph fake for run_fix_loop's 3 | ||
| 393 | + root-cause perspective Work Nodes (F) -- all agree 'coder'.""" | ||
| 394 | + return {"category": "coder", "detail": "because reasons"} | ||
| 395 | + | ||
| 396 | + | ||
| 391 | class SchedulerContinuesPastBlockedTaskTest(unittest.TestCase): | 397 | class SchedulerContinuesPastBlockedTaskTest(unittest.TestCase): |
| 392 | """REQ-05 failure / REQ-09 boundary, against the REAL scheduler (U-032) -- | 398 | """REQ-05 failure / REQ-09 boundary, against the REAL scheduler (U-032) -- |
| 393 | test_org_loop_route.py's own test for this literally says "no scheduler | 399 | test_org_loop_route.py's own test for this literally says "no scheduler |
| @@ -430,7 +436,9 @@ class SchedulerContinuesPastBlockedTaskTest(unittest.TestCase): | @@ -430,7 +436,9 @@ class SchedulerContinuesPastBlockedTaskTest(unittest.TestCase): | ||
| 430 | goals = [("/tmp/proj-1", "goal-ok-1"), ("/tmp/proj-1", "goal-doomed"), ("/tmp/proj-1", "goal-ok-2")] | 436 | goals = [("/tmp/proj-1", "goal-ok-1"), ("/tmp/proj-1", "goal-doomed"), ("/tmp/proj-1", "goal-ok-2")] |
| 431 | with patch.object(scheduler.e2e_pipeline, "run_task_lifecycle", side_effect=fake_run_task_lifecycle), \ | 437 | with patch.object(scheduler.e2e_pipeline, "run_task_lifecycle", side_effect=fake_run_task_lifecycle), \ |
| 432 | patch.object(scheduler.e2e_pipeline, "_read_artifact_content", return_value="benchmark content"), \ | 438 | patch.object(scheduler.e2e_pipeline, "_read_artifact_content", return_value="benchmark content"), \ |
| 433 | - patch("src.org_node_trigger.trigger_org_node", side_effect=_fix_loop_trigger): | 439 | + patch("src.org_node_trigger.trigger_org_node", side_effect=_fix_loop_trigger), \ |
| 440 | + patch.object(work_graph_execute, "write_workflow_script"), \ | ||
| 441 | + patch.object(work_graph_execute, "execute_work_graph", side_effect=_fix_loop_perspective_execute): | ||
| 434 | results = scheduler.run_scheduler_once(self.db_path, goals, max_workers=3) | 442 | results = scheduler.run_scheduler_once(self.db_path, goals, max_workers=3) |
| 435 | 443 | ||
| 436 | by_goal = {r["goal_text"]: r for r in results} | 444 | by_goal = {r["goal_text"]: r for r in results} |