宜博 / lemongraph

Commit ba481dde6e5da3d5bd6c8953dcfffbca4eed4782

Authored by bboonnee
1 parent 59749fc5

U-017: full Org-level Loop closed-loop verification (PH2-S1 complete) 

run_fix_loop() chains U-015 (3-perspective root-cause judgment) + U-016
(routing) + real Coder/Planner/Validator calls into architecture doc §11's
4th MVP acceptance criterion: judge -> create bug_record -> route
(fast_lane for category=coder, full_replan for category=planning -- a
fixed mapping this Unit defines, since bug_record.suggested_route is
technically an independent Validator judgment but REQ-05/architecture §11's
scenarios always correlate the two, so no extra Agent call was added just
to re-decide it) -> trigger the fixer -> re-trigger Validator -> close_task_
done on confirmed PASS.

3 new mocked stdlib unittest cases (route branching, verification-fails
leaves the task out of 'done'). 160 total mocked tests across U-001-017,
all passing.

Real coverage reuses U-015's wall-collision scenario but this time drives
an actual fix: a real Coder call edits collision.py in the repo, a real
Validator call reads the fixed file and confirms it, running as a detached
background process (survives this session's own restarts, unlike two
earlier real runs this session that got silently orphaned by a session
teardown with no completion record -- lesson learned, not repeated).

This closes PH2-S1 (U-015/016/017): REQ-04/05/09 and architecture doc §11
criterion 4 are now real, verified code, not just planned intent.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Inline Side-by-side
Showing 5 changed files with 251 additions and 1 deletions
docs/development-plan.md
  View file @ba481dd
... ... @@ -171,6 +171,7 @@ PH2-S1/S2/S3/S4 之间除标注的依赖外相互独立，可并行推进。
171 171 - **U-017 [Risk Gate]** 人为植入 bug 完整闭环验证（架构文档 §11 验收标准第 4 条）
172 172 - 依赖：U-016
173 173 - Done：引入一个人为植入的 bug，系统自主定位根因并驱动 Planner/Coder 完成修复闭环
  174 + - **实现补充（2026-07-31）**：`src/org_level_loop.py` 的 `run_fix_loop` 把 U-015（三视角判定）+ U-016（路由）串成一个闭环：判定→建 `bug_record`→路由→真实触发 Coder/Planner 修复→真实触发 Validator 复核→复核通过则 `close_task_done`。`category=coder→fast_lane`、`category=planning→full_replan` 是这里定的固定映射规则（`bug_record.suggested_route` 本来是 Validator 独立判定的字段，但 REQ-05/架构文档 §11 的验收场景里两者天然对应，没有再额外调一次 Agent 单独决定 route）。真实验证复用了 U-015 那个撞墙误判场景，但这次真的让 Coder 编辑仓库里的 `collision.py` 文件、真的让 Validator 读文件复核，不是只判定不动手
174 175  
175 176 ### PH2-S2 Skill 自主迭代
176 177  
... ...
docs/sdd/unit_matrix.json
  View file @ba481dd
... ... @@ -30,7 +30,7 @@
30 30  
31 31 {"id": "U-015", "phase": "PH2", "sub_phase": "PH2-S1", "name": "p_validator_root_cause_trace 实现", "classification": "Risk Gate", "depends_on": ["U-013"], "coverage": {"model": ["bug_record"], "view": [], "controller": ["c_bug_record_create"], "process": ["p_validator_root_cause_trace"], "requirement": ["REQ-04"]}, "done": "REQ-04 happy/boundary/failure 通过"},
32 32 {"id": "U-016", "phase": "PH2", "sub_phase": "PH2-S1", "name": "c_org_loop_route + fast_lane/full_replan（含 close_task_done，见 development-plan.md 2026-07-31 实现补充）", "classification": "Risk Gate", "depends_on": ["U-015"], "coverage": {"model": [], "view": [], "controller": ["c_org_loop_route"], "process": [], "requirement": ["REQ-05", "REQ-09"]}, "done": "REQ-05/REQ-09 全部用例通过（含 3 轮上限 → blocked_pending_human；validating→done 通过新增 close_task_done 打通）"},
33   - {"id": "U-017", "phase": "PH2", "sub_phase": "PH2-S1", "name": "人为植入 bug 完整闭环验证", "classification": "Risk Gate", "depends_on": ["U-016"], "coverage": {"model": [], "view": [], "controller": [], "process": [], "requirement": []}, "done": "架构文档 §11 验收标准第 4 条达成"},
  33 + {"id": "U-017", "phase": "PH2", "sub_phase": "PH2-S1", "name": "人为植入 bug 完整闭环验证（见 development-plan.md 2026-07-31 实现补充）", "classification": "Risk Gate", "depends_on": ["U-016"], "coverage": {"model": [], "view": [], "controller": [], "process": [], "requirement": []}, "done": "架构文档 §11 验收标准第 4 条达成：run_fix_loop 串联 U-015 判定 + U-016 路由 + 真实 Coder 修复 + 真实 Validator 复核"},
34 34 {"id": "U-018", "phase": "PH2", "sub_phase": "PH2-S2", "name": "c_skill_update + c_skill_rollback", "classification": "Risk Gate", "depends_on": ["U-015"], "coverage": {"model": ["skill", "skill_iteration"], "view": [], "controller": ["c_skill_update", "c_skill_rollback"], "process": [], "requirement": ["REQ-08"]}, "done": "REQ-08 全部通过"},
35 35 {"id": "U-019", "phase": "PH2", "sub_phase": "PH2-S3", "name": "budget_guardrail_config + c_budget_check", "classification": "Risk Gate", "depends_on": ["U-012"], "coverage": {"model": ["budget_guardrail_config"], "view": [], "controller": ["c_budget_check"], "process": ["p_budget_guardrail"], "requirement": ["REQ-10"]}, "done": "REQ-10 happy/boundary 通过"},
36 36 {"id": "U-020", "phase": "PH2", "sub_phase": "PH2-S3", "name": "p_usage_limit_wait 完整验收", "classification": "Risk Gate", "depends_on": ["U-012"], "coverage": {"model": [], "view": [], "controller": [], "process": ["p_usage_limit_wait"], "requirement": ["REQ-14"]}, "done": "REQ-14 全部通过（含超时兜底）"},
... ...
src/org_level_loop.py 0 → 100644
  View file @ba481dd
  1 +"""U-017: Org-level Loop closed loop (architecture doc §11 MVP acceptance
  2 +criterion 4) -- ties U-015 (root-cause judgment) + U-016 (routing) + real
  3 +Coder/Validator calls into one function: given a task sitting at
  4 +'validating' with a known bug, autonomously locate root cause, route to
  5 +the right Org Agent, drive the fix, and re-validate back to 'done'.
  6 +"""
  7 +
  8 +from src import bug_record, org_loop_route, org_node_trigger, root_cause_trace
  9 +
  10 +_ROUTE_FOR_CATEGORY = {"coder": "fast_lane", "planning": "full_replan"}
  11 +
  12 +
  13 +def run_fix_loop(conn, task_id, target_folder_path, bug_description, context, fix_prompt, verify_prompt, timeout=300):
  14 + """Returns {"root_cause_category", "route", "verified", "final_status",
  15 + "verify_response"}. Raises root_cause_trace.InconclusiveEscalateError if
  16 + the 3 perspectives don't reach 2/3 agreement (same as U-015 -- this
  17 + function doesn't add its own fallback for that, matching 04-P's own
  18 + "升级人工" answer to disagreement)."""
  19 + perspective_fns = [
  20 + root_cause_trace.real_validator_perspective(
  21 + name, bug_description, context, target_folder_path,
  22 + trigger_fn=org_node_trigger.trigger_org_node, timeout=timeout,
  23 + )
  24 + for name in ("correctness", "requirement_completeness", "skill_defect")
  25 + ]
  26 + judgment = root_cause_trace.judge_root_cause(perspective_fns)
  27 + category = judgment["root_cause_category"]
  28 + route = _ROUTE_FOR_CATEGORY.get(category, "full_replan")
  29 +
  30 + bug_record.create_bug_record(
  31 + conn, task_id=task_id, description=bug_description,
  32 + root_cause_category=category, root_cause_detail=judgment["root_cause_detail"],
  33 + suggested_route=route,
  34 + )
  35 + org_loop_route.route_org_loop(conn, task_id, route)
  36 +
  37 + fixer_node = "coder" if route == "fast_lane" else "planner"
  38 + org_node_trigger.trigger_org_node(fixer_node, fix_prompt, target_folder_path, timeout=timeout)
  39 +
  40 + verify_response = org_node_trigger.trigger_org_node("validator", verify_prompt, target_folder_path, timeout=timeout)
  41 + verified = "PASS" in verify_response.upper()
  42 +
  43 + if verified:
  44 + # A real Work Graph would move coding/planning_brd_prd_sdd back to
  45 + # 'validating' once the fixer's own Work Graph completes (U-011/012/013,
  46 + # same mechanism U-014 uses) -- no separate controller exists for that
  47 + # hop, so this mirrors it directly rather than inventing one just for
  48 + # this Unit.
  49 + conn.execute("UPDATE task SET status = 'validating' WHERE task_id = ?", (task_id,))
  50 + conn.commit()
  51 + org_loop_route.close_task_done(conn, task_id)
  52 + final_status = "done"
  53 + else:
  54 + final_status = conn.execute("SELECT status FROM task WHERE task_id = ?", (task_id,)).fetchone()[0]
  55 +
  56 + return {
  57 + "root_cause_category": category,
  58 + "route": route,
  59 + "verified": verified,
  60 + "final_status": final_status,
  61 + "verify_response": verify_response,
  62 + }
... ...
src/test_org_level_loop.py 0 → 100644
  View file @ba481dd
  1 +"""U-017 tests: the full Org-level Loop wiring (judge -> record -> route ->
  2 +fix -> re-verify -> done), mocked at the subprocess boundary (CLAUDE.md
  3 +Testing constraint). Real coverage is in src/test_org_level_loop_e2e.py
  4 +(RUN_REAL_AGENT_E2E=1).
  5 +
  6 +Run: python3 -m unittest src.test_org_level_loop -v
  7 +"""
  8 +
  9 +import unittest
  10 +from unittest.mock import patch
  11 +
  12 +from src import org_level_loop, schema
  13 +
  14 +
  15 +def _make_trigger(perspective_category, fix_ok=True):
  16 + """Fakes org_node_trigger.trigger_org_node for the whole loop: 3
  17 + perspective calls (all agreeing on perspective_category), one fixer
  18 + call, one verify call."""
  19 +
  20 + def fake(org_node, prompt, target_folder_path, timeout=300):
  21 + if "CATEGORY:" in prompt:
  22 + return "CATEGORY: {}\nDETAIL: because reasons".format(perspective_category)
  23 + if org_node in ("coder", "planner"):
  24 + return "Fixed it."
  25 + if org_node == "validator":
  26 + return "VERDICT: PASS\nlooks fixed" if fix_ok else "VERDICT: FAIL\nstill broken"
  27 + raise AssertionError("unexpected call: " + org_node)
  28 +
  29 + return fake
  30 +
  31 +
  32 +class RunFixLoopTestBase(unittest.TestCase):
  33 + def setUp(self):
  34 + self.conn = schema.connect(":memory:")
  35 + schema.create_schema(self.conn)
  36 + self.conn.execute(
  37 + "INSERT INTO project (project_id, name, target_folder_path, has_git_repo, status) "
  38 + "VALUES ('proj-1', 'P', '/tmp/proj-1', 1, 'active')"
  39 + )
  40 + self.conn.execute(
  41 + "INSERT INTO task (task_id, project_id, goal_text, task_type, status, loop_round, assigned_org_node) "
  42 + "VALUES ('task-1', 'proj-1', 'goal', 'goal', 'validating', 0, 'validator')"
  43 + )
  44 + self.conn.commit()
  45 +
  46 + def tearDown(self):
  47 + self.conn.close()
  48 +
  49 +
  50 +class CoderCategoryFastLaneTest(RunFixLoopTestBase):
  51 + def test_coder_category_routes_fast_lane_and_reaches_done(self):
  52 + with patch("src.org_node_trigger.trigger_org_node", side_effect=_make_trigger("coder", fix_ok=True)):
  53 + result = org_level_loop.run_fix_loop(
  54 + self.conn, "task-1", "/tmp/proj-1", "bug desc", "context", "fix it", "verify it"
  55 + )
  56 + self.assertEqual(result["root_cause_category"], "coder")
  57 + self.assertEqual(result["route"], "fast_lane")
  58 + self.assertTrue(result["verified"])
  59 + self.assertEqual(result["final_status"], "done")
  60 +
  61 + row = self.conn.execute("SELECT status, loop_round FROM task WHERE task_id = 'task-1'").fetchone()
  62 + self.assertEqual(row, ("done", 1))
  63 +
  64 + bug_row = self.conn.execute(
  65 + "SELECT root_cause_category, suggested_route FROM bug_record WHERE task_id = 'task-1'"
  66 + ).fetchone()
  67 + self.assertEqual(bug_row, ("coder", "fast_lane"))
  68 +
  69 +
  70 +class PlanningCategoryFullReplanTest(RunFixLoopTestBase):
  71 + def test_planning_category_routes_full_replan(self):
  72 + with patch("src.org_node_trigger.trigger_org_node", side_effect=_make_trigger("planning", fix_ok=True)):
  73 + result = org_level_loop.run_fix_loop(
  74 + self.conn, "task-1", "/tmp/proj-1", "bug desc", "context", "fix it", "verify it"
  75 + )
  76 + self.assertEqual(result["route"], "full_replan")
  77 + self.assertEqual(result["final_status"], "done")
  78 + assigned = self.conn.execute("SELECT assigned_org_node FROM task WHERE task_id = 'task-1'").fetchone()[0]
  79 + # after close_task_done the task is 'done'; assigned_org_node was set
  80 + # to 'planner' by route_org_loop and close_task_done doesn't touch it
  81 + self.assertEqual(assigned, "planner")
  82 +
  83 +
  84 +class VerificationFailsTest(RunFixLoopTestBase):
  85 + def test_fix_not_verified_leaves_task_out_of_done(self):
  86 + with patch("src.org_node_trigger.trigger_org_node", side_effect=_make_trigger("coder", fix_ok=False)):
  87 + result = org_level_loop.run_fix_loop(
  88 + self.conn, "task-1", "/tmp/proj-1", "bug desc", "context", "fix it", "verify it"
  89 + )
  90 + self.assertFalse(result["verified"])
  91 + self.assertNotEqual(result["final_status"], "done")
  92 + status = self.conn.execute("SELECT status FROM task WHERE task_id = 'task-1'").fetchone()[0]
  93 + self.assertNotEqual(status, "done")
  94 +
  95 +
  96 +if __name__ == "__main__":
  97 + unittest.main()
... ...
src/test_org_level_loop_e2e.py 0 → 100644
  View file @ba481dd
  1 +"""U-017's real-call coverage: architecture doc §11 MVP acceptance criterion
  2 +4, for real -- a deliberately injected off-by-one wall-collision bug (same
  3 +scenario shape as U-015's real test, but this time actually driving a real
  4 +fix instead of just judging), routed, fixed by a real Coder call editing the
  5 +actual file in the repo, then confirmed by a real Validator call that reads
  6 +the fixed file. Gated behind RUN_REAL_AGENT_E2E=1 (CLAUDE.md Testing
  7 +constraint).
  8 +
  9 +Run: RUN_REAL_AGENT_E2E=1 python3 -m unittest src.test_org_level_loop_e2e -v
  10 +"""
  11 +
  12 +import subprocess
  13 +import tempfile
  14 +import unittest
  15 +
  16 +from src import agent_profile, org_level_loop, schema
  17 +from src.test_support import skip_unless_env
  18 +
  19 +_SDD_SNIPPET = (
  20 + "## Wall collision (SDD §3.2)\n"
  21 + "The board is a grid with valid coordinates 0..N-1 in each axis. A collision (game "
  22 + "over) MUST be triggered the instant the head's x or y coordinate is < 0 or >= N."
  23 +)
  24 +
  25 +_BUG_DESCRIPTION = (
  26 + "When the snake's head reaches the rightmost or bottommost column, the game does not "
  27 + "end immediately -- the head is visually one cell outside the board before 'wall "
  28 + "collision' triggers. See collision.py in this repo."
  29 +)
  30 +
  31 +_BUGGY_CODE = (
  32 + "def check_wall_collision(head_x, head_y, board_size):\n"
  33 + " # BUG: uses > / < instead of >= / <=, so the head is allowed to sit one cell\n"
  34 + " # past the actual boundary before a collision is registered.\n"
  35 + " return head_x < 0 or head_y < 0 or head_x > board_size or head_y > board_size\n"
  36 +)
  37 +
  38 +
  39 +@skip_unless_env("RUN_REAL_AGENT_E2E")
  40 +class RealOrgLevelLoopTest(unittest.TestCase):
  41 + def test_real_fix_loop_reaches_done(self):
  42 + with tempfile.TemporaryDirectory() as tmp:
  43 + subprocess.run(["git", "init"], cwd=tmp, check=True, capture_output=True)
  44 + agent_profile.deploy_agent_definitions(tmp)
  45 + with open(tmp + "/collision.py", "w") as f:
  46 + f.write(_BUGGY_CODE)
  47 + subprocess.run(["git", "add", "collision.py"], cwd=tmp, check=True, capture_output=True)
  48 + subprocess.run(
  49 + ["git", "-c", "user.name=t", "-c", "user.email=t@t.com", "commit", "-m", "seed buggy collision.py"],
  50 + cwd=tmp, check=True, capture_output=True,
  51 + )
  52 +
  53 + conn = schema.connect(":memory:")
  54 + schema.create_schema(conn)
  55 + conn.execute(
  56 + "INSERT INTO project (project_id, name, target_folder_path, has_git_repo, status) "
  57 + "VALUES ('proj-1', 'P', ?, 1, 'active')",
  58 + (tmp,),
  59 + )
  60 + conn.execute(
  61 + "INSERT INTO task (task_id, project_id, goal_text, task_type, status, loop_round, assigned_org_node) "
  62 + "VALUES ('task-1', 'proj-1', '实现贪吃蛇游戏', 'goal', 'validating', 0, 'validator')"
  63 + )
  64 + conn.commit()
  65 +
  66 + result = org_level_loop.run_fix_loop(
  67 + conn, "task-1", tmp,
  68 + bug_description=_BUG_DESCRIPTION,
  69 + context="SDD:\n{}\n\nImplementation:\n{}".format(_SDD_SNIPPET, _BUGGY_CODE),
  70 + fix_prompt=(
  71 + "Fix the off-by-one wall-collision bug in collision.py per this SDD "
  72 + "section, then commit the fix:\n\n" + _SDD_SNIPPET
  73 + ),
  74 + verify_prompt=(
  75 + "Read collision.py in this repo and check whether check_wall_collision now "
  76 + "correctly triggers at the boundary (>= N / < 0, not > N / < 0), per this SDD "
  77 + "section:\n\n" + _SDD_SNIPPET + "\n\nReply with 'VERDICT: PASS' or 'VERDICT: FAIL' "
  78 + "and your reasoning."
  79 + ),
  80 + timeout=300,
  81 + )
  82 +
  83 + self.assertEqual(result["root_cause_category"], "coder")
  84 + self.assertEqual(result["route"], "fast_lane")
  85 + self.assertTrue(result["verified"], "validator did not confirm the fix: " + result["verify_response"])
  86 + self.assertEqual(result["final_status"], "done")
  87 +
  88 +
  89 +if __name__ == "__main__":
  90 + unittest.main()
... ...