-
Recovers tasks abandoned mid-flight (status in queued/planning_brd_prd_sdd/ coding/validating with no recent execution_event) by re-driving them through task_execution.run_or_resume_task_lifecycle on a background poll, guarded by an in-process in-flight lock to avoid double dispatch. Responds to a real 2026-08-06 incident where a task sat stuck in 'validating' for 15+ hours with nothing polling it. Scoped to FR-2 only from docs/todo/scheduler-production-polling-loop- requirements.md; FR-1 (new-goal auto-dispatch) stays out per that doc's own non-goals -- it's a separate, larger autonomy-boundary decision. Design recorded in docs/todo/scheduler-production-polling-loop-design.md. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
-
Per CLAUDE.md's docs/ role split (todo = pending, done = shipped into the SDD/code): all seven Graph Engineering series items with a real implementation (B, D, E, F, G, H -- A had no separate doc, resolved directly in graphlemon_architecture.md; C is explicitly won't-do) now have their requirements+design docs moved from docs/todo/ to docs/done/, git mv'd to preserve history. Every cross-reference to the 12 moved files -- across the other moved docs themselves, graph-engineering-open-items.md, 2 Skill files, and 9 src/*.py docstrings/comments -- updated from docs/todo/<file> to docs/done/<file>. Verified no stale docs/todo/ references to any of the 12 filenames remain anywhere in the repo. graph-engineering-open-items.md itself stays in docs/todo/ (not moved) -- it's a living index that keeps gaining new entries from later comparison-note/gap-analysis passes (H itself was added this way, and a new item already appeared while this cleanup was in progress), not a single finished conclusion. Its own header now reflects that all of A/B/D/E/F/G/H are done and explains why it wasn't moved. Also fixed two pre-existing stray path references (missing todo/done/ prefix entirely, predating this cleanup) in its G section. Full src suite (625 tests) passes -- this is a pure documentation/comment change, no behavior touched. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
-
Implements docs/todo/role-package-abstraction-design.md. Requirements review (docs/todo/graph-engineering-open-items.md's H section) had already corrected the requirements doc's own dependency analysis before this; scope confirmed with user 2026-08-06: relax all 4 org_node CHECK constraints in one pass, not staged. Format is JSON, not YAML: 05 §5.7's pure-stdlib constraint rules out PyYAML, and unlike Skill files (whose YAML frontmatter only Claude Code itself ever parses, never this codebase's own Python), role.json genuinely needs to be read by agent_profile.py -- json.load is the zero-risk choice over a hand-rolled parser. role.json is a REFERENCE manifest, not a content copy: identity.soul_md_path/ agents_md_path still point at the untouched agents/<role>/SOUL.md/AGENTS.md files. This is what makes "deployed output must stay identical" close to automatic -- render_agent_definition()'s body-assembly logic doesn't change at all, only the frontmatter's data source does (hardcoded FRONTMATTER dict -> parsed role.json). Verified byte-for-byte: diffed the old (git HEAD) render_agent_definition() output against the new one for all three roles, zero differences. agent_profile.py: ORG_NODES is now discovered by scanning roles/ for role.json files (_discover_role_ids()) instead of a hardcoded 3-tuple; FRONTMATTER/_DESCRIPTIONS/_MIGRATED_FROM_PROFILE dicts are gone. Every public function takes an optional roles_dir override specifically so a test can prove "a new role needs zero code changes" against a throwaway fixture directory (NewRoleWithoutCodeChangeTest) rather than asserting it by inspection. schema.py: skill.owner_org_node/task.assigned_org_node/work_graph_run.org_node/ agent_profile.org_node relaxed from CHECK(x IN ('planner','coder','validator')) to CHECK(x != ''). Real blast radius found during design (bigger than the requirements doc's own risk note): task/work_graph_run/skill are referenced by several other tables' foreign keys, but since only the parent tables' own CHECK text changes (child FK definitions are untouched), only these 4 parent tables need rebuilding -- followed SQLite's documented procedure for schema changes under active foreign keys (foreign_keys=OFF for the duration, rebuild, foreign_key_check to confirm nothing went dangling, foreign_keys=ON). migrate_relax_org_node_constraints() depends on D's migrate_add_work_graph_fork_columns() having run (called unconditionally, idempotent) so work_graph_run's rebuild has the right column set either way. ux_work_graph_run_active and task's 3 indexes verified to survive the rebuild. work_graph_run_query.py's own separately-hardcoded _VALID_ORG_NODES (a real dependency the requirements doc's risk section missed) now reuses agent_profile.ORG_NODES instead of maintaining a second copy. Explicitly NOT done (scoped out during requirements review, not an oversight): frontend/app.js's 4 hardcoded 3-role spots (route regex, sort order, dropdown, lookup object) -- a new role still won't be recognized by the PH3 console. A `role` table with real FK referential integrity was also not built; this uses the simpler "just relax the enum" approach. Full src suite (623 tests) + supervisor suite (125 tests) pass with zero regressions. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> -
Implements docs/todo/work-graph-time-travel-debugging-design.md, backend and console together (user confirmed 2026-08-06). Key finding that shrank the v1 scope: ux_work_graph_run_active is a PARTIAL unique index (only covers non-terminal statuses), not a blanket one -- forking from a parent run that's already reached a terminal status never collides with it at all. v1 restricts forking to exactly that case, so no index/constraint change is needed, only two new nullable columns (parent_run_id, fork_of_node_exec_id). Resolved the requirements doc's FR-1 ambiguity ("override input_payload, re-trigger execution") as OUTPUT SUBSTITUTION instead: the fork point's own output_payload is replaced with the human-supplied content directly, it is never re-executed via a new claude -p call -- only the strict downstream subgraph actually re-runs. This sidesteps reconciling the different build_prompt/input_payload conventions each caller uses (Planner's artifact_type lookup, E's generic task_prompt field, F's jury) -- work_graph_fork.py only needs the persisted graph_manifest and each node's output_payload. Implementation needs ZERO changes to the execution engine: pre-seeding every non-downstream node as a 'done' work_node_execution row before calling the unmodified run_graph() on the unmodified graph_manifest is enough -- run_work_node()'s existing REQ-06 breakpoint-recovery short-circuit (already-'done' nodes return their persisted output without calling run_fn) does all the "reuse without re-executing" work for free. _reconstruct_build_prompt supports two graph shapes generically from persisted state alone (Planner's canonical doc chain via e2e_pipeline._planner_prompt_builder, and E's generic task_prompt field); anything else raises ForkGraphShapeUnsupportedError rather than producing broken prompts. New: src/work_graph_fork.py (create_fork/execute_fork/fork_work_graph_run/ start_fork_in_background, the last splitting fast DB-only setup from slow real execution so an HTTP caller gets run_id back immediately, same pattern task_execution.start_task_execution_in_background already uses). work_graph_run gets parent_run_id/fork_of_node_exec_id (SCHEMA_SQL + migrate_add_work_graph_fork_columns for existing databases). work_graph_run_query.py/task_query.py surface both fields. New endpoint POST /api/work-nodes/:node_exec_id/fork. Console: a "从这里分叉" form on each 'done' node in v_run_detail, a fork badge linking back to the parent run, and a 🔀 marker on v_graph_console's run list -- no diff/merge UI (non-goal #3), read-only branch visibility only. Verified end-to-end for real: started the actual backend, sent a real HTTP fork request, confirmed the new run persisted with correct parent_run_id/fork_of_node_exec_id lineage. Tests cover requirements doc acceptance standards 1-3 (PRD swapped -> SDD/Benchmark genuinely re-execute, BRD untouched; original run completely unaffected; the same node forked twice produces independent branches) plus schema migration idempotency, query-layer field surfacing, the HTTP route, and 3 frontend smoke-test assertions. Full src suite (608 tests) + all 3 frontend smoke tests pass. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> -
Implements docs/todo/validator-loop-until-dry-skill-design.md. Pure Skill + Agent-definition wiring, no execution-engine changes -- the convergence loop runs inside a single Work Node's one claude -p session (multi-turn tool use), not a graph_manifest-level cycle, so topological_layers()/ run_graph() are untouched. New Skill skills/unknown-scope-defect-sweep/SKILL.md: K=2 consecutive no-new-finding rounds to stop, 20-round hard cap, dedup key must be a structural identifier (file path + line, route path, package name) not a natural-language description (the same finding gets reworded call to call, so text-keyed dedup never converges), dedup set is "everything ever seen" not "everything confirmed" (the common bug the 14-step reference names: a rejected candidate reappearing as a false "new finding" every round). Each candidate is verified via qa-task-validation's existing PASS/FAIL/INCONCLUSIVE framework -- deliberately NOT root_cause_trace's three-perspective jury (F), since that answers a different question (root cause AFTER a failure) from this one (is this candidate real at all) -- documented in the design doc's own tradeoff note. Wired into agent_profile.py's FRONTMATTER["validator"]["skills"] and agents/validator/AGENTS.md's "必须加载" section with a concrete trigger signal (task description can't state an exact expected count/boundary up front). Only FR-4 (Skill wired into the rendered .claude/agents/validator.md) has deterministic, mockable behavior -- new test test_validator_references_unknown_scope_defect_sweep_skill. Acceptance criteria 1-3 (actually finds everything, actually dedups correctly, actually stops at the hard cap) need real multi-turn claude -p behavior, out of scope for mocked tests (RUN_REAL_AGENT_E2E=1 territory), same category of limitation as F's own real-concurrency-timing note. Full src suite (587 tests) passes with zero regressions. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
-
Reviewed docs/todo/role-package-abstraction-requirements.md against the actual codebase before confirming direction. Two corrections to the document's own risk analysis: 1. Its dependency list for agent_profile.ORG_NODES/FRONTMATTER was wrong in one place and missing the largest one: work_graph_orchestrate.py doesn't import agent_profile at all (only org_node_trigger.py and project_scaffold.py::deploy_agent_definitions() do -- smaller real footprint than claimed). But the document missed two real hardcoded 3-role dependencies entirely: src/work_graph_run_query.py's _VALID_ORG_NODES, and -- the actually large one -- four separate hardcoded spots in frontend/app.js (a route regex, a sort-order array, a task-creation <option> dropdown, and a lookup object literal). 2. Acceptance criterion 1 ("a new role needs no code/schema changes") only covers schema.py + agent_profile.py per its own non-goal #1 (a new role doesn't need real Org Graph wiring) -- the console genuinely won't recognize it either way. Flagged so the design doc states this explicitly instead of letting "no code changes" read as end-to-end. Confirmed for design phase with these two additions folded in; the document's own open questions (§8) are left for the design pass, not pre-decided here. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> -
Implements docs/todo/verifier-fan-out-jury-design.md. root_cause_trace.py: _tally_perspectives() extracts the existing >=2/3 rule out of judge_root_cause so the closure-based version and the new judge_root_cause_via_work_graph() share one implementation, not two that could drift apart. The fan-in vote stays plain Python (zero_token aggregation per the design-dynamic-work-graph Skill's own edge_type distinction) rather than becoming a fourth Work Node -- no reason to pay for an Agent call just to count votes. judge_root_cause_via_work_graph(): each perspective becomes an independent Work Node with real schema-constrained structured output ({"category", "detail"} directly), not real_validator_perspective's manual "CATEGORY:"/"DETAIL:" text-line parsing. judge_root_cause / real_validator_perspective are kept, not removed -- documented as the lightweight fallback for scenarios that don't need worktree/Work Graph overhead. org_level_loop.run_fix_loop (the one real production caller) now calls the Work Graph version; the fixer/verify calls are unchanged flat trigger_org_node calls. Real bug found and fixed along the way: this is the first production code path that actually triggers a run_graph() multi-node CONCURRENT layer (every graph_manifest before F was single-node or a strict serial chain, so G's ThreadPoolExecutor path had never really run in production). scheduler.py::run_goal_to_completion and task_execution.py::start_task_execution_in_background both created their conn without check_same_thread=False -- fixed both, since their conn can now reach run_graph()'s worker threads via run_fix_loop. Test boundary migrated in test_org_level_loop.py/test_scheduler.py: the 3 perspective calls are now mocked at work_graph_execute.write_workflow_script/ execute_work_graph (matching test_work_graph_orchestrate.py's pattern) instead of org_node_trigger.trigger_org_node. New tests cover: three real independent Work Node executions, judgment-rule equivalence with the closure version, clean per-node input_payload, and concurrent wall-clock overlap across the three perspectives. Full src suite (586 tests) + supervisor suite (125 tests) pass with zero regressions. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> -
Implements docs/todo/dynamic-work-graph-design-skill-design.md. G already answered FR-2 (how to mark a parallel-eligible node group): no new field needed -- topological_layers() infers independence purely from depends_on/edges, so the new Skill's job is just "express real dependencies honestly, don't add fake ones." New Skill skills/design-dynamic-work-graph/SKILL.md: the node-boundary / zero_token-vs-needs_judgment / per-node self-contained task_prompt methodology, wired into agent_profile.py's FRONTMATTER for coder and validator (Planner's own doc-chain path is untouched -- FR-4 only names Coder/Validator). work_graph_plan.py: new _GENERIC_PROMPT_TEMPLATE + generic_node_prompt_builder(), used whenever org_node != 'planner'. Planner keeps its existing fixed brd/prd/sdd/benchmark/... chain (_PROMPT_TEMPLATE) completely unchanged -- zero regression risk to the one real production planning path. Scoped per the requirements doc's non-goal #1 ("prove the mechanism works, don't cover every real scenario yet"): does NOT wire into e2e_pipeline.py::implement_code()'s existing single-node production path. New tests prove the mechanism itself: a Coder scenario that naturally splits into 3 independent modules produces a correctly-shaped graph_manifest, topological_layers() puts all three in one concurrent layer (ties E directly to G), and generic_node_prompt_builder drives it through run_graph() to completion. Full src suite (580 tests) passes with zero regressions. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> -
Implements docs/todo/work-graph-parallel-execution-design.md: run_graph() now schedules graph_manifest nodes layer-by-layer (topological_layers(), Kahn's algorithm grouped by round instead of flattened), running a multi-node layer's nodes concurrently via ThreadPoolExecutor (default 4 workers) with a barrier between layers; a single-node layer stays synchronous, matching every real graph_manifest observed so far. topological_order() is now a thin flattening wrapper, unchanged behavior (regression-tested). _LockedConn/_FetchedRows makes the existing scattered conn.execute() call sites in work_node_run.py/work_graph_orchestrate.py/execution_log.py safe to call from multiple threads without touching those call sites -- note the lock must cover the full execute()-through-fetchone()/fetchall() cycle, not just execute() itself, or two threads can still collide on the shared sqlite3 connection handle. That gap was caught for real (not theoretically) by RunGraphRealSqliteConcurrencyTest during development -- committing the fixed version, not the first draft. create_worktree() (work_node_run.py) gets a module-level lock around the `git worktree add` call for concurrent layers touching the same repo. New tests: layer grouping, same-layer concurrency + inter-layer barrier, one node's failure not cancelling a concurrent sibling (WorkGraphLayerError aggregates), breakpoint-recovery resume of a concurrent layer, a quota-wait timeout coexisting with a sibling's real persisted output, real (non-mocked) sqlite under concurrent writes across several rounds, and real concurrent `git worktree add` calls. Full src (575 tests) + supervisor (125 tests) suites pass with zero regressions. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
-
…de13-main-checkout-drift
-
The literal token value was committed in plaintext across 4 files (agent-memory notes, a stored run artifact, and a test assertion) while documenting that it had been redacted from migrated skill files -- the documentation itself failed to redact it. Replacing with placeholders/omission ahead of pushing this repo to a remote for the first time; the pre-existing commits (2a71bd5, 0cbf4b1, 14dee05) still contain the plaintext value in history. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
-
…85fa-main-checkout-drift
-
Root cause traced (docs/implementation-gap-analysis.md): the Benchmark document Validator relies on for its verdict was frequently a narrative about the work session ("The planner workflow completed and produced a Benchmark defining 18 scenarios...") instead of the actual scenarios -- found via a real skill-detail-page task's benchmark.md, then confirmed present since the earliest real self-hosting runs. Validator's own judgment was, in practice, resting on thin, sometimes-vacuous prose. Planner's work graph (brd->prd->sdd->benchmark) extends to benchmark->test_dataset->test_script: test_dataset/test_script are always present as nodes, but the generating subagent judges per-goal whether a real dataset/script applies (many goals, e.g. UI wiring, legitimately don't have one -- replying 'N/A: <reason>' is a real, expected answer, not a failure). test_script gets a genuinely mechanical verify_fn (real py_compile/node --check, not LLM judgment) -- the one node type where an objective check is actually possible. All four document prompts also gained an explicit anti-narration instruction (only 'brd' had a weaker version of this before). Coder's implementation prompt now receives the real benchmark/dataset/ script and must make the Planner's real script pass for real, not just whatever tests it invents on its own. Validator gets its own 3-node Work Graph (validator_benchmark -> validator_test_dataset -> validator_test_script), generated independently from SDD/PRD -- deliberately NOT reusing Coder's or Planner's own copies, same reasoning as a held-out eval set vs. the dev set a model was tuned against. The resulting script, if real, is actually executed against the current repository state; a non-zero real exit code force-overrides the verdict to FAIL in code (user-confirmed hard gate: the script is not up for LLM debate on a failure, only on a pass can subjective judgment add INCONCLUSIVE/risk notes). schema.py: artifact.artifact_type's CHECK constraint gains the 5 new types, plus migrate_expand_artifact_types() -- SQLite has no ALTER-based constraint migration, so this is the standard rebuild-copy-swap dance, applied for real to the live ~/.graphlemon/graphlemon.db (42 rows, verified byte-identical count before/after, integrity_check clean). skills/qa-task-validation.md and skills/project-sdd-planning.md already gained a standing principle from this same incident: verify cross-task data dependencies against real system state, not seeded fixtures alone (see prior commit d7035fc) -- this commit is the mechanism-level half of the same fix, not just the prose-guidance half. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> -
Real incident (2026-08-06): the skill-content migration task and the skill detail page bug fix were two separate tasks. Neither was asked to verify that migrated content is actually reachable through the DB-backed lookup the detail page uses. Both passed their own tests cleanly -- because both only ever exercised the mechanism against seeded/mocked fixture data, never checked whether the real producer's real output actually exists in the real system. The result: the skill detail page returned "not found" for all 12 real migrated skills, and nobody caught it until a human clicked through the real console. qa-task-validation (§8): passing unit tests against seeded fixtures proves the mechanism is correct given valid input -- it does not prove that input exists for real. When a feature reads data another task/process was supposed to have already written, Validator must independently query the real running system against a real (not newly-created) object. project-sdd-planning (§10): when a task's acceptance depends on another task's real output, the Benchmark must name a real, specific object as the acceptance anchor, not just describe the mechanism. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
-
…11fd-main-checkout-drift
-
…c231-main-checkout-drift
-
_launch_candidate() spawns a real `python -m src.backend_server` process (`revived`) to re-verify the previous version during a post-cutover rollback. On the success path it's correctly left running -- it IS the new Live. But on every failure path (never bound live_port, failed re-verification, git revert conflict) nothing ever terminated it -- found via 31 accumulated orphaned processes on this machine, one per failed rollback_path_b call across many real test runs this session. This is also the likely cause of RollbackPathBFailureTest's own reported non-determinism (three independent real Validator sessions saw it fail, never reproduced manually): a leaked process from an earlier failed run can still be squatting on the port a later run's _find_free_port() picks, producing a false "alive" health-check reading. Fix: call deploy_cutover._stop_process(revived.pid) in both failure branches before raising, mirroring the pattern cutover() already uses for its own stop_timeout-bounded process teardown. Also fixes RollbackPathBHappyPathTest's own matching leak (it re-appended bad_live.pid a second time instead of tracking the actual revived process, a copy-paste bug, not real cleanup) by capturing the real pid via a thin _launch_candidate wrapper. 31 pre-existing orphaned processes killed by hand; verified 5 repeated test runs after the fix leak zero new ones. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
-
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
-
Three real self-hosting fixes (log content never written, skill detail page missing real content, artifact list not clickable), two design questions set aside per user's request (unified Planner-driven work graph across all org agents, loop visualization on the graph), three bugs left for a future batch, and two infrastructure bugs found and fixed along the way (manual_resume not actually continuing execution for most block reasons, run_goal_to_completion crashing instead of blocking on unrouted exceptions). Also upgrades the RollbackPathBFailureTest flakiness note -- three independent real sessions have now hit it. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
-
…1a79-main-checkout-drift