宜博 / lemongraph

Implements org-graph-unification-phase2-universal-self-planning-design.md
§2.3, §3, §6 -- Coder's implement_code() no longer runs one hardcoded
synthetic "implement" node; it self-plans via plan_work_graph(conn,
"coder", ..., append_to_run_id=run_id), appending its own nodes onto
Planner's existing work_graph_run instead of creating a separate one.

- work_graph_plan.py: per-org_node suggested node-type catalogs
  (_CODER_NODE_CATALOG/_VALIDATOR_NODE_CATALOG, "starting points, not
  requirements" same as _PLANNER_NODE_CATALOG), new judgment/on_fail/
  promotes_to_artifact fields in the generic planning prompt. New
  plan_work_graph(..., append_to_run_id=...) parameter: merges newly
  planned nodes into an existing run (node_id collision -> raise, never
  silent overwrite), returns new_node_ids so callers can avoid
  reprocessing nodes an earlier stage already promoted to artifacts. New
  shared dispatch_build_prompt() for graphs mixing Planner-shaped
  (artifact_type) and Coder-shaped (task_prompt) nodes side by side.

- e2e_pipeline.py: _write_completed_node_artifacts() now also promotes
  nodes declaring promotes_to_artifact (not just Planner's artifact_type
  nodes), with an only_node_ids scope so a merged-graph caller doesn't
  re-touch artifacts an earlier stage already wrote. implement_code()
  reworked around self-planning; code_diff stays Python-computed (a real
  git diff, not any node's own content -- no node can know the diff ahead
  of running it), test_report is promoted from whatever the self-planned
  graph declares, falling back to a synthesized concatenation if it
  declares none. needs_worktree=True now passed to run_graph() for Coder
  (found missing entirely during this rework -- harmless with the old
  single-node manifest, a real 03-C §3.6 gap once concurrent code-writing
  nodes are possible). Also fixed a real bug found while wiring this up:
  _review_planner_output_with_retries()'s "reuse existing artifacts"
  branch hardcoded run_id=None, even though every real artifact row
  already has a valid produced_by_run_id -- Coder had nothing to append
  into on that resume path.

- work_graph_fork.py: _reconstruct_build_prompt now dispatches per-node
  (via the new shared dispatch_build_prompt) instead of requiring the
  whole graph to be one uniform shape -- a real regression Phase 2b would
  otherwise have introduced silently, since a forked merged graph can now
  genuinely mix both node shapes.

Sequencing note (unchanged from Phase 2a's own finding): work_graph_run's
org_node column removal and the ux_work_graph_run_active index narrowing
(design doc §2.1/§2.2) stay deferred to Phase 2c -- Validator still
creates its own separate run until Validator also appends into the shared
one, so narrowing the index to task_id-only now would collide with it.

python3 -m unittest discover -s src (752 tests) + discover -s supervisor
-p "test_*.py" (126 tests) both green, zero regressions.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Inserts Planner -> planner_reviewer -> Coder -> Validator into the fixed Org
Graph flow (was Planner -> Coder -> Validator), per
docs/todo/org-graph-unification-phase1-review-gate-design.md. A new
planner_reviewer role independently judges whether Planner's BRD/PRD/SDD/
Benchmark output is real structured content (not narrative prose) before
Coder is allowed to start, with a 3-attempt retry loop and a narration
heuristic informed by the real narrative-output incident
(docs/todo/validator-test-script-narrative-output-issue.md).

Also fixes, in the same code path:
- The P0 bug in task-execution-resume-assumes-planner-artifacts-exist.md:
  resuming a task no longer assumes sdd/prd/benchmark artifacts exist --
  falls through to real replanning when they don't.
- A fresh-run KeyError (not resume-specific) when a self-planned graph omits
  the prd/benchmark node types.
- agent_profile.migrated_from_profile was NOT NULL, which broke as soon as a
  role with no legacy Hermes profile (planner_reviewer) was recorded.

org_node_lifecycle._REQUIRED_ARTIFACTS["coder"] is narrowed from
(brd,prd,sdd,benchmark) to (sdd,) -- the old 4-artifact gate never actually
blocked anything since artifact_write.py sets validation_status='passed'
unconditionally; planner_reviewer is the first real semantic gate.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

… work, compress bloated incident log

- Delete org-graph-as-work-graph-unified-loop-requirements.md: fully
  superseded by org-graph-unification-and-universal-self-planning-
  requirements.md; folded its one non-duplicated detail (the 2084590a
  benchmark-narration trigger incident) into that doc's background instead.
- Split scheduler-production-polling-loop-{requirements,design}.md by
  completion status: FR-2 (in-flight task reconciliation, shipped) moves to
  docs/done/scheduler-task-reconciliation-{requirements,design}.md; FR-1
  (new-goal auto-dispatch, still undecided) becomes a small standalone
  docs/todo/scheduler-new-goal-dispatch-requirements.md instead of staying
  bundled with a doc that's mostly done.
- Compress validator-test-script-narrative-output-issue.md's six
  ad-hoc "第N轮更新" paragraphs crammed into one status line into a scannable
  timeline table; moved the task 2084590a benchmark evidence into §1 where
  the other repro examples live instead of leaving it stranded in the header.
- implementation-gap-analysis.md: logged two new gaps found today (no
  pre-Coder review of Planner's output; node execution still depends on
  native Dynamic Workflow) plus the architecture.md §10.1 staleness itself,
  per that doc's own "edit in place" maintenance convention.
- Updated the doc-path references left in src/task_execution.py and
  src/task_reconciliation.py docstrings to point at the new locations.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Recovers tasks abandoned mid-flight (status in queued/planning_brd_prd_sdd/
coding/validating with no recent execution_event) by re-driving them through
task_execution.run_or_resume_task_lifecycle on a background poll, guarded by
an in-process in-flight lock to avoid double dispatch. Responds to a real
2026-08-06 incident where a task sat stuck in 'validating' for 15+ hours with
nothing polling it.

Scoped to FR-2 only from docs/todo/scheduler-production-polling-loop-
requirements.md; FR-1 (new-goal auto-dispatch) stays out per that doc's own
non-goals -- it's a separate, larger autonomy-boundary decision. Design
recorded in docs/todo/scheduler-production-polling-loop-design.md.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Implements docs/todo/verifier-fan-out-jury-design.md. root_cause_trace.py:
_tally_perspectives() extracts the existing >=2/3 rule out of
judge_root_cause so the closure-based version and the new
judge_root_cause_via_work_graph() share one implementation, not two that
could drift apart. The fan-in vote stays plain Python (zero_token
aggregation per the design-dynamic-work-graph Skill's own edge_type
distinction) rather than becoming a fourth Work Node -- no reason to pay
for an Agent call just to count votes.

judge_root_cause_via_work_graph(): each perspective becomes an
independent Work Node with real schema-constrained structured output
({"category", "detail"} directly), not real_validator_perspective's
manual "CATEGORY:"/"DETAIL:" text-line parsing. judge_root_cause /
real_validator_perspective are kept, not removed -- documented as the
lightweight fallback for scenarios that don't need worktree/Work Graph
overhead.

org_level_loop.run_fix_loop (the one real production caller) now calls
the Work Graph version; the fixer/verify calls are unchanged flat
trigger_org_node calls.

Real bug found and fixed along the way: this is the first production code
path that actually triggers a run_graph() multi-node CONCURRENT layer
(every graph_manifest before F was single-node or a strict serial chain,
so G's ThreadPoolExecutor path had never really run in production).
scheduler.py::run_goal_to_completion and
task_execution.py::start_task_execution_in_background both created their
conn without check_same_thread=False -- fixed both, since their conn can
now reach run_graph()'s worker threads via run_fix_loop.

Test boundary migrated in test_org_level_loop.py/test_scheduler.py: the
3 perspective calls are now mocked at work_graph_execute.write_workflow_script/
execute_work_graph (matching test_work_graph_orchestrate.py's pattern)
instead of org_node_trigger.trigger_org_node. New tests cover: three real
independent Work Node executions, judgment-rule equivalence with the
closure version, clean per-node input_payload, and concurrent wall-clock
overlap across the three perspectives.

Full src suite (586 tests) + supervisor suite (125 tests) pass with zero
regressions.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Root cause traced (docs/implementation-gap-analysis.md): the Benchmark
document Validator relies on for its verdict was frequently a narrative
about the work session ("The planner workflow completed and produced a
Benchmark defining 18 scenarios...") instead of the actual scenarios --
found via a real skill-detail-page task's benchmark.md, then confirmed
present since the earliest real self-hosting runs. Validator's own
judgment was, in practice, resting on thin, sometimes-vacuous prose.

Planner's work graph (brd->prd->sdd->benchmark) extends to
benchmark->test_dataset->test_script: test_dataset/test_script are always
present as nodes, but the generating subagent judges per-goal whether a
real dataset/script applies (many goals, e.g. UI wiring, legitimately
don't have one -- replying 'N/A: <reason>' is a real, expected answer, not
a failure). test_script gets a genuinely mechanical verify_fn (real
py_compile/node --check, not LLM judgment) -- the one node type where an
objective check is actually possible. All four document prompts also gained
an explicit anti-narration instruction (only 'brd' had a weaker version of
this before).

Coder's implementation prompt now receives the real benchmark/dataset/
script and must make the Planner's real script pass for real, not just
whatever tests it invents on its own.

Validator gets its own 3-node Work Graph (validator_benchmark ->
validator_test_dataset -> validator_test_script), generated independently
from SDD/PRD -- deliberately NOT reusing Coder's or Planner's own copies,
same reasoning as a held-out eval set vs. the dev set a model was tuned
against. The resulting script, if real, is actually executed against the
current repository state; a non-zero real exit code force-overrides the
verdict to FAIL in code (user-confirmed hard gate: the script is not up
for LLM debate on a failure, only on a pass can subjective judgment add
INCONCLUSIVE/risk notes).

schema.py: artifact.artifact_type's CHECK constraint gains the 5 new
types, plus migrate_expand_artifact_types() -- SQLite has no ALTER-based
constraint migration, so this is the standard rebuild-copy-swap dance,
applied for real to the live ~/.graphlemon/graphlemon.db (42 rows,
verified byte-identical count before/after, integrity_check clean).

skills/qa-task-validation.md and skills/project-sdd-planning.md already
gained a standing principle from this same incident: verify cross-task
data dependencies against real system state, not seeded fixtures alone
(see prior commit d7035fc) -- this commit is the mechanism-level half of
the same fix, not just the prose-guidance half.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

New target projects never got .claude/agents/*.md deployed, so their first
Planner call always failed immediately (claude --agent planner finds nothing
in cwd) -- self-target usage never noticed since those files are already
committed as part of this repo. create_project() now deploys and commits
them alongside the existing .graphlemon/ init commit.

Also: unroutable execution failures were recorded as a bare
blocked_reason='execution_error' with no way to see what actually broke.
scheduler._block_task()/task_execution._mark_blocked() now persist the real
exception message into execution_event.payload (blocked_reason itself is
unchanged -- budget_guardrail matches its exact value elsewhere).

Found via a real console Start-button failure on a freshly created project;
see docs/implementation-gap-analysis.md for the full writeup.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Users had no way to actually start a queued goal from the console --
task.status just sits at 'queued' forever. Traced the root cause through
03-C: c_task_create's own contract ("若 max_concurrent_goals 已达上限，
任务落 queued 但不触发调度") implies task creation auto-triggers
scheduling otherwise, and 03-C/04-P repeatedly assume an always-on Org
Graph scheduler that watches the queue -- src/scheduler.py builds the
mechanism but nothing ever spins it up as a running process, so no
"start" button was ever designed because none was supposed to be needed.

Design (confirmed with user, both recommended options chosen):
- Pause/Stop are a soft stop, not a real subprocess kill -- the current
  in-flight `claude` call finishes naturally, only the next phase is
  withheld. Hard-killing would need Popen + PID tracking, out of scope.
- No new task.status enum value -- reuses blocked_pending_human with
  blocked_reason='user_paused'/'user_stopped', riding the existing
  manual_resume mechanism instead of a schema change.

New src/task_execution.py: run_or_resume_task_lifecycle() resumes from
whichever phase actually hasn't run yet (queued/planning_brd_prd_sdd/
coding), start_task_execution_in_background() runs it to completion on
its own daemon thread + sqlite connection, request_pause() blocks an
in-flight task. Extended /route's manual_resume handling so resuming
from a user-initiated pause/stop actually re-continues execution in the
background, not just flips the status field back.

Real gap found while building this: backend_server.create_server() never
stored db_path on the server object -- needed for the new background
thread to open its own connection to the same db.

22 new tests (14 src/test_task_execution.py + 8 TaskExecutionRouteTest in
test_backend_server.py, HTTP wiring mocked around the already-unit-tested
background thread mechanics). Verified against a real running server via
curl: /execute 409s a non-queued task, /pause 409s a queued task and a
double-pause, succeeds on a coding task. src/ suite: 496 tests, all green.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>