宜博 / lemongraph

Implements docs/done/work-graph-node-execution-deblackbox-requirements.md
end to end. The three documented call sites (e2e_pipeline.py::
_check_prd_consistency, work_graph_orchestrate.py::_llm_judgment_verify_fn,
work_graph_orchestrate.py::_make_run_fn) all used to render a one-off
Dynamic Workflow .js script and run it via `claude -p 'Run /<name>...'`
purely to get "agent identity + schema-shaped output" -- a workaround for
--json-schema silently conflicting with --agent (05 §5.4.1.1). That
workaround was never actually necessary: work_graph_plan.py::
plan_work_graph() solved the identical problem from day one with a flat
`claude --agent ... -p ...` call + ```json-fence extraction from free text,
already validated in production.

New org_node_trigger.trigger_org_node_structured() unifies all three call
sites onto that same flat-call approach. extract_json_payload() generalizes
work_graph_plan.py's fence-extraction algorithm; _extract_graph_manifest is
now a thin wrapper over it. Two more call sites not in the original design
doc (written before they existed) were found and migrated too:
_run_reflection_perspective and _fix_skill_for_node (Phase 2a's
Reflection-review/Fix-Skill). work_graph_fork.py needed no changes -- it
only reaches _make_run_fn indirectly via run_graph(), so it benefits
automatically.

FR-4's real-agent verification (RUN_REAL_AGENT_E2E=1, mandatory, not
optional -- src/test_org_node_trigger_e2e.py) confirmed persona loading,
--model override, and JSON-fence extraction all work against a real model.

write_workflow_script/execute_work_graph/execute_work_graph_with_usage_
limit_retry are confirmed to have no remaining production callers;
docstrings mark them historical, the file is kept (not deleted).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Inserts Planner -> planner_reviewer -> Coder -> Validator into the fixed Org
Graph flow (was Planner -> Coder -> Validator), per
docs/todo/org-graph-unification-phase1-review-gate-design.md. A new
planner_reviewer role independently judges whether Planner's BRD/PRD/SDD/
Benchmark output is real structured content (not narrative prose) before
Coder is allowed to start, with a 3-attempt retry loop and a narration
heuristic informed by the real narrative-output incident
(docs/todo/validator-test-script-narrative-output-issue.md).

Also fixes, in the same code path:
- The P0 bug in task-execution-resume-assumes-planner-artifacts-exist.md:
  resuming a task no longer assumes sdd/prd/benchmark artifacts exist --
  falls through to real replanning when they don't.
- A fresh-run KeyError (not resume-specific) when a self-planned graph omits
  the prd/benchmark node types.
- agent_profile.migrated_from_profile was NOT NULL, which broke as soon as a
  role with no legacy Hermes profile (planner_reviewer) was recorded.

org_node_lifecycle._REQUIRED_ARTIFACTS["coder"] is narrowed from
(brd,prd,sdd,benchmark) to (sdd,) -- the old 4-artifact gate never actually
blocked anything since artifact_write.py sets validation_status='passed'
unconditionally; planner_reviewer is the first real semantic gate.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

ConsistencyCheckFailedError is now raised by both the PRD-consistency
check and the test_script mechanical compile-check (07a0a27), but
scheduler._blocked_reason_for still labeled every instance
'prd_consistency_check_failed', misleadingly blaming the PRD for
test_script node failures. Distinguishes by node_id substring match
(see docstring for the ponytail ceiling/upgrade path).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Implements docs/todo/verifier-fan-out-jury-design.md. root_cause_trace.py:
_tally_perspectives() extracts the existing >=2/3 rule out of
judge_root_cause so the closure-based version and the new
judge_root_cause_via_work_graph() share one implementation, not two that
could drift apart. The fan-in vote stays plain Python (zero_token
aggregation per the design-dynamic-work-graph Skill's own edge_type
distinction) rather than becoming a fourth Work Node -- no reason to pay
for an Agent call just to count votes.

judge_root_cause_via_work_graph(): each perspective becomes an
independent Work Node with real schema-constrained structured output
({"category", "detail"} directly), not real_validator_perspective's
manual "CATEGORY:"/"DETAIL:" text-line parsing. judge_root_cause /
real_validator_perspective are kept, not removed -- documented as the
lightweight fallback for scenarios that don't need worktree/Work Graph
overhead.

org_level_loop.run_fix_loop (the one real production caller) now calls
the Work Graph version; the fixer/verify calls are unchanged flat
trigger_org_node calls.

Real bug found and fixed along the way: this is the first production code
path that actually triggers a run_graph() multi-node CONCURRENT layer
(every graph_manifest before F was single-node or a strict serial chain,
so G's ThreadPoolExecutor path had never really run in production).
scheduler.py::run_goal_to_completion and
task_execution.py::start_task_execution_in_background both created their
conn without check_same_thread=False -- fixed both, since their conn can
now reach run_graph()'s worker threads via run_fix_loop.

Test boundary migrated in test_org_level_loop.py/test_scheduler.py: the
3 perspective calls are now mocked at work_graph_execute.write_workflow_script/
execute_work_graph (matching test_work_graph_orchestrate.py's pattern)
instead of org_node_trigger.trigger_org_node. New tests cover: three real
independent Work Node executions, judgment-rule equivalence with the
closure version, clean per-node input_payload, and concurrent wall-clock
overlap across the three perspectives.

Full src suite (586 tests) + supervisor suite (125 tests) pass with zero
regressions.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Found for real: a work_graph_execute.WorkGraphExecuteError ("API Error:
Response stalled mid-stream") wasn't in _ROUTABLE_EXCEPTIONS, so it
propagated uncaught and crashed the calling process entirely -- leaving the
task neither running nor blocked_pending_human, just stuck with
blocked_reason=NULL. run_scheduler_once's own try/except only protects
OTHER goals in the same batch from one goal's failure; it never applied the
task-level blocked_pending_human consequence either.

Add a catch-all mirroring task_execution._mark_blocked's existing fallback
for its own (Start-button) entry point, so both ways of running a task now
guarantee it always lands on a real terminal-ish status with the real error
message persisted, never a caller-crashes-and-the-task-is-just-stuck outcome
nothing can manual_resume from.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

New target projects never got .claude/agents/*.md deployed, so their first
Planner call always failed immediately (claude --agent planner finds nothing
in cwd) -- self-target usage never noticed since those files are already
committed as part of this repo. create_project() now deploys and commits
them alongside the existing .graphlemon/ init commit.

Also: unroutable execution failures were recorded as a bare
blocked_reason='execution_error' with no way to see what actually broke.
scheduler._block_task()/task_execution._mark_blocked() now persist the real
exception message into execution_event.payload (blocked_reason itself is
unchanged -- budget_guardrail matches its exact value elsewhere).

Found via a real console Start-button failure on a freshly created project;
see docs/implementation-gap-analysis.md for the full writeup.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Adds paths.repo_root()/is_self_target() and forces needs_worktree=True in
work_graph_orchestrate.run_graph() whenever target_folder_path is graphlemon's
own checkout, regardless of caller intent -- 05 §5.8's unconditional rule,
since a self-target in-place edit risks corrupting the currently-running
process's own source out from under it.

Adds src/core_engine_classify.py (path-based rule: .claude/agents/*.md,
src/*, supervisor/* = core_engine; frontend/, docs/ = not) and
bug_record.record_core_engine_change() (a PASS-path audit record, distinct
from create_bug_record's FAIL-path root-cause attribution -- no route
decision, status='resolved' from creation). scheduler.finish_task_lifecycle()
records this audit trail as an additive side effect after close_task_done()
on self-target PASS closures.

Found and resolved a real SDD-internal conflict (confirmed with the user via
AskUserQuestion) before wiring this up: supervisor-auto-rollback-design.md §1
triggers Deploy Supervisor on self-target+Validator-PASS alone, but 03-C
§3.11/04-P §4.6 literally gate it on an existing core_engine bug_record --
and task_create.py confirms goal-type self-improvement tasks (the exact
self-development-feasibility.md walkthrough scenario) never produce a
bug_record on their happy path, which would silently defeat that whole task
class under the literal reading. Resolved per user decision: trigger on
self-target+PASS alone; the bug_record is a byproduct, not a gate.
p_deploy_supervisor_blue_green (04-P §4.6) never touches task.status, so
close_task_done() still runs unconditionally -- Deploy Supervisor is a fully
decoupled state machine, matching §2's "no shared failure domain" principle.

org_loop_route.py is intentionally untouched -- fast_lane/full_replan is
FAIL-path retry routing, orthogonal to this PASS-path trigger; deployment
row creation stays U-037's job per its own Done criterion.

417 tests green; RealGitWorktreeIsolationTest (U-013/REQ-03's real git
worktree regression) re-verified passing per this Unit's own Done criterion.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

CLAUDE.md is explicit that PH5 (self-modification/auto-rollback) must not
start until PH4 has genuinely closed -- not just "all unit tests green,"
but 06-验收与测试锚点.md's REQ-01~14 happy/boundary/failure cases actually
covered. A REQ-by-REQ audit (the same discipline that caught U-031's
integration gap) found 5 real gaps:

- REQ-03 failure: work_node_execution.status jumped straight
  running->retrying, skipping 'reflecting' (declared in the schema enum,
  never once written by any code).
- REQ-05 failure / REQ-09 boundary: the "scheduler keeps serving other
  queued tasks past one blocked_pending_human" scenario was never tested
  against a real scheduler -- the old test literally said "no scheduler
  exists yet to test against directly" (stale since U-032).
- REQ-06 / REQ-07 / REQ-14 failure (the biggest gap, shared root cause):
  CorruptPersistedStateError, ClassifierBlockedError, and
  UsageLimitWaitTimeoutError were each correctly raised and each correctly
  marked their own work_node_execution 'blocked', but nothing ever routed
  that to task.status='blocked_pending_human' with the blocked_reason each
  REQ's acceptance text names. e2e_pipeline.run_end_to_end is split into
  run_task_lifecycle() so scheduler.py's run_goal_to_completion can create
  the task itself and catch these three specific exceptions, following the
  same UPDATE+execution_event pattern budget_guardrail.check_budget()
  already established for REQ-10.
- REQ-02 boundary: PRD internal-contradiction check (verifying->reflecting)
  had no implementation anywhere -- artifact_write always marked
  validation_status='passed' unconditionally. Added an optional verify_fn
  hook to work_graph_orchestrate.run_graph() (one regeneration attempt,
  then ConsistencyCheckFailedError) and a real consistency-check agent()
  call scoped to the PRD node only. The first real run of this new call
  caught a genuine bug: the model returned "consistent": "false" as a JSON
  string, not a boolean, which `if ok:` silently treated as truthy -- fixed
  with explicit _coerce_bool().
- REQ-13 failure: minor test-assertion weakness (WS 404 rejection test
  didn't check the JSON error body).

384 tests green. REQ-01~14 now have real, exercised coverage rather than a
"tests are green" assumption -- PH4 can be treated as genuinely closed.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

04-P's blocked_pending_human branch, 01-M's task-table indexing rationale,
and REQ-09's boundary case all presume an autonomous dispatcher scans queued
tasks and keeps serving the queue past a blocked one -- but no such component
existed. org_node_lifecycle._check_concurrency (U-010) only gates a single
trigger call; e2e_pipeline.run_end_to_end (U-014) drives exactly one task,
synchronously, only ever called from tests.

Adds src/scheduler.py: run_scheduler_once() dispatches multiple Goals via
ThreadPoolExecutor bounded by org_concurrency_config.max_concurrent_goals,
each worker opening its own sqlite connection (with a busy_timeout) rather
than sharing one across threads -- U-027's SIGSEGV already proved that needs
a hand-held lock, which would just serialize everything back to front.

Also closes a gap run_end_to_end's own docstring already flagged: it never
called close_task_done (PASS) or org_level_loop.run_fix_loop (FAIL), leaving
every task stuck at 'validating'. finish_task_lifecycle() applies that step
without changing run_end_to_end's own tested behavior or scope.

Real-write concurrency (multiple threads, separate connections, same db
file) is stress-tested for real in test_scheduler.py without spending API
budget -- the actually-new risk surface here is the threading/connection
design, not the already-validated (U-031/U-017) work itself.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>