宜博 / lemongraph

The 4th real self-hosting attempt (first to reach the actual doc-writing
Work Graph after the previous three bugs) hit a real git merge failure:
"local changes would be overwritten by merge", exit code 2.

Root cause: _check_prd_consistency() calls execute_work_graph() directly
against target_folder_path, completely bypassing work_node_run.run_work_node's
worktree isolation. It uses agentType='planner' with memory:project, which
writes a real update to .claude/agent-memory/planner/MEMORY.md in whatever
cwd it runs with. Every prior real run of this check only ever targeted a
throwaway clone, so this never mattered -- until self-targeting, where cwd
IS the main checkout: the memory write landed as an uncommitted local
change on master, which then blocked the SDD node's own (correctly
isolated) worktree merge right after it.

Fix: apply 05 §5.8's same unconditional self-target worktree rule here,
reusing work_node_run's existing create_worktree/ensure_worktree_committed/
merge_worktree/remove_worktree rather than inventing a new mechanism.
Non-self-target calls are unaffected (verified by both regression tests).

Also cleaned up the real, now-recovered state from the failed 4th attempt:
merged a real SDD-node worktree branch that got stranded uncommitted
(conflict was just two near-identical memory file wordings, resolved by
hand), and removed three orphaned worktrees left by earlier test runs.

557 tests green (437 src + 120 supervisor).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

The third real self-hosting attempt (first to actually reach stage_deployment
itself, past the earlier bootstrap/commit bugs) hit an uncaught
FileNotFoundError: _copy_prod_db() assumed prod_db_path always exists and
shutil.copy2()'d it unconditionally. graphlemon has never actually run as a
persistent service, so ~/.graphlemon/graphlemon.db never existed.
stage_deployment() already handles this exact class of "no prior state"
bootstrap case for previous_commit_hash/previous_instance_dir -- prod_db_path
never got the same treatment, and every existing test seeds a prod db in
setUp so this was never exercised.

Fix: when prod_db_path doesn't exist, create a fresh schema'd db instead of
copying -- semantically correct, since a genuinely-first deployment has no
production data to inherit. New regression test (BootstrapProdDbTest) covers
it. 555 tests green (435 src + 120 supervisor).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

A real self-hosting run (U-042, first real exercise of needs_worktree=True
since U-035 forces it unconditionally for self-target) exposed a real bug
RealGitWorktreeIsolationTest never caught: its own run_fn always committed
its edits, but a real Coder isn't instructed anywhere to git commit its own
changes. The Coder genuinely edited src/org_loop_route.py and verified it
in its own test_report -- but never committed, so merge_worktree()'s `git
merge` was a true no-op ("Already up to date"), and remove_worktree() then
silently discarded the dirty worktree along with the real work. code_diff
came out empty; the change never reached the branch at all.

Adds ensure_worktree_committed(): checks `git status --porcelain` in the
worktree before merging and auto-commits any dirty state found. A no-op
when the worktree is legitimately clean (e.g. a pure investigation task).
Same "判断用 Agent，机制用代码" principle already used elsewhere in this
codebase -- don't rely on persona prompt-engineering for a mechanical
correctness guarantee that the system itself should hold.

Two new regression tests: one reproducing the exact "edited but never
committed" scenario, one confirming a legitimately-clean worktree still
merges fine. 554 tests green (435 src + 119 supervisor).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

U-004 migrated the Hermes personas into agents/{planner,coder,validator}/SOUL.md
(the canonical source) but nothing had ever actually deployed the compiled
.claude/agents/*.md into graphlemon's own repo -- every real test throughout
PH1-PH4 ran against throwaway target-project clones that got this step via
project_scaffold, never against graphlemon self-targeting itself. The real
U-042 self-hosting attempt surfaced this immediately: `claude --agent planner`
failed outright with "--agent 'planner' not found". Deployed via
agent_profile.deploy_agent_definitions() and committed -- self-target worktree
isolation only sees committed content, so this has to be tracked, not just
written to disk, for any future self-target task to find its own persona.

Also commits the real agent-memory files a genuine self-target run generated
(architecture doc §8 decision 1: memory must be git-traceable) and gitignores
.claude/workflows/ (one-shot, uniquely-named dynamic-workflow scripts written
per Work Node call -- pure runtime clutter, never reused, not source content).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Adds src/deployment_query.py (list_deployments/get_current_deployment/
get_deployment -- no project_id filter needed, deployment has no project_id
column since there's exactly one graphlemon self-target) + three new
backend_server.py routes (GET /api/deployments, /current, /:id, the last
with recent deploy_health_signal history) + a new v_deployment_status page
in frontend/app.js polling every 5s (deployment.status changes don't go
through c_ws_subscribe -- Deploy Supervisor writes deployment/
deploy_health_signal directly, no execution_event of its own).

No 02-V spec exists for this view (PH5 was added after 02-V was written) --
DOM/class conventions follow the existing pages' own patterns. Verified
against a real running server via urllib (browser extension unavailable
this session), plus 9 deployment_query unit tests and 5 real HTTP route
tests.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Adds supervisor/self_hosting_e2e.py: run_self_hosting_closed_loop() chains
every PH5 module built this session (U-034~U-041) with the Graph Engine's
own scheduler (U-032) into one real, callable path matching self-development-
feasibility.md's 9-step scenario -- self-target Goal -> Planner/Coder/
Validator -> core_engine classification -> stage -> health check -> cutover
(or rollback path A if health check fails) -> monitor until either
monitoring_window_clean (finalized) or anomaly_detected (rollback path B).
Both real outcome paths are branches of the same call, not two separate
scripted scenarios. Explicitly documented as this acceptance scenario's own
orchestration glue, not a claim about Deploy Supervisor's eventual
production poll loop (U-036's discovery-loop hook is still unfilled;
building that is future work).

Mocked wiring tests (6, run in normal regression) stub each already
independently-tested module boundary and only verify the branching logic.

The real, literal acceptance test is built and ready (RealSelfHostingClosedLoopTest,
reproduces the document's own "loop_round > 3 should be >= 3" example) but
gated behind RUN_REAL_SELF_HOSTING_E2E=1 -- deliberately a separate,
distinctly-named env var from this project's usual RUN_REAL_AGENT_E2E, since
this is the highest-blast-radius action in the project (can replace the
running graphlemon process and git-revert commits on this real repo).
Per this Unit's own explicit build instruction, not executed as part of
implementing it -- doing so requires checking with the user first.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Adds supervisor/deploy_rollback.py: rollback_path_a (pre-cutover, health
check failed -- terminate candidate, delete its one-time DB copy, git
revert, record a bug_record(core_engine) audit trail) and rollback_path_b
(post-cutover, anomaly detected -- stop the bad Live, revive
previous_instance_dir, re-run the same three health-check signals directly
against it, git revert only once re-verification passes).

Design choices follow established conventions rather than inventing new
ones: no unified rollback() entry point (deployment has no PID columns,
same reason deploy_stage/deploy_cutover already require callers to pass
pids explicitly); ROLLBACK_FAILED is the raised error name while the
persisted deployment.status is 'blocked_pending_human' (same pattern
CheckoutFailedError/CutoverFailedError already use). git revert conflicts
fold into the same rollback-itself-failed fallback both paths share, per
design doc §10 open question 3's own recommendation, with `git revert
--abort` run afterward so a failed attempt doesn't leave the repo mid-conflict.

This is PH5's highest-rework-risk Unit per development-plan.md's own flag,
so its own "rollback itself fails" fallback is tested for real, not assumed:
a genuinely broken previous_instance_dir (a real empty directory with no
Python package in it) that really fails to bind its port, and a genuinely
conflicting git history (a later commit touching the same line the
self-modification changed) that really fails `git revert` -- both correctly
land on blocked_pending_human without retrying forever or silently
succeeding, and the conflict case leaves the repo clean afterward.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Adds supervisor/deploy_monitor.py: run_monitor_cycle() is one sampling
pass, called repeatedly by Deploy Supervisor's own poll loop (03-C's own
text: not a one-shot call). process_alive/http_health/canary_task/
validator_approval_rate sampled in order, stopping at the first anomaly.

validator_approval_rate's "approve rate" has no literal definition anywhere
in 01-M/03-C/04-P (no such execution_event type exists) -- derived instead
from a verified structural fact in scheduler.finish_task_lifecycle: PASS
closes straight to done with no bug_record ever created, FAIL/INCONCLUSIVE
always creates one via judge_root_cause. A task counts as approved iff no
bug_record(source='validator') exists for it. Baseline is computed by the
same function looking back MONITORING_WINDOW_HOURS from cutover_at rather
than persisted as a new column; baseline=None (cold start) never flags an
anomaly rather than defaulting to 0.

Crash-count/consecutive-http-fail windows are approximated by sample count
(N = CRASH_WINDOW / MONITOR_SAMPLE_INTERVAL) rather than a true rolling
time window -- documented simplification, correct as long as sampling stays
roughly interval-paced.

Caught my own design misreading via the tests: process_alive's monitoring-
phase anomaly condition is MAX_CRASH_COUNT crashes within CRASH_WINDOW (a
crash-loop detector), not "currently not alive" -- unlike U-038's health
check where any single failure fails closed. A single dead reading during
monitoring is recorded but must not trigger an immediate rollback by itself.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Adds supervisor/deploy_cutover.py: stops the old Live process, retires the
health-check-mode candidate, and relaunches the same already-verified
candidate checkout bound to LIVE_PORT/PROD_DB_PATH -- http.server has no
live socket-rebind operation, so "switch to production port/db" is
implemented as stop-then-relaunch-from-the-same-dir rather than an in-place
rebind (functionally equivalent, the only mechanism stdlib actually offers).

Adds LIVE_PORT=8420 to 05 §5.9's MVP-fixed parameter table -- the one
parameter that table was missing (frontend/app.js never needed one, same-
origin relative fetches don't care, but successive deployments need to keep
landing on the same port for a human bookmark to keep working).

"在途 work_node_execution 重启后可恢复" is satisfied by relying on the
existing REQ-06 recovery machinery unchanged -- cutover doesn't add any
special-case logic, a cutover-triggered restart is just another interruption
REQ-06 already survives.

A real (not mocked) test caught a real bug: deploy_supervisor.is_alive()'s
os.kill(pid, 0) check can't distinguish "truly running" from "exited but
never reaped by its own parent" -- a zombie's PID slot answers kill(pid, 0)
successfully until waited on. U-036's own subprocess test never caught this
because that skeleton deletes its own PID file before exiting, sidestepping
is_alive() entirely; deploy_cutover's test is the first one to actually poll
for a stopped process to disappear, and it hung until fixed. Added
reap_if_exited() (real os.waitpid for processes we spawned ourselves, falls
back to is_alive() for pids from a prior session where ChildProcessError
means we're not the real parent) and switched deploy_cutover._stop_process,
deploy_health_check.check_process_alive, and deploy_supervisor.stop() to use
it.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Adds supervisor/deploy_health_check.py: run_health_check() runs the three
pre-cutover checks in order, stopping at the first failure (03-C: "任一检查项
fail 立即整体判定 health_check_failed，不等待其余检查项跑完") and persisting
a deploy_health_signal row for every check that actually ran.

- process_alive: polls deploy_supervisor.is_alive() up to HEALTH_CHECK_STARTUP_TIMEOUT.
- http_health: hits the candidate's /api/health -- added to backend_server.py
  (U-037 deliberately left this for U-038). Not just "process exists" --
  runs a real SELECT 1 to catch a wedged sqlite connection a live-but-stuck
  process would otherwise pass.
- canary_task: a real (not mocked) project+task creation round-trip against
  the candidate over HTTP -- no real Org Graph/claude calls, since
  c_task_create's synchronous 'queued' landing is already a deterministic,
  always-should-pass result, which is what a canary needs.

Timeouts are the MVP-fixed drafts from 05 §5.9, explicitly flagged there as
needing real calibration later, not hardcoded as authoritative.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Adds supervisor/deploy_stage.py: stage_deployment() creates a deployment row,
git-worktree-checks-out the target commit into a fresh directory (--detach,
never in-place git pull), copies the production DB to a one-time candidate
copy, and Popens a real candidate backend process on a free port.

Return value follows 04-P §4.6's state-machine table (status after the call
is 'candidate_starting') rather than 03-C's one-line 'staged' summary --
04-P's own trigger text ("c_deploy_supervisor_stage 完成" moves the row to
candidate_starting) is the more detailed/authoritative source when the two
disagree, same precedent used throughout this project.

Found and closed two real gaps along the way:
- deployment's own field list never included where a candidate's checkout
  directory lives -- added candidate_instance_dir to schema.py (also what
  the NEXT staging call's previous_instance_dir bootstraps from once a
  deployment reaches 'finalized').
- backend_server.py had no CLI entry point at all since U-007 -- create_server()
  was only ever instantiated in-process by tests. Added main() (argparse +
  serve_forever + SIGTERM->shutdown() from a separate thread, per
  http.server's own same-thread-deadlock warning).

RealCandidateLaunchTest spawns and HTTP-probes a genuine
`python3 -m src.backend_server` subprocess against a real git-worktree
checkout of this repo (not mocked) -- caught a real ordering issue doing so:
git worktree only sees committed content, so testing before this Unit's own
commit landed would check out backend_server.py without main() at all.
Fixed by using `git stash create` (builds a commit object from the current
working tree without touching HEAD/index) as the commit-to-test-against,
falling back to plain HEAD once the tree is clean post-commit.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Ports graphloop's test_isolated_coding_workspace.py/test_workspace_security.py/
test_change_manifest.py/test_selective_apply.py into supervisor/isolation/
(only the import path changed -- the graphlemon modules were already adapted
verbatim from graphloop, confirmed API-identical) plus a few extra edge
cases the originals didn't cover. 52 tests green, replacing what had been a
single manual smoke test since these modules were staged.

Adds supervisor/deploy_supervisor.py: process lifecycle only (PID file,
atomic status file, start/stop/is_running), no deploy decision logic yet --
that's U-037+'s job, injected via the poll_fn hook run_foreground() calls on
a schedule. PID/status conventions adapted (not imported) from graphloop's
hermes_watchdog.py atomic-write pattern. Fail-closed: any exit path
(normal, SIGTERM, or an exception from poll_fn) unconditionally writes
status='stopped' and removes the PID file, so is_running() never lies.
RealSubprocessLifecycleTest actually spawns and stops a real
`python3 -m supervisor.deploy_supervisor` process rather than mocking it,
per this Unit's own "能独立启停" Done criterion.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Adds paths.repo_root()/is_self_target() and forces needs_worktree=True in
work_graph_orchestrate.run_graph() whenever target_folder_path is graphlemon's
own checkout, regardless of caller intent -- 05 §5.8's unconditional rule,
since a self-target in-place edit risks corrupting the currently-running
process's own source out from under it.

Adds src/core_engine_classify.py (path-based rule: .claude/agents/*.md,
src/*, supervisor/* = core_engine; frontend/, docs/ = not) and
bug_record.record_core_engine_change() (a PASS-path audit record, distinct
from create_bug_record's FAIL-path root-cause attribution -- no route
decision, status='resolved' from creation). scheduler.finish_task_lifecycle()
records this audit trail as an additive side effect after close_task_done()
on self-target PASS closures.

Found and resolved a real SDD-internal conflict (confirmed with the user via
AskUserQuestion) before wiring this up: supervisor-auto-rollback-design.md §1
triggers Deploy Supervisor on self-target+Validator-PASS alone, but 03-C
§3.11/04-P §4.6 literally gate it on an existing core_engine bug_record --
and task_create.py confirms goal-type self-improvement tasks (the exact
self-development-feasibility.md walkthrough scenario) never produce a
bug_record on their happy path, which would silently defeat that whole task
class under the literal reading. Resolved per user decision: trigger on
self-target+PASS alone; the bug_record is a byproduct, not a gate.
p_deploy_supervisor_blue_green (04-P §4.6) never touches task.status, so
close_task_done() still runs unconditionally -- Deploy Supervisor is a fully
decoupled state machine, matching §2's "no shared failure domain" principle.

org_loop_route.py is intentionally untouched -- fast_lane/full_replan is
FAIL-path retry routing, orthogonal to this PASS-path trigger; deployment
row creation stays U-037's job per its own Done criterion.

417 tests green; RealGitWorktreeIsolationTest (U-013/REQ-03's real git
worktree regression) re-verified passing per this Unit's own Done criterion.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>