Commit 5a046a4fb8d35a2f542a1c37e3cd75e635428e98

Authored by bboonnee
1 parent abfb3cb5

U-037: c_deploy_supervisor_stage (candidate checkout + one-time DB copy + launch)

Adds supervisor/deploy_stage.py: stage_deployment() creates a deployment row,
git-worktree-checks-out the target commit into a fresh directory (--detach,
never in-place git pull), copies the production DB to a one-time candidate
copy, and Popens a real candidate backend process on a free port.

Return value follows 04-P §4.6's state-machine table (status after the call
is 'candidate_starting') rather than 03-C's one-line 'staged' summary --
04-P's own trigger text ("c_deploy_supervisor_stage 完成" moves the row to
candidate_starting) is the more detailed/authoritative source when the two
disagree, same precedent used throughout this project.

Found and closed two real gaps along the way:
- deployment's own field list never included where a candidate's checkout
  directory lives -- added candidate_instance_dir to schema.py (also what
  the NEXT staging call's previous_instance_dir bootstraps from once a
  deployment reaches 'finalized').
- backend_server.py had no CLI entry point at all since U-007 -- create_server()
  was only ever instantiated in-process by tests. Added main() (argparse +
  serve_forever + SIGTERM->shutdown() from a separate thread, per
  http.server's own same-thread-deadlock warning).

RealCandidateLaunchTest spawns and HTTP-probes a genuine
`python3 -m src.backend_server` subprocess against a real git-worktree
checkout of this repo (not mocked) -- caught a real ordering issue doing so:
git worktree only sees committed content, so testing before this Unit's own
commit landed would check out backend_server.py without main() at all.
Fixed by using `git stash create` (builds a commit object from the current
working tree without touching HEAD/index) as the commit-to-test-against,
falling back to plain HEAD once the tree is clean post-commit.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
docs/development-plan.md
... ... @@ -361,6 +361,9 @@ PH2-S1/S2/S3/S4 之间除标注的依赖外相互独立,可并行推进。
361 361 - **U-037 [Risk Gate]** `c_deploy_supervisor_stage`:候选实例拉起(全新目录 checkout,不 `git pull` 原地改)+ 一次性 DB 副本机制(候选实例验证期间绝不接触生产库)
362 362 - 依赖:U-034, U-036
363 363 - Done:给定一个已合并到主分支的 commit,能生成 `deployment` 记录(`status=staged`)并拉起一个监听临时端口、连接一次性 DB 副本的候选进程
  364 + - **实现补充(2026-08-02)**:新增 `supervisor/deploy_stage.py`:`stage_deployment()` 建 `deployment` 行、`git worktree add --detach`(不是 `-b <branch>`,候选检出是只读验证用途,不打算再接受提交,和 `work_node_run.create_worktree()` 的编码用 worktree 是两回事)到全新目录、复制生产 DB 到一次性副本、`subprocess.Popen` 拉起候选进程。**返回值订正(低风险,按判断处理未升级询问)**:03-C `output_contract` 字面写返回 `{deployment_id, status:'staged'}`,但 04-P §4.6 状态转移表明确写"`c_deploy_supervisor_stage` 完成"这个触发条件本身就是把 `deployment.status` 从 `staged` 推进到 `candidate_starting`——两处矛盾,按"更详细的 04-P 状态机表为准"处理,函数完成时返回并持久化的都是 `candidate_starting`,不是字面上的 `staged`。**发现并补上两个真实缺口**:① `deployment` 表 01-M 字段清单里完全没有"候选检出目录存在哪"这一列——`candidate_port`/`candidate_db_path` 都有,唯独检出目录没地方存,Deploy Supervisor 若在 staging 和 cutover 之间重启就会彻底丢失候选目录位置;`schema.py` 补了 `candidate_instance_dir` 列(同一次 finalized 部署的这个字段,就是下一次 staging 时 `previous_instance_dir` 的来源——candidate 一旦顺利转正就是新 Live,回滚要回退到的正是它)。② `backend_server.py` 从 U-007 建骨架以来就没有任何 CLI 入口——`create_server()` 只在测试里以进程内方式实例化过,从没人真正把它作为独立 OS 进程启动过;候选进程恰好需要这个能力,顺手补了 `main()`(`python3 -m src.backend_server --host --port --db-path --skill-repo-path`,`SIGTERM` 走 `server.shutdown()`——必须在另一个线程调用,同线程调用 `serve_forever()` 期间调 `shutdown()` 会按 `http.server` 自己的文档死锁)。`/api/health` 端点本身留给 U-038 自己去加,不在这里超范围实现。
  365 + - **验证方式**:`BootstrapPreviousTest`/`StageDeploymentHappyPathTest`/`CheckoutFailedTest` 用真实 sqlite + 真实 git worktree 操作(只 mock 掉真正拉候选进程这一步,验证 `stage_deployment()` 自己的逻辑);`RealCandidateLaunchTest` 完全不 mock——对着 graphlemon 自己仓库真实 `git worktree add --detach` 检出 + 真实 `python3 -m src.backend_server` 子进程拉起 + 真实 HTTP 请求验证候选进程确实在真实监听、真实响应,验证完 `SIGTERM`(超时兜底 `SIGKILL`)+ `git worktree remove` 清理干净。这个真实测试意外命中一个开发时序问题:`git worktree add` 只看已提交内容,而这次实现是先写代码再测试再提交(标准流程),测试跑的时候 `main()` 还没提交,导致 checkout 出来的 `backend_server.py` 是没有 `main()` 的旧版本,候选进程"启动"后什么也不做直接退出——用 `git stash create`(不碰工作区/HEAD,只生成一个代表当前工作区状态的 commit 对象)替代直接 `HEAD`,让测试始终对着"当前工作区实际内容"验证,而不是"上一次提交的内容";提交后本地工作区变干净时 `git stash create` 打印空字符串,自动回退到普通 `HEAD`,两种时序都覆盖。
  366 + - 覆盖:03-C `c_deploy_supervisor_stage`;01-M `deployment.candidate_instance_dir`(补字段);`backend_server.py` 新增可独立运行的 CLI 入口
364 367  
365 368 - **U-038 [Risk Gate]** `c_deploy_supervisor_health_check`:`process_alive`/`http_health`/`canary_task` 三项 pre-cutover 检查
366 369 - 依赖:U-037
... ...
docs/sdd/unit_matrix.json
... ... @@ -54,7 +54,7 @@
54 54 {"id": "U-034", "phase": "PH5", "sub_phase": "PH5-S1", "name": "01-M 扩展:deployment/deploy_health_signal + core_engine 枚举(见 development-plan.md 2026-08-02 实现补充)", "classification": "Enabler", "depends_on": ["U-001"], "coverage": {"model": ["deployment", "deploy_health_signal", "bug_record"], "view": [], "controller": [], "process": [], "requirement": []}, "done": "两张新表建表成功(共 17 张);bug_record 新枚举值向后兼容 U-015/U-018 已有测试;deployment 状态机用 01-M 1.15 完整 15 值枚举;ux_deployment_active 表达式索引强制全局单例非终态约束;bug_record.py 应用层白名单刻意未跟着放开 core_engine,classification+路由逻辑留给 U-035"},
55 55 {"id": "U-035", "phase": "PH5", "sub_phase": "PH5-S1", "name": "self-target worktree 无条件强制 + core_engine 分类审计(见 development-plan.md 2026-08-02 实现补充)", "classification": "Risk Gate", "depends_on": ["U-013", "U-016", "U-034"], "coverage": {"model": [], "view": [], "controller": [], "process": [], "requirement": ["REQ-03"]}, "done": "self-target 时无条件走 worktree(work_graph_orchestrate.run_graph 内强制);core_engine 分类(src/core_engine_classify.py,路径规则)在 self-target 任务 PASS 收尾时记一条审计用 bug_record;且改动落地后重跑 U-013/REQ-03(worktree 隔离)现有回归用例全部通过。**2026-08-02 用户决策修正**:Deploy Supervisor 触发条件按 supervisor-auto-rollback-design.md §1 字面设计(self-target+PASS 即触发,bug_record 是旁路产物不是前提),而非 03-C/04-P 字面文本要求的\"先有 core_engine bug_record\"——后者会让从不产生 bug_record 的 goal 型自我改进任务永远触发不了 Deploy Supervisor,已用 AskUserQuestion 与用户确认此订正。c_org_loop_route 未改动(fast_lane/full_replan 是 FAIL 路径重试路由,与此处 PASS 路径的蓝绿触发正交),deployment 行创建仍是 U-037 的范围,不在本 Unit 提前做"},
56 56 {"id": "U-036", "phase": "PH5", "sub_phase": "PH5-S2", "name": "supervisor/isolation 补齐单测 + Deploy Supervisor 进程骨架(见 development-plan.md 2026-08-02 实现补充)", "classification": "Enabler", "depends_on": [], "coverage": {"model": [], "view": [], "controller": [], "process": [], "requirement": []}, "done": "四模块各有 test_*.py(供 python3 -m unittest discover -s supervisor/isolation 运行,52 测试全绿);Deploy Supervisor 骨架(supervisor/deploy_supervisor.py)独立进程可启停(真实子进程验证,非 mock),不受 Live/Candidate 崩溃影响;本 Unit 只做进程生命周期骨架,不含任何真实部署决策逻辑(U-037+ 的 poll_fn 挂载点)"},
57   - {"id": "U-037", "phase": "PH5", "sub_phase": "PH5-S2", "name": "c_deploy_supervisor_stage", "classification": "Risk Gate", "depends_on": ["U-034", "U-036"], "coverage": {"model": ["deployment"], "view": [], "controller": ["c_deploy_supervisor_stage"], "process": [], "requirement": []}, "done": "能生成 deployment 记录并拉起监听临时端口、连一次性 DB 副本的候选进程"},
  57 + {"id": "U-037", "phase": "PH5", "sub_phase": "PH5-S2", "name": "c_deploy_supervisor_stage(见 development-plan.md 2026-08-02 实现补充)", "classification": "Risk Gate", "depends_on": ["U-034", "U-036"], "coverage": {"model": ["deployment"], "view": [], "controller": ["c_deploy_supervisor_stage"], "process": [], "requirement": []}, "done": "能生成 deployment 记录并拉起监听临时端口、连一次性 DB 副本的候选进程(supervisor/deploy_stage.py,真实 git worktree + 真实子进程验证过);顺带补上 deployment.candidate_instance_dir 缺字段 + backend_server.py 首个 CLI 入口(此前只能进程内启动,从未真正作为独立 OS 进程跑过)"},
58 58 {"id": "U-038", "phase": "PH5", "sub_phase": "PH5-S2", "name": "c_deploy_supervisor_health_check", "classification": "Risk Gate", "depends_on": ["U-037"], "coverage": {"model": [], "view": [], "controller": ["c_deploy_supervisor_health_check"], "process": [], "requirement": []}, "done": "process_alive/http_health/canary_task 三项检查全过 -> health_check_passed;任一失败 -> health_check_failed 且 Live 未受影响"},
59 59 {"id": "U-039", "phase": "PH5", "sub_phase": "PH5-S2", "name": "c_deploy_supervisor_cutover", "classification": "Risk Gate", "depends_on": ["U-038"], "coverage": {"model": ["deployment"], "view": [], "controller": ["c_deploy_supervisor_cutover"], "process": [], "requirement": []}, "done": "健康检查通过后完成真实切换;在途 work_node_execution 重启后可恢复;previous_instance_dir 保留至监控窗口结束"},
60 60 {"id": "U-040", "phase": "PH5", "sub_phase": "PH5-S3", "name": "c_deploy_supervisor_monitor", "classification": "Risk Gate", "depends_on": ["U-039"], "coverage": {"model": ["deploy_health_signal"], "view": [], "controller": ["c_deploy_supervisor_monitor"], "process": [], "requirement": []}, "done": "四类信号采样正常;validator_approval_rate 偏离基线阈值时触发 anomaly_detected"},
... ...
src/backend_server.py
... ... @@ -675,3 +675,45 @@ def create_server(host=&quot;127.0.0.1&quot;, port=0, db_path=None, skill_repo_path=None):
675 675 server.conn_lock = threading.Lock()
676 676 server.skill_repo_path = skill_repo_path if skill_repo_path is not None else skill_query._REPO_ROOT
677 677 return server
  678 +
  679 +
  680 +def main():
  681 + """U-037: CLI entry point (`python3 -m src.backend_server ...`) -- until
  682 + this Unit, `create_server()` was only ever instantiated in-process by
  683 + tests; there was no way to actually launch graphlemon's own backend as a
  684 + standalone OS process. c_deploy_supervisor_stage needs exactly that (a
  685 + real candidate process, `subprocess.Popen`, listening on a real port) --
  686 + found while building it, added here since it's the natural, minimal home
  687 + for "run this server for real" regardless of who launches it.
  688 + """
  689 + import argparse
  690 + import signal
  691 +
  692 + parser = argparse.ArgumentParser()
  693 + parser.add_argument("--host", default="127.0.0.1")
  694 + parser.add_argument("--port", type=int, default=0)
  695 + parser.add_argument("--db-path", default=None)
  696 + parser.add_argument("--skill-repo-path", default=None)
  697 + args = parser.parse_args()
  698 +
  699 + server = create_server(args.host, args.port, args.db_path, args.skill_repo_path)
  700 +
  701 + def _handle_sigterm(signum, frame):
  702 + # server.shutdown() blocks until serve_forever()'s loop notices and
  703 + # returns -- calling it from the SAME thread that's running
  704 + # serve_forever() (the signal handler runs on the main thread, same
  705 + # as serve_forever() below) deadlocks, per http.server's own docs.
  706 + threading.Thread(target=server.shutdown, daemon=True).start()
  707 +
  708 + signal.signal(signal.SIGTERM, _handle_sigterm)
  709 + print("graphlemon backend listening on {}:{}".format(*server.server_address), flush=True)
  710 + try:
  711 + server.serve_forever()
  712 + except KeyboardInterrupt:
  713 + pass
  714 + finally:
  715 + server.server_close()
  716 +
  717 +
  718 +if __name__ == "__main__":
  719 + main()
... ...
src/schema.py
... ... @@ -289,6 +289,15 @@ CREATE TABLE deployment (
289 289 )),
290 290 candidate_port INTEGER,
291 291 candidate_db_path TEXT,
  292 + -- Not in supervisor-auto-rollback-design.md §3's field table (found
  293 + -- missing while building U-037's c_deploy_supervisor_stage): the
  294 + -- checkout directory a candidate is actually running from has to be
  295 + -- persisted somewhere, or a Deploy Supervisor restart between staging
  296 + -- and cutover would lose track of it entirely -- the same class of gap
  297 + -- REQ-06 exists to prevent everywhere else. Once a deployment reaches
  298 + -- 'live'/'finalized', this same directory IS what the next deployment's
  299 + -- `previous_instance_dir` bootstraps from (see deploy_stage.py).
  300 + candidate_instance_dir TEXT,
292 301 started_at TEXT,
293 302 cutover_at TEXT,
294 303 monitoring_closed_at TEXT,
... ...
supervisor/deploy_stage.py 0 → 100644
  1 +"""c_deploy_supervisor_stage (03-C §3.15 / 04-P §4.6): candidate-instance
  2 +staging -- the entry point Deploy Supervisor's own poll loop (the `poll_fn`
  3 +hook `deploy_supervisor.run_foreground()` calls, U-036) invokes once it
  4 +discovers a self-target merge ready to deploy (the `bug_record(root_cause_
  5 +category=core_engine)` audit trail U-035 records is that discovery signal --
  6 +this module doesn't do the discovering itself, it's handed a commit hash by
  7 +whatever does).
  8 +
  9 +**Return-value note (documented deviation, low-stakes -- resolved by
  10 +judgment, not escalated)**: 03-C's `output_contract` literally says this
  11 +returns `{deployment_id, status: 'staged'}`, but 04-P §4.6's own transition
  12 +table says the trigger "c_deploy_supervisor_stage 完成" is what moves
  13 +`deployment.status` FROM 'staged' TO 'candidate_starting' -- i.e. by the
  14 +time this function's side effects (checkout, DB copy, candidate launch) are
  15 +done and it's ready to return, the row's real, persisted status is already
  16 +'candidate_starting', not 'staged'. 04-P's full state-machine table is more
  17 +detailed/authoritative here than 03-C's one-line summary (same "SDD 内部冲突
  18 +以更详细的为准" precedent CLAUDE.md states), so this returns the row's actual
  19 +post-call status rather than literally echoing 'staged'.
  20 +
  21 +**Missing field found and added (see schema.py's own comment)**:
  22 +`deployment.candidate_instance_dir` -- 01-M's own field table never listed
  23 +where a candidate's checkout directory persists, even though cutover/
  24 +rollback both need to know it and a Deploy Supervisor restart between
  25 +staging and cutover would otherwise lose track of it entirely.
  26 +"""
  27 +
  28 +import os
  29 +import shutil
  30 +import socket
  31 +import subprocess
  32 +import sys
  33 +import uuid
  34 +
  35 +from src import paths
  36 +
  37 +
  38 +class CheckoutFailedError(Exception):
  39 + """CHECKOUT_FAILED -- 03-C §3.15: fresh-directory checkout failed
  40 + (e.g. disk full). Not retried -- the caller routes this to
  41 + `deployment.status='blocked_pending_human'`, already applied by the
  42 + time this is raised."""
  43 +
  44 +
  45 +def _last_finalized_deployment(conn):
  46 + """Returns (git_commit_hash, candidate_instance_dir) of the most recent
  47 + `finalized` deployment, or None if there isn't one yet (the bootstrap
  48 + case schema.py's own comment already flags: the very first deployment
  49 + ever has no prior finalized deployment to point at)."""
  50 + row = conn.execute(
  51 + "SELECT git_commit_hash, candidate_instance_dir FROM deployment "
  52 + "WHERE status = 'finalized' ORDER BY rowid DESC LIMIT 1"
  53 + ).fetchone()
  54 + return tuple(row) if row else None
  55 +
  56 +
  57 +def _current_commit(repo_path):
  58 + return subprocess.run(
  59 + ["git", "rev-parse", "HEAD"], cwd=repo_path, capture_output=True, text=True, check=True
  60 + ).stdout.strip()
  61 +
  62 +
  63 +def _find_free_port():
  64 + with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock:
  65 + sock.bind(("127.0.0.1", 0))
  66 + return sock.getsockname()[1]
  67 +
  68 +
  69 +def _candidates_dir():
  70 + d = os.path.join(paths.platform_home(), "deploy_supervisor", "candidates")
  71 + os.makedirs(d, exist_ok=True)
  72 + return d
  73 +
  74 +
  75 +def _candidate_dbs_dir():
  76 + d = os.path.join(paths.platform_home(), "deploy_supervisor", "candidate-dbs")
  77 + os.makedirs(d, exist_ok=True)
  78 + return d
  79 +
  80 +
  81 +def _checkout_candidate(repo_root, git_commit_hash, deployment_id, checkout_parent_dir=None):
  82 + """`git worktree add --detach` (not `-b <branch>`, unlike work_node_run.
  83 + create_worktree's coding worktrees -- a candidate checkout is read-only
  84 + verification, never meant to receive further commits of its own) into a
  85 + FRESH directory, never an in-place `git pull` (03-C §3.15's own wording,
  86 + since the currently-running Live process may still be reading repo_root
  87 + off disk)."""
  88 + parent = checkout_parent_dir or _candidates_dir()
  89 + candidate_dir = os.path.join(parent, deployment_id)
  90 + subprocess.run(
  91 + ["git", "worktree", "add", "--detach", candidate_dir, git_commit_hash],
  92 + cwd=repo_root, check=True, capture_output=True, text=True,
  93 + )
  94 + return candidate_dir
  95 +
  96 +
  97 +def _copy_prod_db(prod_db_path, deployment_id):
  98 + """MVP: whole-file copy (supervisor-auto-rollback-design.md §10 open
  99 + question 2 already flags this as the simple-first choice, revisit only
  100 + if a large production DB makes this slow)."""
  101 + dest = os.path.join(_candidate_dbs_dir(), deployment_id + ".db")
  102 + shutil.copy2(prod_db_path, dest)
  103 + return dest
  104 +
  105 +
  106 +def _launch_candidate(candidate_dir, port, db_path, python_executable=None):
  107 + python_executable = python_executable or sys.executable
  108 + return subprocess.Popen(
  109 + [python_executable, "-m", "src.backend_server", "--host", "127.0.0.1", "--port", str(port), "--db-path", db_path],
  110 + cwd=candidate_dir, stdin=subprocess.DEVNULL, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL,
  111 + )
  112 +
  113 +
  114 +def stage_deployment(
  115 + conn, git_commit_hash, repo_root=None, prod_db_path=None,
  116 + checkout_parent_dir=None, port=None, python_executable=None,
  117 +):
  118 + """Returns {deployment_id, status, candidate_port, candidate_pid,
  119 + candidate_instance_dir}. Raises CheckoutFailedError (deployment row
  120 + already marked blocked_pending_human by the time it's raised) if the
  121 + fresh-directory checkout fails."""
  122 + repo_root = repo_root or paths.repo_root()
  123 + prod_db_path = prod_db_path or paths.db_path()
  124 + deployment_id = str(uuid.uuid4())
  125 +
  126 + last = _last_finalized_deployment(conn)
  127 + if last is not None:
  128 + previous_commit_hash, previous_instance_dir = last
  129 + else:
  130 + # Bootstrap: no prior finalized deployment exists yet -- the
  131 + # currently-running instance IS the "previous known-good" to roll
  132 + # back to, per schema.py's own note.
  133 + previous_commit_hash = _current_commit(repo_root)
  134 + previous_instance_dir = repo_root
  135 +
  136 + conn.execute(
  137 + "INSERT INTO deployment "
  138 + "(deployment_id, git_commit_hash, previous_commit_hash, status, previous_instance_dir, started_at) "
  139 + "VALUES (?, ?, ?, 'staged', ?, CURRENT_TIMESTAMP)",
  140 + (deployment_id, git_commit_hash, previous_commit_hash, previous_instance_dir),
  141 + )
  142 + conn.commit()
  143 +
  144 + try:
  145 + candidate_dir = _checkout_candidate(repo_root, git_commit_hash, deployment_id, checkout_parent_dir)
  146 + except subprocess.CalledProcessError as e:
  147 + conn.execute(
  148 + "UPDATE deployment SET status = 'blocked_pending_human' WHERE deployment_id = ?", (deployment_id,)
  149 + )
  150 + conn.commit()
  151 + raise CheckoutFailedError(
  152 + "fresh-directory checkout failed for deployment {} at commit {}: {}".format(
  153 + deployment_id, git_commit_hash, e.stderr
  154 + )
  155 + ) from e
  156 +
  157 + candidate_db_path = _copy_prod_db(prod_db_path, deployment_id)
  158 + candidate_port = port if port is not None else _find_free_port()
  159 + process = _launch_candidate(candidate_dir, candidate_port, candidate_db_path, python_executable)
  160 +
  161 + conn.execute(
  162 + "UPDATE deployment SET status = 'candidate_starting', candidate_port = ?, candidate_db_path = ?, "
  163 + "candidate_instance_dir = ? WHERE deployment_id = ?",
  164 + (candidate_port, candidate_db_path, candidate_dir, deployment_id),
  165 + )
  166 + conn.commit()
  167 +
  168 + return {
  169 + "deployment_id": deployment_id, "status": "candidate_starting",
  170 + "candidate_port": candidate_port, "candidate_pid": process.pid,
  171 + "candidate_instance_dir": candidate_dir,
  172 + }
... ...
supervisor/test_deploy_stage.py 0 → 100644
  1 +"""U-037 tests: c_deploy_supervisor_stage. Real sqlite, real git worktree
  2 +operations, real one-time DB file copy -- only `_launch_candidate` (spawning
  3 +yet another real backend_server subprocess) is mocked in most tests, since
  4 +that's the expensive/slow part and is proven for real in exactly one test
  5 +(RealCandidateLaunchTest) rather than every test.
  6 +
  7 +Run: python3 -m unittest discover -s supervisor -p "test_*.py" -v
  8 +"""
  9 +
  10 +import os
  11 +import shutil
  12 +import subprocess
  13 +import sys
  14 +import tempfile
  15 +import time
  16 +import unittest
  17 +import urllib.error
  18 +import urllib.request
  19 +from unittest.mock import patch
  20 +
  21 +sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
  22 +
  23 +from supervisor import deploy_stage
  24 +from src import paths, schema
  25 +
  26 +
  27 +def _init_git_repo(path):
  28 + subprocess.run(["git", "init", "-q"], cwd=path, check=True)
  29 + subprocess.run(["git", "config", "user.email", "test@graphlemon.local"], cwd=path, check=True)
  30 + subprocess.run(["git", "config", "user.name", "test"], cwd=path, check=True)
  31 +
  32 +
  33 +def _commit_all(path, message):
  34 + subprocess.run(["git", "add", "-A"], cwd=path, check=True)
  35 + subprocess.run(["git", "commit", "-q", "-m", message], cwd=path, check=True)
  36 + return subprocess.run(
  37 + ["git", "rev-parse", "HEAD"], cwd=path, capture_output=True, text=True, check=True
  38 + ).stdout.strip()
  39 +
  40 +
  41 +class DeployStageTestBase(unittest.TestCase):
  42 + def setUp(self):
  43 + self.home_tmp = tempfile.mkdtemp(prefix="deploy-stage-home-")
  44 + self.addCleanup(shutil.rmtree, self.home_tmp, ignore_errors=True)
  45 + self.old_home = os.environ.get("GRAPHLEMON_HOME")
  46 + os.environ["GRAPHLEMON_HOME"] = self.home_tmp
  47 +
  48 + self.repo_dir = tempfile.mkdtemp(prefix="deploy-stage-repo-")
  49 + self.addCleanup(shutil.rmtree, self.repo_dir, ignore_errors=True)
  50 + _init_git_repo(self.repo_dir)
  51 + with open(os.path.join(self.repo_dir, "README.md"), "w", encoding="utf-8") as f:
  52 + f.write("v1\n")
  53 + self.commit_v1 = _commit_all(self.repo_dir, "v1")
  54 + with open(os.path.join(self.repo_dir, "README.md"), "w", encoding="utf-8") as f:
  55 + f.write("v2\n")
  56 + self.commit_v2 = _commit_all(self.repo_dir, "v2")
  57 +
  58 + self.conn = schema.connect(":memory:")
  59 + schema.create_schema(self.conn)
  60 +
  61 + self.prod_db_path = os.path.join(self.home_tmp, "prod.db")
  62 + prod_conn = schema.connect(self.prod_db_path)
  63 + schema.create_schema(prod_conn)
  64 + prod_conn.close()
  65 +
  66 + def tearDown(self):
  67 + if self.old_home is None:
  68 + os.environ.pop("GRAPHLEMON_HOME", None)
  69 + else:
  70 + os.environ["GRAPHLEMON_HOME"] = self.old_home
  71 + self.conn.close()
  72 +
  73 +
  74 +class BootstrapPreviousTest(DeployStageTestBase):
  75 + def test_no_prior_finalized_deployment_bootstraps_from_current_repo_state(self):
  76 + # setUp() leaves repo_dir's HEAD at commit_v2 (the last commit made);
  77 + # bootstrap must read the repo's *current* HEAD, not the commit being
  78 + # staged (git worktree add --detach doesn't move repo_dir's own HEAD).
  79 + with patch.object(deploy_stage, "_launch_candidate") as mock_launch:
  80 + mock_launch.return_value.pid = 12345
  81 + result = deploy_stage.stage_deployment(
  82 + self.conn, self.commit_v2, repo_root=self.repo_dir, prod_db_path=self.prod_db_path, port=9999,
  83 + )
  84 + row = self.conn.execute(
  85 + "SELECT previous_commit_hash, previous_instance_dir FROM deployment WHERE deployment_id = ?",
  86 + (result["deployment_id"],),
  87 + ).fetchone()
  88 + self.assertEqual(row[0], self.commit_v2)
  89 + self.assertEqual(row[1], self.repo_dir)
  90 +
  91 + def test_prior_finalized_deployment_supplies_previous_fields(self):
  92 + self.conn.execute(
  93 + "INSERT INTO deployment (deployment_id, git_commit_hash, previous_commit_hash, status, "
  94 + "candidate_instance_dir, previous_instance_dir) "
  95 + "VALUES ('dep-0', 'commit-0', 'commit-minus-1', 'finalized', '/old/candidate/dir', '/older/dir')"
  96 + )
  97 + self.conn.commit()
  98 + with patch.object(deploy_stage, "_launch_candidate") as mock_launch:
  99 + mock_launch.return_value.pid = 12345
  100 + result = deploy_stage.stage_deployment(
  101 + self.conn, self.commit_v2, repo_root=self.repo_dir, prod_db_path=self.prod_db_path, port=9999,
  102 + )
  103 + row = self.conn.execute(
  104 + "SELECT previous_commit_hash, previous_instance_dir FROM deployment WHERE deployment_id = ?",
  105 + (result["deployment_id"],),
  106 + ).fetchone()
  107 + self.assertEqual(row[0], "commit-0")
  108 + self.assertEqual(row[1], "/old/candidate/dir")
  109 +
  110 +
  111 +class StageDeploymentHappyPathTest(DeployStageTestBase):
  112 + def test_creates_candidate_starting_row_with_real_worktree_and_db_copy(self):
  113 + with patch.object(deploy_stage, "_launch_candidate") as mock_launch:
  114 + mock_launch.return_value.pid = 54321
  115 + result = deploy_stage.stage_deployment(
  116 + self.conn, self.commit_v2, repo_root=self.repo_dir, prod_db_path=self.prod_db_path, port=9999,
  117 + )
  118 +
  119 + self.assertEqual(result["status"], "candidate_starting")
  120 + self.assertEqual(result["candidate_port"], 9999)
  121 + self.assertEqual(result["candidate_pid"], 54321)
  122 +
  123 + row = self.conn.execute(
  124 + "SELECT status, candidate_port, candidate_db_path, candidate_instance_dir, git_commit_hash "
  125 + "FROM deployment WHERE deployment_id = ?",
  126 + (result["deployment_id"],),
  127 + ).fetchone()
  128 + self.assertEqual(row[0], "candidate_starting")
  129 + self.assertEqual(row[1], 9999)
  130 + self.assertEqual(row[4], self.commit_v2)
  131 +
  132 + # real git worktree: candidate dir exists and is checked out at v2
  133 + candidate_dir = row[3]
  134 + self.assertTrue(os.path.isdir(candidate_dir))
  135 + with open(os.path.join(candidate_dir, "README.md"), encoding="utf-8") as f:
  136 + self.assertEqual(f.read(), "v2\n")
  137 +
  138 + # real DB copy: a distinct file, not the same path as prod
  139 + candidate_db_path = row[2]
  140 + self.assertNotEqual(candidate_db_path, self.prod_db_path)
  141 + self.assertTrue(os.path.isfile(candidate_db_path))
  142 +
  143 + def test_only_one_non_terminal_deployment_allowed(self):
  144 + with patch.object(deploy_stage, "_launch_candidate") as mock_launch:
  145 + mock_launch.return_value.pid = 1
  146 + deploy_stage.stage_deployment(
  147 + self.conn, self.commit_v2, repo_root=self.repo_dir, prod_db_path=self.prod_db_path, port=9001,
  148 + )
  149 + with self.assertRaises(Exception): # schema's ux_deployment_active partial unique index
  150 + deploy_stage.stage_deployment(
  151 + self.conn, self.commit_v1, repo_root=self.repo_dir, prod_db_path=self.prod_db_path, port=9002,
  152 + )
  153 +
  154 +
  155 +class CheckoutFailedTest(DeployStageTestBase):
  156 + def test_nonexistent_commit_marks_blocked_pending_human_and_raises(self):
  157 + with self.assertRaises(deploy_stage.CheckoutFailedError):
  158 + deploy_stage.stage_deployment(
  159 + self.conn, "0" * 40, repo_root=self.repo_dir, prod_db_path=self.prod_db_path, port=9003,
  160 + )
  161 + row = self.conn.execute("SELECT status FROM deployment").fetchone()
  162 + self.assertEqual(row[0], "blocked_pending_human")
  163 +
  164 + def test_checkout_failure_leaves_no_candidate_process_launched(self):
  165 + with patch.object(deploy_stage, "_launch_candidate") as mock_launch:
  166 + with self.assertRaises(deploy_stage.CheckoutFailedError):
  167 + deploy_stage.stage_deployment(
  168 + self.conn, "0" * 40, repo_root=self.repo_dir, prod_db_path=self.prod_db_path, port=9004,
  169 + )
  170 + mock_launch.assert_not_called()
  171 +
  172 +
  173 +class RealCandidateLaunchTest(unittest.TestCase):
  174 + """Not mocked -- stages a real candidate checkout of graphlemon's OWN
  175 + repo (read-only `git worktree add --detach` at the current HEAD, cleaned
  176 + up in tearDown) and launches a real `python3 -m src.backend_server`
  177 + subprocess against it, then hits it over real HTTP, per this Unit's own
  178 + "拉起监听临时端口...的候选进程" Done criterion -- proves backend_server's
  179 + new CLI entry point (added by this Unit) actually works end to end, not
  180 + just that stage_deployment() calls the right mocked function."""
  181 +
  182 + def setUp(self):
  183 + self.home_tmp = tempfile.mkdtemp(prefix="deploy-stage-real-home-")
  184 + self.addCleanup(shutil.rmtree, self.home_tmp, ignore_errors=True)
  185 + self.old_home = os.environ.get("GRAPHLEMON_HOME")
  186 + os.environ["GRAPHLEMON_HOME"] = self.home_tmp
  187 +
  188 + self.conn = schema.connect(":memory:")
  189 + schema.create_schema(self.conn)
  190 + self.addCleanup(self.conn.close)
  191 +
  192 + self.prod_db_path = os.path.join(self.home_tmp, "prod.db")
  193 + prod_conn = schema.connect(self.prod_db_path)
  194 + schema.create_schema(prod_conn)
  195 + prod_conn.close()
  196 +
  197 + def tearDown(self):
  198 + if self.old_home is None:
  199 + os.environ.pop("GRAPHLEMON_HOME", None)
  200 + else:
  201 + os.environ["GRAPHLEMON_HOME"] = self.old_home
  202 +
  203 + @staticmethod
  204 + def _commit_to_test_against():
  205 + """`git worktree add` only ever sees COMMITTED content -- if this
  206 + test runs before this Unit's own changes are committed (the normal
  207 + implement-then-test-then-commit order), plain HEAD would check out
  208 + a backend_server.py without this Unit's new main() entry point at
  209 + all. `git stash create` builds a real commit object representing
  210 + the current working tree (including uncommitted tracked-file
  211 + changes) WITHOUT touching the working tree, index, or HEAD -- exactly
  212 + what's needed here. Once this Unit is actually committed the working
  213 + tree is clean and `git stash create` prints nothing, so this falls
  214 + back to plain HEAD in that case (post-commit reruns of this test)."""
  215 + stash_sha = subprocess.run(
  216 + ["git", "stash", "create"], cwd=paths.repo_root(), capture_output=True, text=True, check=True
  217 + ).stdout.strip()
  218 + return stash_sha or deploy_stage._current_commit(paths.repo_root())
  219 +
  220 + def test_real_candidate_process_actually_serves_http(self):
  221 + current_commit = self._commit_to_test_against()
  222 + result = deploy_stage.stage_deployment(
  223 + self.conn, current_commit, repo_root=paths.repo_root(), prod_db_path=self.prod_db_path,
  224 + )
  225 + pid = result["candidate_pid"]
  226 + port = result["candidate_port"]
  227 + candidate_dir = result["candidate_instance_dir"]
  228 + try:
  229 + _UNSET = object()
  230 + deadline = time.time() + 10
  231 + last_error = None
  232 + status = _UNSET
  233 + while time.time() < deadline:
  234 + try:
  235 + with urllib.request.urlopen(
  236 + "http://127.0.0.1:{}/api/settings/concurrency".format(port), timeout=1
  237 + ) as resp:
  238 + status = resp.status
  239 + break
  240 + except urllib.error.URLError as e: # server not up yet (connection refused)
  241 + last_error = e
  242 + time.sleep(0.2)
  243 + self.assertIsNot(status, _UNSET, "candidate server never responded: {}".format(last_error))
  244 + self.assertEqual(status, 200) # real route, real response, real HTTP round trip
  245 + finally:
  246 + os.kill(pid, 15) # SIGTERM
  247 + deadline = time.time() + 5
  248 + while time.time() < deadline and _pid_alive(pid):
  249 + time.sleep(0.1)
  250 + if _pid_alive(pid):
  251 + os.kill(pid, 9)
  252 + subprocess.run(
  253 + ["git", "worktree", "remove", "--force", candidate_dir], cwd=paths.repo_root(),
  254 + capture_output=True, text=True,
  255 + )
  256 +
  257 +
  258 +def _pid_alive(pid):
  259 + try:
  260 + os.kill(pid, 0)
  261 + except OSError:
  262 + return False
  263 + return True
  264 +
  265 +
  266 +if __name__ == "__main__":
  267 + unittest.main()
... ...