• check_canary_task() creates a real project + task through the candidate's
    own HTTP API to verify the write path works, but only ever cleaned up its
    throwaway git directory -- the DB rows it created via the API were never
    removed. Every real health check permanently grew whatever DB the candidate
    was using by one dead project (pointing at an already-deleted temp dir) and
    one task stuck at 'queued' forever.
    
    Found by inspecting the real ~/.graphlemon/graphlemon.db, which had 5 such
    leaked 'deploy-canary' projects from earlier real health-check runs (now
    archived by hand). Fix: archive the created project in a finally block that
    covers every return path, including task-creation/verification failures
    where only the project (not the task) got created. Best-effort -- the
    archive call failing must not override the real check result.
    
    Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
    bboonnee authored
     
    Browse File »

  • graphlemon authored
     
    Browse File »
  • Attempts 7 and 11 both failed at exactly the same spot: process_alive read
    'ok' within milliseconds of Popen, then http_health hit ECONNREFUSED in the
    same second and gave up. Real cause: process_alive only confirms the
    process EXISTS, not that backend_server.py has finished importing modules
    and actually bound its listen socket -- a real, non-zero startup gap. The
    old check_http_health did one urlopen(timeout=X) call; that timeout only
    bounds how long to wait for a response once connected, not how long to
    wait for the port to start accepting connections at all -- ECONNREFUSED
    fires essentially instantly when nothing's listening yet, regardless of
    timeout.
    
    This touches an explicit 03-C design statement ("不重试——重试属于给坏版本
    第二次机会"), so confirmed with the user before changing it rather than
    just reinterpreting the policy unilaterally. Resolution: tolerating normal
    startup latency within ONE health-check pass isn't "retrying a failed
    check" -- check_process_alive already does exactly this (polls up to 30s
    for the pid to exist). http_health now does the same, polling within its
    own timeout for connection-refused-class errors specifically. A genuine
    non-2xx HTTP response is still a definitive answer from a live server and
    still fails immediately, no polling -- that boundary is unchanged.
    
    Two new real regression tests: a real server that starts listening 0.3s
    late is correctly waited for and passes; a real 500 response still fails
    immediately without being mistaken for "not up yet".
    
    564 tests green (441 src + 123 supervisor).
    
    Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
    bboonnee authored
     
    Browse File »
  • graphlemon authored
     
    Browse File »

  • Adds supervisor/deploy_cutover.py: stops the old Live process, retires the
    health-check-mode candidate, and relaunches the same already-verified
    candidate checkout bound to LIVE_PORT/PROD_DB_PATH -- http.server has no
    live socket-rebind operation, so "switch to production port/db" is
    implemented as stop-then-relaunch-from-the-same-dir rather than an in-place
    rebind (functionally equivalent, the only mechanism stdlib actually offers).
    
    Adds LIVE_PORT=8420 to 05 §5.9's MVP-fixed parameter table -- the one
    parameter that table was missing (frontend/app.js never needed one, same-
    origin relative fetches don't care, but successive deployments need to keep
    landing on the same port for a human bookmark to keep working).
    
    "在途 work_node_execution 重启后可恢复" is satisfied by relying on the
    existing REQ-06 recovery machinery unchanged -- cutover doesn't add any
    special-case logic, a cutover-triggered restart is just another interruption
    REQ-06 already survives.
    
    A real (not mocked) test caught a real bug: deploy_supervisor.is_alive()'s
    os.kill(pid, 0) check can't distinguish "truly running" from "exited but
    never reaped by its own parent" -- a zombie's PID slot answers kill(pid, 0)
    successfully until waited on. U-036's own subprocess test never caught this
    because that skeleton deletes its own PID file before exiting, sidestepping
    is_alive() entirely; deploy_cutover's test is the first one to actually poll
    for a stopped process to disappear, and it hung until fixed. Added
    reap_if_exited() (real os.waitpid for processes we spawned ourselves, falls
    back to is_alive() for pids from a prior session where ChildProcessError
    means we're not the real parent) and switched deploy_cutover._stop_process,
    deploy_health_check.check_process_alive, and deploy_supervisor.stop() to use
    it.
    
    Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
    bboonnee authored
     
    Browse File »
  • Adds supervisor/deploy_health_check.py: run_health_check() runs the three
    pre-cutover checks in order, stopping at the first failure (03-C: "任一检查项
    fail 立即整体判定 health_check_failed,不等待其余检查项跑完") and persisting
    a deploy_health_signal row for every check that actually ran.
    
    - process_alive: polls deploy_supervisor.is_alive() up to HEALTH_CHECK_STARTUP_TIMEOUT.
    - http_health: hits the candidate's /api/health -- added to backend_server.py
      (U-037 deliberately left this for U-038). Not just "process exists" --
      runs a real SELECT 1 to catch a wedged sqlite connection a live-but-stuck
      process would otherwise pass.
    - canary_task: a real (not mocked) project+task creation round-trip against
      the candidate over HTTP -- no real Org Graph/claude calls, since
      c_task_create's synchronous 'queued' landing is already a deterministic,
      always-should-pass result, which is what a canary needs.
    
    Timeouts are the MVP-fixed drafts from 05 §5.9, explicitly flagged there as
    needing real calibration later, not hardcoded as authoritative.
    
    Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
    bboonnee authored
     
    Browse File »