• _launch_candidate() spawns a real `python -m src.backend_server` process
    (`revived`) to re-verify the previous version during a post-cutover
    rollback. On the success path it's correctly left running -- it IS the new
    Live. But on every failure path (never bound live_port, failed
    re-verification, git revert conflict) nothing ever terminated it -- found
    via 31 accumulated orphaned processes on this machine, one per failed
    rollback_path_b call across many real test runs this session.
    
    This is also the likely cause of RollbackPathBFailureTest's own reported
    non-determinism (three independent real Validator sessions saw it fail,
    never reproduced manually): a leaked process from an earlier failed run
    can still be squatting on the port a later run's _find_free_port() picks,
    producing a false "alive" health-check reading.
    
    Fix: call deploy_cutover._stop_process(revived.pid) in both failure
    branches before raising, mirroring the pattern cutover() already uses for
    its own stop_timeout-bounded process teardown. Also fixes
    RollbackPathBHappyPathTest's own matching leak (it re-appended bad_live.pid
    a second time instead of tracking the actual revived process, a
    copy-paste bug, not real cleanup) by capturing the real pid via a thin
    _launch_candidate wrapper.
    
    31 pre-existing orphaned processes killed by hand; verified 5 repeated
    test runs after the fix leak zero new ones.
    
    Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
    bboonnee authored
     
    Browse File »

  • The 7th real self-hosting attempt was the first to actually reach a real
    health check against a real candidate process -- it correctly failed and
    triggered rollback_path_a for real: candidate terminated, git revert landed
    cleanly. But the candidate's `git worktree add --detach` checkout
    (candidate_instance_dir, tracked since U-037 specifically to know where
    this lives) was never removed -- left permanently on disk and still
    registered in `git worktree list`. A confirmed-bad candidate is never
    needed again after path A, unlike a successful cutover's candidate (which
    correctly becomes previous_instance_dir, kept until the monitoring window
    closes) -- these are different lifecycles that shouldn't share the same
    "leave it" behavior.
    
    rollback_path_a gains an optional candidate_instance_dir parameter
    (default None, existing tests unaffected) and removes the worktree before
    git revert when provided. self_hosting_e2e.py now threads through the
    value stage_deployment() already returns. New regression test uses a real
    `git worktree add --detach` checkout and confirms both the directory and
    its git worktree registration are gone after rollback.
    
    Also cleaned up 3 orphaned processes left by earlier test_deploy_rollback.py
    runs (backing temp dirs already gone) and the real leftover candidate
    worktree from this run.
    
    561 tests green (440 src + 121 supervisor).
    
    Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
    bboonnee authored
     
    Browse File »

  • Adds supervisor/deploy_rollback.py: rollback_path_a (pre-cutover, health
    check failed -- terminate candidate, delete its one-time DB copy, git
    revert, record a bug_record(core_engine) audit trail) and rollback_path_b
    (post-cutover, anomaly detected -- stop the bad Live, revive
    previous_instance_dir, re-run the same three health-check signals directly
    against it, git revert only once re-verification passes).
    
    Design choices follow established conventions rather than inventing new
    ones: no unified rollback() entry point (deployment has no PID columns,
    same reason deploy_stage/deploy_cutover already require callers to pass
    pids explicitly); ROLLBACK_FAILED is the raised error name while the
    persisted deployment.status is 'blocked_pending_human' (same pattern
    CheckoutFailedError/CutoverFailedError already use). git revert conflicts
    fold into the same rollback-itself-failed fallback both paths share, per
    design doc §10 open question 3's own recommendation, with `git revert
    --abort` run afterward so a failed attempt doesn't leave the repo mid-conflict.
    
    This is PH5's highest-rework-risk Unit per development-plan.md's own flag,
    so its own "rollback itself fails" fallback is tested for real, not assumed:
    a genuinely broken previous_instance_dir (a real empty directory with no
    Python package in it) that really fails to bind its port, and a genuinely
    conflicting git history (a later commit touching the same line the
    self-modification changed) that really fails `git revert` -- both correctly
    land on blocked_pending_human without retrying forever or silently
    succeeding, and the conflict case leaves the repo clean afterward.
    
    Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
    bboonnee authored
     
    Browse File »