---

name: code-agent-engineering description: Execute and evaluate repository-level Code Agent work using atomic task acquisition, bounded repo exploration, TDD, systematic debugging, minimal semantic patches, risk-driven testing, independent review, recoverable state machines, and evidence-based delivery. version: 1.0.0 metadata: hermes:

tags: [code-agent, software-engineering, tdd, debugging, testing, code-review, state-machine, evaluation]

Code Agent Engineering

Use this skill for autonomous or semi-autonomous repository changes where the result must be executable, tested, reviewable, and safely recoverable. It governs the engineering loop; project-specific task systems and repo instructions remain authoritative.

For the detailed rationale, metrics, benchmark caveats, and research sources, read references/software-engineering-loop-and-evaluation.md.

For project-specific task-state routing in the graphagent repo, read references/graphagent-base-state-machine.md.

Core Contract

• Process exactly one task per run unless the user explicitly requests otherwise.
• Treat the task system as the state source of truth and repository instructions as the implementation source of truth.
• Never modify production code before task scope, workspace, baseline, and a valid reproduction/test oracle are understood.
• A task is not complete until implementation has been exercised, evidence captured, and independent review passed.
• Do not overwrite or roll back user changes; only revert changes whose provenance belongs to the current run.

Required State Machine

ELIGIBLE → CLAIMED → CONTEXT_READY → BASELINE_VERIFIED
→ RED_VERIFIED → IMPLEMENTING → NARROW_GREEN
→ REGRESSION_GREEN → SPEC_REVIEW → QUALITY_REVIEW → READY_FOR_TEST

Each transition requires machine evidence. Invalid or missing evidence fails closed.

Recovery States

• Requirement facts missing or contradictory → BLOCKED_REQUIREMENT.
• Environment or baseline not reproducible → BLOCKED_ENVIRONMENT.
• Three independent repair hypotheses disproved → REPLAN_REQUIRED.
• Review rejected → return to IMPLEMENTING, then rerun affected gates.
• Transient tool failure → retry with bounded backoff and idempotency.
• Crash/restart → verify task version, workspace hash, diff, checkpoint, and lease before resuming.

Procedure

1. Acquire Atomically

• Select one eligible, approved task.
• Claim with a conditional update on task id, status, approval, and ownership.
• Use a run_id idempotency key and a lease for long-running work.
• Re-read the record; continue only if owner, status, and version match.
• Preserve parent/child and Bug/requirement boundaries; do not duplicate work across records.

2. Build the Task Contract

Extract:

• objective and observable acceptance criteria;
• in-scope and explicitly out-of-scope behavior;
• inputs, outputs, side effects, errors, compatibility, security, and concurrency expectations;
• dependencies and required tests;
• known facts, design decisions, hypotheses, and blocking unknowns.

Do not guess blocking business rules. Return or block an underspecified task.

3. Explore the Repository as a Funnel

1. Confirm absolute path, branch/commit, existing diff, and repo instructions.
2. Read README, manifests, build scripts, CI, and test configuration.
3. Search issue terms, symbols, errors, and configuration names with bounded output.
4. Trace definition → callers → data/side effects → analogous working path.
5. Locate existing tests, fixtures, assertion style, and integration boundaries.
6. Produce a compact repo map: relevant files, symbols, call path, commands, and risks.

Avoid whole-repository context dumps. Narrow noisy searches and compress stale observations.

4. Verify the Baseline

• Establish the documented environment and record its fingerprint.
• Run the narrow relevant baseline plus a smoke check when warranted.
• Separate pre-existing failures from task-caused failures.
• If the environment cannot reproduce the baseline, stop as blocked rather than attributing failures to future edits.

5. Establish a Trustworthy RED

A regression/reproduction test is valid only when it:

• fails before the patch for the intended behavioral reason;
• passes after the patch;
• fails again if the production patch is reverted;
• does not copy production logic or mock away the real boundary.

Generated reproduction tests are candidate oracles until validated this way. For pure refactors, use characterization tests that are GREEN before the change; never fabricate RED for ceremony.

6. Debug Systematically

Maintain a hypothesis ledger: evidence → hypothesis → predicted observation → smallest experiment → conclusion.

• Reproduce precisely and preserve full errors.
• Check recent diffs, configuration, and environment differences.
• Inspect component boundaries and trace bad data upstream.
• Compare against a working repository pattern.
• Test one hypothesis at a time and revert disproved experiments.
• Fix the root cause after locking it with a regression test.
• After three independent failed hypotheses, stop patching and revisit architecture or requirements.

7. Make the Minimal Semantic Patch

Before editing, declare an expected change budget: files, symbols, public contracts, data, dependencies, and risk level.

• Avoid unrelated formatting, renaming, dependency upgrades, or cleanup.
• Recheck the diff after every meaningful edit.
• Escalate risk automatically for public API, schema, permission, lockfile, migration, concurrency, or external-side-effect changes.
• “Minimal” means the smallest complete semantic fix, not the fewest characters; do not leave half-fixes at interface boundaries.

8. Select Tests by Impact Cone

• T0: syntax, types, lint, targeted static/security checks — every patch.
• T1: RED test and touched unit tests — every patch.
• T2: module/package, callers, and adjacent contracts — cross-function/module changes.
• T3: repository regression, build, integration/E2E — public/shared behavior changes or when affordable before delivery.
• T4: security, migration, compatibility, concurrency, performance, fault injection — when their risk triggers are present.

If a layer cannot run, record the omitted command, reason, substitute evidence, and downstream CI gate. Never silently omit it.

9. Require Independent Two-Stage Review

1. Specification review: map every acceptance criterion to code/test evidence; reject omissions and scope creep.
2. Quality review: inspect correctness, edge/error paths, security, concurrency, compatibility, maintainability, and test adequacy.

The reviewer reads the original task, final diff, and raw test logs. The implementer’s narrative is supporting evidence only. Return structured pass | fail | uncertain, severity, location, and reproduction command. Treat high-risk uncertainty as failure. The implementer cannot be the sole reviewer.

10. Deliver Evidence and Transition

The evidence bundle must contain:

• task id, run id, baseline commit, and environment fingerprint;
• changed files and diff summary;
• baseline, RED, GREEN, and regression commands with exit codes, test counts, and relevant output;
• specification and quality review verdicts;
• omitted tests, residual risks, and follow-up gates;
• final task-state re-read.

Only then transition to the project’s equivalent of “ready for test.”

Event Log and Idempotency

Persist append-only action/observation events with run id, sequence, state before/after, action, input hash, result, artifacts, and timestamp. Create checkpoints at every gate with task version, base hash, diff, environment fingerprint, and test logs.

Reads may be replayed. Writes require an idempotency key or check-before-write. On recovery, workspace/checkpoint mismatch requires review rather than blind continuation.

Evaluation

Do not equate a public benchmark score with production capability. Evaluate with four layers:

1. public comparable benchmark;
2. fresh temporal holdout with hidden tests and post-training-cutoff tasks;
3. decontaminated project regression set;
4. production shadow/canary without automatic merge.

Hold model/scaffold experiments separately: same model across harnesses and same harness across models. Fix model version, scaffold commit, container, seed, budget, visible information, and network policy. Report pass@1, confidence intervals, cost, trajectory, test-oracle audit, and semantic patch review.

Core Metrics

• Verified Delivery Rate;
• Accepted Without Rework;
• Escaped Defect, Reopen, Revert, and Change Failure Rate;
• Requirement Coverage;
• cost per accepted task;
• RED validity and baseline reproducibility;
• localization precision and patch scope expansion;
• T0–T4 execution coverage;
• specification/quality review first-pass rate;
• crash recovery success, illegal transitions, and duplicate side effects.

Pair efficiency metrics with defects, rework, and requirement coverage to prevent Goodhart effects.

Pitfalls

• Claiming multiple tasks in one run.
• Editing before checking repo instructions, existing diffs, or baseline.
• Treating import/fixture failures as a valid RED.
• Accepting an LLM-generated test as ground truth without pre/post/revert validation.
• Repeated symptom patches without a root-cause hypothesis.
• Measuring minimality only by LOC.
• Running only the narrow test or only the full suite without impact analysis.
• Letting the implementer self-approve.
• Transitioning task status on planned, fabricated, partial, or stale evidence.
• Comparing benchmark scores across different scaffolds, budgets, environments, or data visibility as if they measured only the model.