name: project-artifact-content-quality description: code_diff/test_report artifacts in this project's self-hosting runs sometimes contain placeholder/wrong text instead of a real diff or report — don't trust artifact content, verify the actual working tree and run tests yourself metadata:
type: project
Observed on task 7400d876-a797-4d87-bcff-4e59a3befba1 (v_graph_console
/ #/graph, 2026-08-03): the code_diff artifact
(.graphlemon/runs/<task_id>/code_diff.patch) contained only the literal
text (no diff -- see test_report), and the test_report artifact
(.graphlemon/runs/<task_id>/test_report.md) contained an unrelated
dispatch/notification message ("Launched the graphlemon Work Node (coder)
workflow in the background... I'll notify you when it completes") rather
than an actual summary of tests run and their results. The real code
changes existed and were correct — verified independently via git diff
on the actual target repo, running python3 -m unittest discover -s src
(513 passed), running the Coder-left node
frontend/test_graph_console_smoke.js (passed), and hitting the live
backend with curl — but the artifact files meant to document that work
did not reflect it.
Why: this matters because [[feedback-no-js-test-framework]] and the general validator mandate ("developer_claim only, must be independently re-verified") both assume you'll actually re-run things — this is a concrete instance where trusting the artifact content alone would have been actively misleading (it looked like nothing was tested), even though the underlying implementation was solid.
How to apply: always independently git diff/re-run tests against the
real target repo rather than reading code_diff.patch/test_report.md at
face value — treat a garbage-looking artifact as a prompt to dig into the
actual working tree/db state (as done here), not as automatic grounds for
FAIL, since the artifact-generation step and the actual code-change step
can apparently diverge in this pipeline. If this recurs, it may be worth
raising as a Coder/skill_defect root-cause candidate on some future task
even when the code itself passes.