critique.sh vs CI: Different Layers of Proof
CI runs the checks you have encoded. Critique challenges an agent’s completion claim before—or alongside—the path to a merge decision.
Before CI, not instead of it
A stronger finish handoff for agent-written code
critique.sh
Critique is not a CI replacement. CI is the repository’s executable source of truth for builds, tests, linters, scanners, packaging, and deployment gates. Critique addresses a prior and adjacent problem: a coding agent says a task is finished, but has anyone independently checked whether the change, its test plan, and that conclusion match the claim? Keep CI. Put Critique at the agent handoff where a green command is necessary but not always sufficient.
What CI actually proves
Modern CI is much more than “does it compile?” A GitHub Actions workflow can run unit, integration, functional, and end-to-end tests; lint and type checks; dependency and secret scans; coverage thresholds; deployment smoke tests; and any custom command a team encodes. When those jobs are reliable and protected, they are the repeatable control that turns a repository’s engineering standards into a merge requirement.
The word to notice is encoded. CI can only execute the assertions, environments, and commands the repository has made available. It will faithfully tell you that a selected test suite passed. It cannot, by itself, tell you that a coding agent misread a requirement, wrote a self-validating regression test, skipped an affected call site, or treated a missing environment dependency as a product success. Those are questions about the completion claim, not a criticism of CI.
Each layer owns a distinct question. Blurring them creates either slow agent loops or model verdicts pretending to be deterministic tests.
| Layer | Question | It should own |
|---|---|---|
| Coding agent | What should I implement next? | Implementation, local exploration, targeted tests, and a candidate handoff. |
| Critique | Does this “done” claim survive a fresh, bounded pass? | Reconstruction, consequential hypotheses, evidence, stated limits, and a scoped repair where justified. |
| CI | Did the repository-defined checks pass in this environment? | Repeatable builds, tests, scans, artifacts, branch protection, and deployment gates. |
| Human and policy | May we accept this risk? | Product intent, architecture, exceptions, ownership, and the final decision where policy requires it. |
Why an agent finish pass belongs before the merge ritual
The most expensive agent failures are often not test failures. They are wrong success conditions. An agent might add a test that mirrors its misunderstanding, or pass only the one command it knows how to run. If the error surfaces after CI, a reviewer has to reopen the task, reconstruct the context, and send the agent back into the same session with a vague instruction like “try again.” The flow becomes noisy precisely because the handoff had no clear contract.
A completion layer tightens that moment. The authoring agent has a candidate patch and an intent statement. Critique reconstructs the current working-tree change in a disposable workspace, tries to reproduce the consequential behavior, records evidence and limitations, and can return the smallest verifier-approved repair. The original agent receives structured JSON and continues. By the time the change reaches CI, it has been challenged as a claim—not merely sent through the commands the author selected.
Pre-CI does not mean “always before every CI job”
“Pre-CI” is a useful intent, not a rigid product claim. A local Critique finish pass can happen before the agent opens or updates a pull request. A team can also call Critique from a supervisor or CI orchestrator and use the verdict alongside existing checks. Different repositories will sequence it differently. The invariant is that Critique should not replace the deterministic work your pipeline already does, and CI should not be asked to infer product intent from a green status alone.
Three sensible ways to introduce Critique
Install the CLI and let Codex, Claude Code, or OpenCode call critique finish --json when a meaningful task is ready. The agent handles the response before it creates more review work for a person.
Run Critique on a real repository without blocking merges. Compare its evidence against review outcomes, false positives, and the issues developers actually care about.
Use a structured verdict in the broader merge workflow only after the signal is trusted. Preserve CI, approvals, and override provenance rather than installing a black-box gate.
What each layer catches best
- Compile failures, type errors, failing tests, coverage policies, and static rules.
- Known security and dependency checks with a repeatable scanner or policy.
- Release and deploy checks that must execute identically on every branch.
- Whether the agent’s task interpretation and changed paths deserve a fresh challenge.
- Whether a consequential scenario was actually exercised rather than narrated.
- Whether a confirmed issue has a minimal, bounded repair with proof attached.
The recommendation
Keep investing in CI. The more reliable your test suite and pipeline are, the more useful a finish pass becomes because it can point at real executable evidence rather than vague opinions. Add Critique where an agent declares work complete. That is the boundary CI does not naturally own: the moment between “the author says it is done” and “we are willing to treat it as ready.”
Strengthen the handoff without replacing CI.
Put an independent finish pass between an agent’s self-report and the merge path your team already trusts.
Read the CLI contract