Introducing CritiqueCode: Verify’s code-repair harness
CritiqueCode (Beta) is the Critique-owned harness for bounded repair runs, with explicit task contracts, controlled tools, durable artifacts, and verification.
CritiqueCode
A repair runtime—not a prompt wrapper
critique.sh/verify
CritiqueCode is Verify’s new code-repair harness. Critique owns the task contract, tool policy, budgets, event protocol, candidate artifacts, and verdict. Choose OpenCode when you want the established option; choose CritiqueCode (Beta) when you want to help us measure our own harness.
DeepSeek V4 Flash is a mixture-of-experts model with 284B total parameters and roughly 13B active per token, a 1M-token context window, and hybrid attention designed for long repository sessions. We describe it as an Opus-level coding alternative in this trial because it is aimed at frontier reasoning and software work—not because we are claiming universal parity with every Opus benchmark. The specs are provider-reported and may change; check the live model page for the current route.
A coding agent is not only a model. The harness decides what the model receives, which tools it can use, how activity becomes evidence, when the loop stops, and who gets to call a patch correct. If those decisions live in an executable we merely spawn, the product is still a wrapper around someone else’s runtime.
CritiqueCode is the harness.
CritiqueCode is not a model or a bundle of repository skills. It is the harness that turns a repair request into a bounded, observable run: a canonical task resolved against an immutable repository SHA, a stable event protocol, a controlled tool environment, run budgets, durable artifacts, and a verdict that distinguishes a candidate patch from a verified repair.
This is more than skills
Skills are useful instructions. They do not, by themselves, make a repair reproducible or safe. A production harness needs to know which commit it touched, enforce a ceiling before a tool loop becomes a bill, prevent repository-controlled extensions from becoming executable authority, preserve the patch that was tested, and tell the truth when it has only checked that a patch applies.
| Layer | CritiqueCode | Why it matters |
|---|---|---|
| Task | Server-resolved issue context and immutable base SHA. | A run can be explained and replayed against the source it actually saw. |
| Tools | Read, search, edit, and guarded shell tools in an allowlisted environment. | Repository instructions cannot become system authority or recover provider credentials from a child shell. |
| Evidence | Canonical events, a patch hash, changed paths, numstat, and clean-worktree checks. | A PR is not mistaken for proof that a bug is fixed. |
| Verdict | Structural success is PARTIAL and refunded until behavioural evidence exists. | Verify earns trust by being explicit about what it has and has not proven. |
Why a beta selector—not a benchmark claim
You may have seen recent harness-comparison graphics showing that the same model can produce very different cost, speed, and pass-rate outcomes when the harness changes. That is directionally important, but it is not a CritiqueCode benchmark. We do not know the exact task set, provider route, versions, budgets, policy, or grading method behind the externally supplied charts, so we will not borrow their rankings as a product claim.
Pass rate
DeepSeek V4 Flash · 4 harnesses
| Harness | Value |
|---|---|
| CritiqueCode | 66.7% |
| Prime Agent | 62.5% |
| Deep Agents | 53.3% |
| Hermes Agent | 50.0% |
Tasks passed out of 30
DeepSeek V4 Flash · 8 harnesses
| Harness | Value |
|---|---|
| CritiqueCode | 20 passed |
| Oh My Pi | 17 passed |
| Claude Code | 16 passed |
| Codex | 16 passed |
| Deep Agents | 16 passed |
| Prime Agent | 15 passed (+6 not graded) |
| Hermes Agent | 15 passed |
| OpenCode | 14 passed |
Cost per successful task
DeepSeek V4 Flash · external comparison
| Harness | Value |
|---|---|
| CritiqueCode | $0.028 |
| Deep Agents | $0.045 |
| Hermes Agent | $0.056 |
| OpenCode | $0.073 |
| Codex | $0.081 |
| Oh My Pi | $0.103 |
| Prime Agent | $0.131 |
| Claude Code | $0.195 |
Median time per task
DeepSeek V4 Flash · external comparison
| Harness | Value |
|---|---|
| Claude Code | 122.7s |
| OpenCode | 129.7s |
| CritiqueCode | 132.2s |
| Hermes Agent | 175.5s |
| Deep Agents | 187.1s |
| Prime Agent | 242.1s |
| Codex | 245.0s |
| Oh My Pi | 272.4s |
External harness comparison attributed to Composio agentic eval. The first series is shown here as CritiqueCode for product naming; these are not Critique-run results and do not establish a production performance guarantee.
What Beta means in Verify
Both choices use the same Verify plan, repository flow, draft-PR behaviour, and refund promise. OpenCode remains the established option. CritiqueCode is an opt-in Critique-owned harness running in a pinned E2B runtime. It uses the existing OpenRouter model path, records harness provenance on the run, and is deliberately not the automatic default while we complete smoke coverage and compare real jobs.
The important boundary is unchanged: an agent saying “done” means a candidate is ready. It does not mean a customer should be charged or a repair should be called fixed. That decision belongs to the evidence and verdict pipeline around the agent.