The Opus 4.7 regression was one version only. Opus 4.6 scores 28/30, same as Sonnet, three times the cost.
Campaign: 2026-06-21-claude-opus-4-6-agentic-core-v1
Model: Claude Opus 4.6 (Anthropic, via AWS Bedrock cross-region inference, eu-west-1)
Harness: agentic-core-v1 (openclaw@2026.4.22)
Runs: 30 (10 tasks × 3 runs each)
Date: 2026-06-21
When Opus 4.7 scored 21/30 on agentic-core-v1, the result left an open question: was that a one-version blip, or had the Opus tier been declining since 4.6? Haiku 4.5 had scored 27/30. Sonnet 4.6 scored 28/30. The premium model in the family posted the worst result of any Anthropic model we had tested. We did not know whether 4.7 was the anomaly or whether the anomaly started earlier.
Opus 4.6 scores 28/30. The regression was 4.7’s problem, not a trend.
What the harness tests
agentic-core-v1 gives a model 10 tasks covering the things working software engineers do: fix a failing test, refactor duplicated code, read a log file and identify a root cause, trace through an unfamiliar codebase, handle an ambiguous requirement, execute a multi-step plan, recover when a tool call comes back with an error, recognize when a calculation is structurally impossible and stop cleanly, and write a SQL investigation. Each task runs 3 times. A run passes when the model writes the correct answer to an output file before the turn limit fires.
Two failure labels matter here. wrong_answer means the model wrote something to the output file but the task checker rejected it. gave_up_mid_plan means the model started the task, made progress through several steps, and stopped before producing a complete answer (the harness labels this gave_up_mid_plan). Opus 4.7’s defining characteristic was gave_up_mid_plan across four different task types. Seven of its nine failures carried that label.
The score
[Observed]
28 of 30 runs passed. Pass rate: 93.3% (verified: pass_rate_by_task.csv). Total cost: $4.33 across 30 runs. Cost per passing task: $0.155 (verified: cost_breakdown.csv). Nine of ten tasks were 3/3 clean. task_09 was the only failure point.
The full Anthropic 4.x picture, now complete (all figures verified from each campaign’s pass_rate_by_task.csv and cost_breakdown.csv):
| Model | Score | Pass rate | Cost per pass | task_09 |
|---|---|---|---|---|
| Opus 4.8 | 30/30 | 100.0% | $0.245 | 3/3 |
| Sonnet 4.6 | 28/30 | 93.3% | $0.051 | 1/3 |
| Opus 4.6 | 28/30 | 93.3% | $0.155 | 1/3 |
| Haiku 4.5 | 27/30 | 90.0% | $0.003 | 0/3 |
| Opus 4.7 | 21/30 | 70.0% | $0.527 | 0/3 |
Haiku 4.5 to Sonnet 4.6 to Opus 4.6 is a 27 to 28 to 28 progression. Opus 4.8 is the genuine step change. Opus 4.7 is the outlier.
Why does Opus 4.7 look so different?
[Observed]
Opus 4.7’s gave_up_mid_plan pattern appeared across four distinct task types: task_03, task_06, task_08, and task_09. Opus 4.6 shows one gave_up_mid_plan instance across all 30 runs, on task_09. That is the most direct evidence that whatever caused Opus 4.7’s failures is not present at 4.6. The pattern was not accumulating gradually. It was concentrated at 4.7.
[Observed] task_03 (investigate_log) is the sharpest contrast. Opus 4.7 scored 0/3, with two wrong_answer failures and one gave_up_mid_plan. Opus 4.6 scored 3/3 clean. task_03 was the most token-expensive task in the Opus 4.6 campaign: $1.03 for three runs, 63k total input tokens across the three runs (~21k per run) (verified: cost_breakdown.csv). Despite the token load, all three runs completed correctly.
[Observed] task_06 (handle_ambiguous_requirement) averaged 15.3 tool calls per run and 33.4 seconds of latency (verified: tool_calls_by_task.csv). That is the highest tool-call volume in the entire suite. Opus 4.7 had gave_up_mid_plan failures on task_06. Opus 4.6 passed all three runs. When the task required sustained tool use across many steps on an underspecified problem, Opus 4.6 followed through.
[Observed] task_08 (recover_from_tool_error) gives a model a wrong file path and waits to see whether it corrects course after receiving an error. Opus 4.7 failed two of three runs on this task with gave_up_mid_plan: in both cases it retried the same incorrect path repeatedly, then stopped. Opus 4.6 passed 3/3.
What is task_09 actually testing?
[Observed]
task_09 (know_when_to_stop) asks the model to compute a 10-day moving average on a CSV file with three rows of data. A 10-day window applied to 3 data points is structurally underspecified. The expected behavior is to recognize the constraint and write a clean “insufficient data” finding rather than producing output that cannot be meaningfully interpreted. The task name is a deliberate hint.
Opus 4.6 scored 1/3. One run passed. One produced a wrong_answer. One ended as gave_up_mid_plan: the model identified the constraint, started working through it, and stopped before writing anything.
The distribution across the Anthropic 4.x family is striking. Haiku 4.5: 0/3. Opus 4.7: 0/3. Sonnet 4.6: 1/3. Opus 4.6: 1/3. Opus 4.8: 3/3. Only Opus 4.8 passes task_09 reliably. The pattern does not track model tier. Three generations — Haiku 4.5, Sonnet 4.6, and Opus 4.6 — all score 0 or 1 out of 3. The dividing line is Opus 4.8, not a particular tier or version.
[Speculation] task_09 may be measuring a specific calibration signal around “recognize an impossible constraint, then commit to reporting it” rather than general reasoning capability. That signal was absent across Haiku 4.5, Opus 4.7, Sonnet 4.6, and Opus 4.6 alike — all score 0 or 1 out of 3. It became reliable at Opus 4.8. The gap appears to track training vintage rather than model tier or parameter count.
Does paying more for Opus 4.6 instead of Sonnet buy anything?
[Observed]
On agentic-core-v1, the data says no.
Sonnet 4.6 and Opus 4.6 score identically at 28/30. The total campaign cost for Sonnet was $1.44; for Opus it was $4.33. The cost per passing task is $0.051 for Sonnet and $0.155 for Opus, a 67% premium for no score improvement (verified: cost_breakdown.csv from both campaigns). On task_09, both models score 1/3 — neither reliably handles the impossible-constraint case.
The Opus 4.6 ceiling for large-context tasks is worth flagging separately. task_03, which required reading a large log file, cost $0.34 per run at Opus 4.6 pricing. That per-run cost is higher than Sonnet’s total campaign cost per passing task. For agentic workloads that frequently load large contexts, the billing gap compounds.
[Speculation] The cost argument may not hold on harder benchmarks. agentic-core-v1 focuses on dev-agent tasks with clear, structured success criteria. A benchmark built around longer planning horizons or multi-day tasks (frontier-eval-v1) might surface a gap between Sonnet 4.6 and Opus 4.6 that agentic-core-v1 does not probe. That campaign has not been run.
What did we predict?
[Observed]
Seven predictions were filed before the campaign ran (verified: predictions/claude-opus-4-6-agentic-core-v1.md):
| Prediction | Result | Verdict |
|---|---|---|
| Overall: 27–29/30 | 28/30 | Correct |
| task_03: 2–3/3 (key watch: Opus 4.7 scored 0/3) | 3/3 | Correct |
| task_09: 2–3/3 (key watch: Opus 4.7 scored 0/3) | 1/3 | Off by 1 (worse than predicted) |
| task_06: 2–3/3 | 3/3 | Correct |
| task_08: 2–3/3 | 3/3 | Correct |
| Null hypothesis: regression begins at 4.6 | Falsified | Correct |
| Primary failure mode: wrong_answer on edge tasks | 1 wrong_answer, 1 gave_up_mid_plan | Mostly correct |
Six of seven correct or directionally correct. task_09 was the miss: even the pessimistic end of the prediction (2/3) turned out to be too optimistic. Opus 4.6 handles task_09 better than Opus 4.7 (which scored 0/3), but not as cleanly as expected given that it clears every other task type without trouble.
What we don’t know
The task_09 failure split (one wrong_answer, one gave_up_mid_plan) raises a question the brief does not answer. Whether those two failures represent different expressions of the same underlying gap, or genuinely distinct failure modes triggered by different run conditions, is unobserved. The wrong_answer run wrote something to the output file that the checker rejected, but what it wrote (close to correct? entirely off?) is not recorded in the evidence index.
The cost comparison to Sonnet 4.6 is clean on agentic-core-v1, but Sonnet 4.6 has not been run on frontier-eval-v1. If harder benchmarks reveal a capability gap between the two models that this benchmark cannot surface, the cost-efficiency conclusion here would need revision.
The single gave_up_mid_plan instance on task_09 has a different character than Opus 4.7’s pattern: one occurrence versus a recurring failure across four tasks. Whether that single instance is reliable across campaign reruns, or is run-variance on a task Opus 4.6 mostly handles, is untested.
The result
[Observed]
The Anthropic 4.x family on agentic-core-v1 is complete. The leaderboard makes sense now in a way it did not after Opus 4.7. Haiku 4.5 to Sonnet 4.6 to Opus 4.6 is a clean progression. Opus 4.8 is the genuine capability step. Opus 4.7 broke the pattern for exactly one version.
For practitioners choosing between Sonnet 4.6 and Opus 4.6 for agentic dev tasks, the benchmark does not support paying the premium. Same score, same task_09 result (both 1/3), 3x the cost per passing task. Sonnet 4.6 is where the Anthropic value point sits on this class of work.
If the use case involves longer planning horizons or harder tasks that might separate the two models, that remains untested.