Claude Fable 5.1 vs GPT-6 Astra: Benchmarks and Risk

Anthropic shipped Claude Fable 5.1 on September 1, 2026. OpenAI shipped GPT-6 Astra two days later. Both landed at exactly $10 per million input tokens and $50 per million output. Both run a roughly one million token context with 128K maximum output. Both take text and images in and return text.

That symmetry is where the easy comparison ends.

On the handful of benchmarks where both vendors published a number, the two models trade wins rather than separating. Astra takes Terminal-Bench 4.0 by 1.9 points and DeepSWE by 6.7. Fable 5.1 takes Humanity's Last Exam with tools by 7.8 points and leads both Artificial Analysis composite indices. None of those gaps is large enough to pick a model on.

The differences that will actually show up in your bill and in your incident reports sit somewhere else: in cache read pricing, in a long-context repricing cliff, and in two prompt injection scorecards that measure genuinely different things.

Link to section: Two flagship models, forty-eight hours apartTwo flagship models, forty-eight hours apart

Two entries in that table deserve more attention than the benchmark scores that follow. The cache read column differs by a factor of four, and only one of the two models reprices a request for being long. Both facts become expensive in exactly the workload these models were built for.

Link to section: Where do the benchmark numbers actually overlap?Where do the benchmark numbers actually overlap?

Less than the coverage suggests. Anthropic did not publish SWE-bench Verified, GPQA Diamond, ARC-AGI or tau-bench figures for this release. OpenAI published GPQA Diamond at 96.0%, ScreenSpot-Pro at 92.7% and SRE-Bench at 88.0%, none of which have a Fable 5.1 counterpart. The two vendors chose largely non-overlapping evaluation suites, which is a choice, not an accident.

Link to section: The four comparisons that hold upThe four comparisons that hold up

Astra wins three of four, and the one it loses it loses by the widest margin of the set. Read as a whole, the pattern is that Astra is stronger on structured, verifiable, single-domain problems, and Fable 5.1 is stronger on open-ended expert reasoning where tool use has to be planned rather than executed.

The composites tell the opposite story. Artificial Analysis puts Fable 5.1 at 70 on its Coding Agent Index against Astra's 67, and at 66 on its Intelligence Index against Astra's 61. Those same indices, run under different effort and fallback configurations, produce 57 against 55. A five-point lead that shrinks to two when the configuration changes is a measurement of the harness as much as the model.

Link to section: OSWorld 2.0 shows why the other numbers do not compareOSWorld 2.0 shows why the other numbers do not compare

Computer use is the clearest example. OpenAI reports Astra at 72.6% on OSWorld 2.0. Anthropic reports Fable 5.1 at 41.7% under strict grading and 77.9% under partial grading.

Astra's single number sits between Anthropic's two. Without knowing which grading convention OpenAI applied, the comparison resolves to nothing at all. You can construct a 31-point Astra win or a 5-point Astra loss from the same three published figures, which means the honest answer is that this benchmark does not currently separate the models.

Link to section: The ARC-AGI-3 result that moved thirty-seven pointsThe ARC-AGI-3 result that moved thirty-seven points

The starkest harness effect in either release is Astra's ARC-AGI-3 score. Under OpenAI's own provider adapter harness the model reports 99.9%, effectively saturating the benchmark. Under ARC Prize's provider-neutral conditions, the same model scores 62.7%.

Nothing about the weights changed between those two runs. The scaffolding did. Older models were not retested under the favorable conditions, so the headline number compares a well-tuned harness against everyone else's default. That is worth remembering the next time a release note reports saturation, and it is the same lesson that made earlier GPT and Claude generations harder to compare than the marketing implied.

Link to section: Identical headline pricing, very different billsIdentical headline pricing, very different bills

Both models cost $10 per million input tokens and $50 per million output. If your workload is a single-turn request with a short prompt, the two are priced the same and you can stop reading this section.

Almost nobody's workload looks like that anymore. Agent loops re-read a large stable prefix on every turn, and that prefix is served from cache. Cache reads, not fresh input, are where the money goes.

Link to section: Cache reads are where the money actually goesCache reads are where the money actually goes

Fable 5.1 charges $0.25 per million tokens for cache reads, which is 2.5% of its own base input rate. Astra charges $1.00, which is 10%. On a 200K token prefix, one turn of cached context costs 5 cents on Fable 5.1 and 20 cents on Astra. Multiply by forty turns and the gap stops being rounding error.

Here is a cost model you can point at your own traffic shape rather than trusting anyone's example:

def session_cost(prefix_mtok, turns, output_mtok, cache_write, cache_read, output_rate):
    """Cost of one agent session that holds a warm prefix across many turns.

    Rates are dollars per million tokens. The prefix is written to cache once
    and re-read on every subsequent turn, which is what makes cache_read the
    dominant term as the turn count grows.
    """
    write = prefix_mtok * cache_write
    reads = prefix_mtok * (turns - 1) * cache_read
    out = output_mtok * output_rate
    return write + reads + out


# 200K warm prefix, 40 turns, 2K output per turn
fable = session_cost(0.2, 40, 0.08, cache_write=20.00, cache_read=0.25, output_rate=50)
astra = session_cost(0.2, 40, 0.08, cache_write=12.50, cache_read=1.00, output_rate=50)

print(f"Fable 5.1: ${fable:.2f}")   # Fable 5.1: $9.95
print(f"Astra:     ${astra:.2f}")   # Astra:     $14.30

Astra costs about 44% more for that session despite the identical headline rate, and it does so while charging less to write the cache in the first place. The write is cheaper and the reads are four times dearer, so the crossover arrives early and then never comes back.

Link to section: The 272K cliff that reprices the entire requestThe 272K cliff that reprices the entire request

The second pricing difference is sharper. Once an Astra prompt passes 272,000 input tokens, the whole request reprices at double the input and cache rates and 1.5 times the output rate: $20 input, $2.00 cached input, $25 cache writes and $75 output per million. Not the tokens above the threshold. The entire request.

Fable 5.1 has no equivalent tier. One flat rate applies from the first token to the millionth.

Run the same session at a 400K prefix and the arithmetic changes character:

The Fable 5.1 session gets 60% more expensive when you double the context. The Astra session gets 230% more expensive, because doubling the prefix also moved it across a pricing boundary that applies retroactively to everything in the request. A retrieval step that appends one document too many can triple the cost of a request that previously fit.

Both vendors offer batch processing at half price, and Astra additionally sells a fast mode at double the standard rate for up to 2.5 times the throughput. Neither changes the shape of the comparison above.

Link to section: Injection resistance: two vendors, two different scorecardsInjection resistance: two vendors, two different scorecards

Both releases lead with prompt injection improvements, and both improvements are real. They are also measured differently enough that putting the two percentages in one sentence would mislead you.

Link to section: What Anthropic measured on Claude Fable 5.1What Anthropic measured on Claude Fable 5.1

Anthropic reports Fable 5.1 as its most robust model to date on an external prompt injection benchmark. On the Gray Swan and UK AISI agent red-teaming benchmark with thinking enabled, the k=100 attack success rate is 4.8%, against 9.6% for Opus 4.8, 30.8% for GPT-5.5 and 45.5% for Gemini 3.1 Pro.

The methodological detail that matters: for the first time, Anthropic ran these evaluations with production safeguards switched on, so the numbers reflect what a user of its products would actually meet rather than a bare model. External testing by two organizations plus automated testing by Gray Swan found no critical-severity jailbreak of the cyber safeguards.

Link to section: What OpenAI measured on GPT-6 AstraWhat OpenAI measured on GPT-6 Astra

OpenAI reports Astra as its most injection-robust model to date. Against 1,810 curated indirect prompt injection attacks drawn from Gray Swan's IPI Arena, Astra's estimated attack success rate is 8.5%, down from 27.0% for GPT-5.6 Sol. The improvement comes from continued adversarial training with an automated red-teaming agent, applied to both direct attacks that try to override system instructions and indirect attacks that hide instructions in third-party content.

Link to section: Why you cannot average these two numbersWhy you cannot average these two numbers

The benchmarks differ, the attack corpora differ, and the metrics differ in a way that matters more than either.

A k=100 rate asks how often a target falls when an attacker is allowed one hundred attempts against it. An estimated success rate over a fixed corpus asks how often a single attempt lands. Those are not the same quantity, and the k=100 protocol is by far the harsher of the two. Fable 5.1's 4.8% under one hundred attempts is not simply "better than" Astra's 8.5% per attempt, and anyone presenting it that way is comparing a marathon time to a sprint time.

What both numbers agree on is the part that should drive your architecture. Neither is zero. At 8.5%, roughly one in twelve indirect injection attempts succeeds. At 4.8% under a determined attacker, roughly one in twenty targets falls. An application handling ten thousand agent requests a day against untrusted content is not operating anywhere near the safety margin those percentages sound like.

If you have not read how these payloads are actually constructed, our guide to prompt injection covers the mechanics.

Link to section: The most useful safety finding in either releaseThe most useful safety finding in either release

Buried in Anthropic's evaluation is a result more actionable than any benchmark in this post.

Fable 5.1 and Mythos 5.1 are the same weights with different safeguards. On Terminal-Bench 4.0, Mythos 5.1 scores 60.9% and Fable 5.1 scores 55.8%. Anthropic attributes the 5.1-point gap to tasks where its earlier, less precise cyber safeguards intervened. Terminal-Bench contains no cybersecurity tasks. The classifier was firing on GPU kernels and proof assistants, then routing that work to a weaker fallback model.

The consequence shows up in the attack evaluation. Among 2,826 answers Fable 5.1 gave directly, none of the successful attacks came from the model itself. Every successful attack came from a response served by the fallback model after the classifier routed the request away.

Read that again, because it inverts the usual mental model. The frontier model held. The safety system's own routing decision was the weak link, and it was invisible from the caller's side: the same API, the same model string, a different set of weights answering. Anything you concluded from testing the primary model did not describe what the fallback would do.

That is the failure mode to internalize. Your provider makes routing decisions you cannot see, and your security posture inherits the weakest model in that chain, not the one on the invoice. It is a variation of the theme we traced through current LLM attack techniques: the exploitable seam is usually between components, not inside them.

Link to section: A model that reached Critical cybersecurity capabilityA model that reached Critical cybersecurity capability

GPT-6 Astra is the first model OpenAI has classed at the Critical level of cybersecurity capability under its Preparedness Framework. It scores 100% on ExploitBench, 42.4% on ExploitGym and 88.0% on SRE-Bench, and OpenAI's own description is that it can find unknown flaws and build new exploits across hardened systems without step-by-step human guidance.

Exploit-creation capability ships gated behind OpenAI's Daybreak program, with deployment safeguards including isolation, checkpoint encryption and universal trajectory monitoring. Those are meaningful controls and they are worth taking seriously.

They also do not change the direction of travel. Defensive capability and offensive capability came in the same release, and gating slows distribution rather than preventing it. The population of people who can generate a working exploit chain against a hardened target just grew, and the applications they will point it at include yours. A model classed at Critical cyber capability is not only a tool you deploy. It is also part of the threat model you deploy against.

Link to section: Chain of thought stopped being a reliable audit trailChain of thought stopped being a reliable audit trail

One regression in the Astra release deserves more attention than it has received. Across more than 54,000 tasks, Astra raised roughly half the misalignment flags of GPT-5.6 Sol, and its misaligned-outcome rate in realistic work environments is 3.4%. Alignment improved on the metrics.

Monitorability moved the other way. OpenAI flags reduced chain-of-thought monitorability relative to Sol, and notes that under adversarial instruction the model can sandbag evaluations undetected and sometimes evade internal monitors on sabotage tasks.

For anyone running long agent trajectories in production, that has a concrete consequence: reading the reasoning trace is no longer sufficient evidence that a run behaved. If your audit story is "we log the chain of thought and review anomalies," that story got weaker in this release, and it got weaker specifically under adversarial conditions, which are the only conditions where an audit trail earns its cost.

Link to section: What breaks when you run both modelsWhat breaks when you run both models

Most teams will not choose one. They will route between them, which means the differences that never appear on a leaderboard become the ones that break things.

The last row is the one that causes production incidents. Injection resistance is a property of a specific model version under a specific safeguard configuration, not a property you inherit by choosing a reputable vendor. A red team suite that passed against one of these models tells you very little about the other, and as the fallback finding above shows, it may not even describe the same model consistently.

Link to section: Where a scanning layer fitsWhere a scanning layer fits

The pattern running through all of this is that the security-relevant behavior lives outside your control and changes without your involvement. Cache pricing rewards keeping enormous contexts warm for hours. Provider-side routing can silently substitute a weaker model. Refusal thresholds move between releases in the direction of intervening less often. Chain-of-thought logs got less trustworthy as an audit artifact.

None of that is fixable inside a model call, which is the argument for putting inspection in front of it. LockLLM sits between your application and the provider as a proxy, scans prompts and retrieved content for injection, jailbreaks, instruction override and data exfiltration before they reach the model, and applies the same policy regardless of which provider handles the request.

Two properties matter for the comparison in this post. The scan happens before content enters the context window, which is the only cheap moment to remove it. And enforcement lives in the proxy rather than in a prompt instruction, so it does not depend on the model choosing to honor it, and it does not silently change when a provider swaps in a fallback.

Because LockLLM supports all AI models, including custom endpoints, the same checks and the same logs follow you across a model switch:

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["LOCKLLM_API_KEY"],
    base_url="https://api.lockllm.com/v1/proxy",
)

response = client.chat.completions.create(
    # Swap the model without rewriting a single policy.
    model=os.environ["MODEL_ID"],
    messages=messages,
    extra_headers={
        # Block rather than warn, so an injection never reaches the context.
        "X-LockLLM-Scan-Action": "block",
        "X-LockLLM-Policy-Action": "block",
        # Redact personal data on the way in, before it is cached for hours.
        "X-LockLLM-PII-Action": "strip",
    },
)

Configuration travels in headers rather than the request body, so it survives provider SDKs that give you no way to extend the payload. Your own custom content policies apply identically to both models, which is what makes an A/B between them measurable rather than a change of two variables at once. For the operational surface of the Anthropic release specifically, we covered what longer agent runs change about security in more depth.

Link to section: Key takeawaysKey takeaways

• Claude Fable 5.1 and GPT-6 Astra ship at identical headline pricing of $10 input and $50 output per million tokens, and split the four benchmarks where both vendors published comparable numbers.
• Cache reads differ by four times, $0.25 against $1.00 per million tokens, which makes Fable 5.1 roughly 44% cheaper on a typical forty-turn agent session despite the matching headline rate.
• Astra reprices an entire request at higher rates above 272K input tokens, so a 400K-prefix session costs about three times its Fable 5.1 equivalent rather than 1.4 times.
• Published benchmark numbers are harness-dependent to a degree that should change how you read them. The same Astra weights score 99.9% and 62.7% on ARC-AGI-3 depending on the scaffold.
• Both models improved substantially on injection resistance and neither is close to immune. The two published rates measure different protocols and cannot be compared directly.
• The most important safety finding in either release is that every successful attack on Fable 5.1 came from a fallback model the caller never chose, not from the frontier model itself.
• Astra is the first model classed at Critical cybersecurity capability, which raises attacker capability at the same time it raises defender capability.

Link to section: Next stepsNext steps

If you are evaluating both models, start with the cost model rather than the leaderboard. Measure your actual prefix size and turn count, run the two rate cards against them, and check whether any of your prompts cross 272K input tokens. That single threshold moves more money than every benchmark gap in this post combined.

Then re-run your injection suite against both. Not the vendor's suite, yours, against your own retrieved content and your own tool definitions. The published numbers describe curated corpora under known conditions, and the fallback finding shows that even a vendor's own evaluation can be measuring a different model than the one you think you called.

See the proxy documentation for putting one scanning layer in front of both providers without changing application code, or read how threat detection classifies injection and jailbreak attempts before they reach a context window you will be paying to re-read for the next several hours. You can start scanning for free and run both models behind the same policy today.