GPT-6 Astra Costs 2.5x More Than GPT-5.6 Sol and Scores About the Same

Book: AI That Ships The series: AI in TypeScript — 5 books, from your first LLM call to agents in production — all five here My project: Hermes IDE | GitHub — an IDE for developers who ship with Claude Code and other AI coding tools Me: xgabriel.com | GitHub A new model lands. Someone on your team opens a pull request that changes one string in one config file, the model id. The diff is green in five minutes. Evals look fine, maybe a point better on the suite you happen to have. It ships. Three weeks later the invoice arrives and it is a different shape than the one before it. Nobody wrote a bad loop. Nobody shipped a prompt-injection. The system does exactly what it did last month. It just costs more to do it, because a one-line diff moved every request from 2 and 10 per million tokens to 10 and 50. OpenAI announced GPT-6 Astra on 3 September 2026. OpenAI calls it the most capable model it has shipped. That is the company’s claim and I am not going to argue with it. But “most capable model available” and “the model your service should call by default” are two different questions, and the distance between them shows up on your infrastructure bill. What the launch numbers say The API list price at launch, per OpenAI: Standard tier: 10 per 1M input tokens, 50 per 1M output tokens Fast tier: 20 per 1M input tokens, 100 per 1M output tokens Astra takes text and image input and returns text only, with a 1M token context window. It went first to a limited set of organisations under OpenAI’s Daybreak Access programme, with wider access to the paid ChatGPT tiers and the API announced as planned for the days after launch. It is also listed on AWS Bedrock and Microsoft Azure. Now the third-party read. Artificial Analysis runs its own evaluations independently of the vendors. On its Intelligence Index, Astra scores 60, which puts it #14 of the 202 models the site tracks. Its cost per Intelligence Index task comes out at 0.96. The median model in that set scores 36 and lists at roughly 2 per 1M input and 10 per 1M output. Two sentences from that page are the reason this post exists. Artificial Analysis states that Astra scores close to GPT-5.6 Sol on the Intelligence Index while pricing is roughly 2.5x Sol’s. It also states that on its Coding Agent Index, Astra scores equal to Claude Fable 5 at lower cost. Both of those can be true at once, and they are the whole argument. The same model is a poor default for general work and a good deal for agentic coding. Which one you get depends on what you route to it. OpenAI published its own benchmark results at launch, and they are vendor numbers rather than independent ones. Two are worth carrying forward, because the routing argument below turns on them: DeepSWE v1.1 at 74.1%, and the offline subset of OSWorld 2.0 at 72.6% at roughly 40 minutes per task. Both measure long agentic runs, which is the one shape of work where an expensive model can be the cheaper choice. The rest of the launch set is reasoning, maths and science scores that no routing decision here depends on; the system card has them. Greg Brockman, OpenAI’s co-founder and president, said “I think it’s not unreasonable to feel that we are now in the AGI era” (VentureBeat). That is his opinion about the field, and you can hold whatever view of it you like. Your finance team will still ask about the invoice, and the invoice is arithmetic. One caveat before any of the numbers below. Everything here is the launch price list. Model pricing moves, tiers get added, discounts appear for batch and cached input. Check the current pricing page before you budget anything. Output tokens are where the bill lives Take a service that handles 1,000 requests a day. Each request sends about 4,000 input tokens and gets back about 800 output tokens. This is an illustrative calculation from the published per-token prices, not a measured bill from anyone’s account. On Astra’s standard tier: Input: 4M tokens at 10 per 1M = 40 a day Output: 0.8M tokens at 50 per 1M = 40 a day Eighty dollars a day, roughly 2,400 over thirty days. Read those two lines again. You sent five times as many input tokens as you received output tokens, and the two halves of the bill are identical, because an output token is priced at five times an input token. That ratio is the thing to internalise. Every prompt-engineering instinct you have is about the input side, and the input side is the cheap half. A change that makes responses more verbose is worth far more on the invoice than a change that makes prompts longer. Watch what happens if output per request goes from 800 tokens to 3,000, which is an ordinary consequence of asking for more reasoning in the response: Input: unchanged at 40 a day Output: 3M tokens at 50 per 1M = 150 a day The bill more than doubles and your request volume did not move. Nobody wrote a loop. Somebody changed a prompt. The fast tier doubles both sides again, to 160 a day on the original workload. It buys latency. Decide whether the requests that need it are all of them or the 5% a human is sitting in front of. For contrast, running the same workload against a model at the median price Artificial Analysis reports (2 in, 10 out) comes to 16 a day, or about 480 over thirty days. That is the gap you are deciding about. It is not marginal. Terseness is a discount The nuance cuts the other way. Per-token price is not per-task price, and Artificial Analysis publishes both. On its index run Astra emitted 16M output tokens against a 62M median across the models it tracks: roughly a quarter of the tokens, for a score of 60 against the median’s 36. That concision is why its cost per Intelligence Index task lands at 0.96, rather than wherever a 50-per-million output price would put it on its own. Be careful what you take from that. The 0.96 is one benchmark suite, measured against the whole tracked field. The 2.5x is a different measurement, per token, against GPT-5.6 Sol specifically. The two do not multiply into anything. What survives is the shape: a verbose model at a low list price and a terse model at a high one can land much closer on a per-task bill than the price sheet suggests, and the ordering can flip either way. This is exactly why a price-per-million-tokens comparison is a bad way to pick a model. Two models with the same list price can differ by 3x on your actual bill, because one of them thinks out loud and the other does not. And a cheap model that fails validation, gets retried twice and then escalates has cost you three calls plus the expensive one. The number that answers the question is cost per successful task. type Outcome = { usd: number; accepted: boolean; }; export function costPerSuccess(rows: Outcome[]): number { const spend = rows.reduce((s, r) => s + r.usd, 0); const wins = rows.filter((r) => r.accepted).length; return wins === 0 ? Infinity : spend / wins; } Log usd and accepted on every request and you can compute this per model, per route, per customer tier. accepted is whatever “this actually worked” means in your domain: the JSON parsed and passed schema validation, the generated patch compiled, the support reply went out without a human editing it, the extracted invoice total matched the ledger. Run the expensive model on 10% of traffic for a week and compare that one number against the cheap model’s. If the expensive model succeeds often enough to beat the cheap model’s retry tax, it is the cheaper option and the per-token price was a distraction. Usually it beats it on a slice of your traffic and loses on the rest, which is the case for routing. Route cheap first, escalate on a signal The pattern is a ladder. Call the cheap model. Check the result against something you trust. If the check fails, escalate to the expensive one. Refuse any call that would push the request past a cost ceiling you set in advance. Start with prices and a cost function. // Launch list prices in USD per 1M tokens. // Verify current pricing before you rely on these. export type Price = { inPerM: number; outPerM: number }; export const PRICES = { cheap: { inPerM: 2, outPerM: 10 }, strong: { inPerM: 10, outPerM: 50 }, } as const satisfies Record<string, Price>; export function costUSD( p: Price, inTok: number, outTok: number, ): number { return (inTok / 1e6) * p.inPerM + (outTok / 1e6) * p.outPerM; } Then the two shapes the router needs: what a model call returns, and what a validation check returns. export type Completion = { text: string; inputTokens: number; outputTokens: number; }; export type ModelCall = (p: string) => Promise; export type Check = | { ok: true; value: T } | { ok: false; reason: string }; export type Validate = (text: string) => Check; Check carries a reason on failure. That string is the most valuable thing this system produces. It tells you why the cheap model was not good enough, which is the input to every future decision about whether the escalation is worth paying for. The router’s result type is where the design decision lives. A routed request and a refused one carry different fields, so make them separate members of a union keyed on stop. export type Tier = “cheap” | “strong”; export type Routed = { stop: “validated”; value: T; model: Tier; usd: number; attempts: string[]; }; export type Refused = { stop: “ceiling” | “exhausted”; value: null; model: “none”; usd: number; attempts: string[]; }; export type RouteResult = Routed | Refused; The options bag is what the caller configures. refuse builds the refusal half of the union in one place, so the two exit paths that give up cannot drift apart. export type RouteOptions = { prompt: string; cheap: ModelCall; strong: ModelCall; validate: Validate; maxUSD: number; maxOutputTokens: number; }; const estTokens = (s: string) => Math.ceil(s.length / 4); function refuse( stop: “ceiling” | “exhausted”, usd: number, attempts: string[], ): Refused { return { stop, value: null, model: “none”, usd, attempts }; } The router itself walks the ladder in order, forecasts what each tier would cost before it calls it, and stops at the first result that validates. export async function route( o: RouteOptions, ): Promise<RouteResult> { const attempts: string[] = []; let usd = 0; const tiers = [ { name: “cheap”, call: o.cheap, price: PRICES.cheap }, { name: “strong”, call: o.strong, price: PRICES.strong }, ] as const; for (const tier of tiers) { const worstCase = costUSD( tier.price, estTokens(o.prompt), o.maxOutputTokens, ); if (usd + worstCase > o.maxUSD) { return refuse(“ceiling”, usd, attempts); } const res = await tier.call(o.prompt); usd += costUSD( tier.price, res.inputTokens, res.outputTokens, ); const check = o.validate(res.text); if (check.ok) { return { stop: “validated”, value: check.value, model: tier.name, usd, attempts, }; } attempts.push({tier.name}: {check.reason}); } return refuse(“exhausted”, usd, attempts); } Three details in there carry the weight. The ceiling check runs before the call, and it forecasts. It prices the worst case for the tier it is about to use, using the output cap you configured rather than the output you hope for. If that projection clears the ceiling, the call never happens. Check after the call and you have already paid for the request that put you over. Note that estTokens is a rough heuristic, so a prompt that tokenises worse than four characters per token can still take the final total slightly past the ceiling. Swap in a real tokeniser if you need the ceiling to be exact rather than close. stop is a return value the compiler understands. A request that ends on ceiling is a different product decision than one that ends on exhausted. The first means you refused to spend; queue it, batch it, or ask the user to confirm. The second means both models tried and neither produced something valid; that is a human’s problem. Throw an error and both collapse into the same thing at the call site. Key the union on stop and the narrowing does double duty: it separates the two refusals, and it is what lets value be a real T in the validated branch instead of a T | null every caller has to re-check. The third detail is usd. It comes back attached to the answer, so you can write it to the row and put it on the trace at the point where you still have it. You cannot compute cost per successful task later if you did not record the cost per attempt now. Pick an escalation signal you can trust The router is only as good as validate. There are three families of signal, and they are not equally good. Deterministic checks are free and they are the ones to reach for first. Schema validation, a JSON parse, an enum membership test, a compile step, a unit test run, a lookup that confirms the entity the model named actually exists in your database. These cost nothing per request and they never lie. type Ticket = { category: “billing” | “bug” | “account” | “other”; urgency: 1 | 2 | 3; }; const CATEGORIES = [“billing”, “bug”, “account”, “other”]; const validateTicket: Validate = (text) => { let raw: unknown; try { raw = JSON.parse(text); } catch { return { ok: false, reason: “not_json” }; } if (typeof raw !== “object” || raw === null) { return { ok: false, reason: “not_object” }; } const t = raw as Record<string, unknown>; if ( typeof t.category !== “string” || !CATEGORIES.includes(t.category) ) { return { ok: false, reason: “bad_category” }; } const u = t.urgency; if (u !== 1 && u !== 2 && u !== 3) { return { ok: false, reason: “bad_urgency” }; } return { ok: true, value: t as Ticket }; }; In a ticket handler the union earns its keep at the call site: inside the validated branch, result.value is a Ticket rather than a Ticket | null, so save takes it without a null check. const result = await route({ prompt: buildPrompt(ticketText), cheap: callCheapModel, strong: callStrongModel, validate: validateTicket, maxUSD: 0.05, maxOutputTokens: 300, }); if (result.stop === “validated”) { await save(result.value); } else { await queueForHuman(ticketText, result); } Set maxUSD so that the full ladder fits inside it, or the ceiling fires before the escalation ever runs and every hard request comes back as ceiling. At these prices, a 1,500-token prompt with a 300-token output cap costs at most 0.006 on the cheap tier and 0.03 on the strong one, so the whole ladder needs about 0.036. A ceiling of 0.05 clears that with headroom and still refuses to escalate a 10,000-token prompt that somebody pasted a log file into. Work out your own from your p95 prompt size rather than copying this one. Model-reported confidence is the weakest signal and the most tempting. Asking a model to rate its own certainty from 0 to 1 gives you a number that correlates with something, but not reliably with correctness. If you use it, treat it as one input among several, and calibrate the threshold against outcomes you have actually labelled. Do not ship it as your only gate. A second opinion costs a second call. Running the cheap model twice and escalating on disagreement works, and it doubles your cheap-tier spend to avoid some of your expensive-tier spend. Whether that trade wins is arithmetic you can do with the numbers above: it pays when the expensive model costs more than the extra cheap call and disagreement is a decent predictor of failure. Rank them in that order. Deterministic first, second opinion when you have no deterministic check available, self-reported confidence last and never alone. Where the expensive model earns its price None of this is an argument for always routing to the cheap model. There is a shape of work where the expensive one is the correct default, and the launch numbers point at it. Long agentic runs. Artificial Analysis puts Astra level with Claude Fable 5 on its Coding Agent Index at lower cost, and OpenAI reports 72.6% on the offline subset of OSWorld 2.0 at roughly 40 minutes per task. On work like that, a weaker model does not fail cleanly. It takes a wrong turn at step 4 and spends another twenty steps building on it, and every one of those steps is a paid call. The cheap model’s failure mode is expensive in a way that per-token price does not show. The heuristic that follows: the longer the autonomous run, the earlier you should escalate. A single classification is a good candidate for the cheap tier with a validator behind it, because the retry costs one call. A forty-minute agentic task is a bad candidate, because the retry costs forty minutes of calls and you find out at the end. The same logic applies in reverse. High-volume, well-specified, schema-checked work — classification, extraction, routing, tagging, short summaries — is where the cheap tier belongs, and where the 2.5x premium is hardest to justify from anything you can measure. The change to make this week Two things, both small. Attach cost to every LLM response your service produces. Not to a dashboard. To the response object, the log line, and the trace span. Multiply tokens by the per-million price at the point of the call. It is four lines of code and without it every question in this post is unanswerable. Then pick your highest-volume LLM endpoint and put a validator in front of the model choice. Run the cheap tier, check the output, escalate on failure. Give the whole thing a per-request ceiling so one pathological input cannot spend unbounded money. Compare cost per successful task after a week. The frontier model is a real achievement and the benchmark numbers are what they are. It is still a component with a price, and you are still the one deciding which requests are worth it. If this was useful Cost ceilings, escalation signals and eval suites that tell you whether a model swap actually helped are the parts of an LLM system that only matter once real traffic is hitting it. That is the territory AI That Ships covers, in TypeScript, with the instrumentation attached. It is book 5 of AI in TypeScript, a five-book series that runs from your first LLM call through to agents you can leave running in production.

View original article