Not necessarily, they may have selected benchmarks where Terra performs relatively well compared to those models. Pretty much every model release these days just cherry-picks & highlights the benchmarks where their latest model performs best. We'll have to wait for the complete benchmark suite to have a better picture.
Looks great on the first chart, but terrible on the Terminal Bench 4.0 where it needs xhigh to come close to 6.1 Sol, and costs more for a slightly worse result.
IMO it's useless because we don't even know what a pelican on a bike is supposed to look like, It's very impressive that that models can make these directly using SVG, but we are way past the "wow" stage and test that can't be gamed and are easy to prove how good the model is in that specific arena are hard to find.
The only thing that pelican have is "models where not trained on this" maybe that true months (or years ago) now it's on every HN post and these AI labs are monitoring HN
People keep asking OpenAI for a mode that's as good as Opus 5.5, and they release this instead. Interesting strategy. Let's see how it works out for them.
Anthropic models are comically overpriced. They also gobble up an obscene # of tokens to get the same result as others. The price delta is literally 5x.
Taking the benchmarks (as reported by OpenAI) at face value, why wouldn't they show Opus 5.5 on the OS world benchmark, and show no Anthropic models at all on the "factual error rate" benchmark
Right?! I think sol has some obsessive tendencies, i literally caught it the other day spawning a sub sub agent doing a final fourth test for a piece of code that isn’t even critical say in embedded or similar, now I have to instruct it to only test once!
This release is interesting for enterprise at API rates.
At least unless the Terminal Bench 4.0 score is more indicative than the other ones, because that one shows it clearly worse than 6.1 Sol for both performance and cost.
They don't have anything competitive with Opus 5.5 yet. There is a supposed model in the works, "Bel" last time I paid attention to gossip, that is thought to be better, later this year. Allegedly Anthropic caught OpenAI off guard and / or their different training approaches ended up working better than anticipated.
My expectation is Fable 5.5 is next overall improvement, and my bet is OpenAI's "Bel" will be the next competitive one (or, their lack of safety culture will bite them/us in the process).
There’s also a chance they surrender the dev market altogether to Ant and just focus on the “normie” segment with dots, computer use, excel etc. In which case they won’t have anything competitive to opus-5.5 anytime soon for SWE.
They're doing its because they recently cut plan usage by about 50%, leaving Anthropic providing about 5x the value on their plans. [0]
The "cost per task" metric is just a weaselly way to sound like they didn't just double the cost for most users, and aren't having their proverbial lunch money stolen. They can no longer claim to provide more tokens, more capable models, or more usage in general. So they made up a new metric where "task" could be defined to mean whatever they need it to mean.
They had to become VERY specific in their wording to present any kind of value proposition that will entice people.
Cost per task, time per task, quality (good luck measuring objectively between models) are the metric that matter. I don't care how much a token costs, I'm not buying tokens, I'm buying solutions.
Second of all, barring more cuts, yes I do think the $20 and $100 plans do deliver more tasks because the models have gotten cheaper in the meanwhile (like this very announcement).
> Are they delivering 50% more than Anthropic per token?
So say the public benchmarks. Pull up artificial analysis, “Cost (USD) to run all evaluations in the Artificial Analysis Intelligence Index”:
Sol 6.1 (max): $1,082
Opus 5.5 (max): $8,708
Sonnet 5.5 (max): $7,260
The original plan doesn't exist anymore. To get the same usage you now need a $500/month plan. The new plan that costs $200 is different in every way that counts. It was actually a 150% price increase for people who want or need what the old plan used to provide.
Don't get me wrong, I genuinely LIKE OpenAI. It's probably worth it even at the new price. I just hate the rhetoric designed to Goodhart past the math.
> the models have gotten cheaper in the meanwhile (like this very announcement).
That is not good news for plan users. Plan users now get less of something that also has less worth to begin with. It would only be good news if they got more of it.
For example, if I buy the "unlimited car wash" package for $19.99/month and the car wash lowers the price per wash from $10 to $1 my subscription becomes less valuable, not more. The breakeven has moved, so I now need to wash my car more than 20 times before the package is justified.
If the car wash then increased the price to $50/month would it be wise for me to keep the package? Would lowering the price per wash make the package more valuable to me somehow? No, it would do the opposite.
What is the point you're trying to make? You can define a task yourself and run two models and see which one costs more, takes longer, and produces better code.
Once you are inserting models into non-creative (and software development is creative work) it is imperative to start creating your own small evals. Not just to check how different models perform in that equation, but to catch model drift over time.
> 5x is calculated in terms of API costs, nobody pays those extreme Anthropic prices outside enterprise users.
Ignore value then. Anthropic still provides about 3x as many raw tokens. The 20x\200 plans provide about 10B for openai versus almost 30B for Anthropic.
Do you really think that Opus uses 3x as many tokens as Sol per task? I don't.
> Most users aren't on their $200 plan.
Fair enough. I should have said most heavy users.
> Most users (including on the $200 plan) continue being subsidized.
They continue to be fifty percent less subsidized, I guess.
Look, I don't fault them for trying to become profitable. I fault them for the weird weaselly business speak about cost per task. It's BS game playing and we all know it.
I'm honestly rooting for them, and hope it all works out. For now though I'm pretty salty that my bill jumped by 150% in one month. Gaslighting about made up metrics just makes me more angry.
People in HN live in a bubble. Plenty of real world use cannot use subscription plans (not even talking about enterprise agentic use) and is absolutely at the mercy of the cost per task metric.
We agonize over cost per task to the extreme for services/functionality built atop of APIs and can absolutely guarantee you that it is the real world metric that counts most right after “is the model good enough for the task at hand,” and furthermore that, yes, in fact, it is quite often that we see certain models/providers easily using 3x or more tokens on average for the same tasks (or even the exact same task) than a supposedly similarly matched model from another provider. Once you become intimately familiar with the thinking traces from specific models, you can preempt this to some extent with manipulations to the system prompt (essentially heading off entire thinking branches) but the nuking of reasoning traces in the name of preventing against distillation attacks has made this harder.
How do you know? Have you done the A/B test yourself?
> Do you really think that Opus uses 3x as many tokens as Sol per task? I don't.
You don’t have to speculate. Pull up artificial analysis, pick either opus or sol and scroll down to total tokens and to total price. Opus (and sonnet) use 4x the tokens and 7x the dollars.
That difference alone removes the difference that semi analysis found.
> How do you know? Have you done the A/B test yourself?
Fair point. I just tried it, and I stand corrected. Claude uses almost exactly 3x as many tokens.
In the end Claude produced 2.25x more code, and used 3.01x more tokens (ignoring the cached system prompt, etc). It's answer was also much more substantial and feature rich - including many features I did not ask for.
I gave them both the prompt "Without searching the internet please write and test a python script that accepts a year (in any common format) as input and provides the date that easter fell or will fall on in that year. The output should be in discordian date format. You may not use any libraries."
The thing I found most interesting were the unit tests. Claude implemented the core code using both the Meeus–Jones–Butcher algorithm AND Gauss, but only used Gauss as a sanity check to ensure that it's primary code worked. Sol just validated that it matched some dates it could remember. That explains a good chunk of the almost doubled output, and it was a reasonable thing to do, though neither answer was "wrong."
Token usage looked like this (based on parsing it out of the jsonl, neither provider seems to make it easy to get real billing information per project/conversation.
> neither provider seems to make it easy to get real billing information per project/conversation
I believe oh-my-pi will be able to ingest your Claude and codex conversations, count tokens, and then assign API-equivalent rates to them. May be worth looking into.
That terminal bench 4.0 result is... bad? Terra XHigh more expensive than Sol Low and still does worse.
It's very likely this is a next checkpoint of GPT-6 "Sol", over-optimised for the web chat and offered to Codex subscribers as a "please go away" offering.
If this was half the price, I'd be very interested, but at 1/5/0.05? Feels like another panic release because they can't get Astra 6.1 out of the door...
Similar 75% price cut trajectory as GPT-5.6 Sol -> GPT-6.1 Sol. Overall, Terra is still half the price of Sol.
But like the original release of Terra, it's still a weird middle child between the small-model-that-got-substantially-better (Luna) and the large-model-that-got-significiantly-cheaper (Sol). The TerminalBench 4.0 cost-per-task results are a bit confusing and I'm not sure why you would pick Terra over Sol in that case. Overall this release seems like a flex on Sonnet 5.5: a model which is having its own middle child problems between the godly Opus 5.5 and the surprisingly capable Haiku 5.5.
It is having improvements in other fields than coding while being more cost efficient.
If all I want is coding at a cheaper price, then deepseek is a good model for that.
I am more than okay to have a bifurcation of models good for coding and models good for everything other than coding. I strangely imagined it many years ago.
Everyone is suggesting OpenAI to release a model close to opus 5.5 yet it's releasing this. On one hand we can imagine that maybe their training cycle wasn't able to produce this whereas gpt 6.1 terra was made.
But on the other hand, I also now wish to see an OpenAI model which is good for just pure coding, like an absolute beast at it while being cheap (so the opposite of terra)
ironically, one of the few appreciations that I might give to OpenAI if they pushed this model to prod and are facing some pushback as developers like us want/only see coding models. They could've benchmaxed it and it would've been slightly irrelevant. I'd much rather see reality.
At the very least, this new model won't make things more expensive for anybody and would only make use-cases cheaper depending on what they are, so it's a fine deal for me.
I also wonder maybe this is why open source labs aren't targeting this space and why the only focus of AI labs is coding nowadays. Everything else just doesn't seem so worth it from investor, even end user, training perspective.
Maybe this is the reason why we have gotten almost an exponential growth in coding capabilities as that became the most (if not only) capability that truly really matters.
To not wish for discussion to go to a tangent, nonetheless this could also explain why our work as a software engineer is going through a slight crisis of meaning as these AI models have gotten impressive at coding.
As an (aspiring) developer (currently in 1st year college), I don't wish for AI to replace coding even more for me or to push it even more or to suck the joy out of it even further. So I am more than happy for these models to stay as they are and rather make advancements elseplace. Ironic because it is us developers who use AI as well and we want cheaper workflows and so we would also want models to be cheaper and better at coding. There are some fine lines of contradictions and irony in all of it. I'd much rather also have open-weight models of such terra-like models but I imagine that because of the reasons that I mentioned above, we are most likely gonna see it the most focused on coding but still open-weights is pretty close.
The more choices the better (though choice-fatigue is a real thing), I hope I was able to explain some of the nuance I feel about this particular launch.
On purpose? I was using qwen3.8-27b the other day and very quickly ended up burning through half of the context (it has 128k total) pretty quickly. Not because of my directions, but simply because the codebase was very large.
256k seems sweet spot. Even large codebase can be handled you just has to map out areas for agent. Anything past 256k usually exception than norm to be.
At least this my discovery when I did evals on my reviewers skills. 10k LOC is high end for high effort or around 150k context using opus. You can push towards 15k but that’s on max effort with about 400-500k ctx.
Fast models like haiku do not even need 1M ctx in my view
Artificial Analysis still has MiMo 2.6 Pro setting the Pareto frontier where it matters: it's as cheap as Terra High/Sol Low while being smarter than Terra Xhigh. Sol Medium beats MiMo in smarts but is priced a lot higher. Meanwhile Terra Max is dominated by Sol High in both price-per-task and smarts.
I recently tried MiMo 2.6 Pro on a simple app and was positively impressed (modular code with sensible tests). I then tried it on a more difficult problem (which Opus 5.5 solved in minutes) and it looped for 12 hours without making any apparent progress before I finally pulled the plug.
But minor credit to Anthropic, at least they've mostly stuck to their four model names (Haiku, Sonnet, Opus, and Fable). You mostly know where a model is MEANT to be positioned from the name alone. OpenAI cannot keep it consistent from release to release.
But, yes, currently Opus Vs. Fable has the same issue as Sol-6.1 Vs. Astra.
I don’t think opus-5.5 is better than fable-5.1. I think opus is better overall because it’s much faster and cheaper than fable but if speed and cost is not a factor fable it’s probably just as if not slightly better than opus, especially at front-end design (tho the difference is marginal and no one should be really using fable day to day due to this alone).
> OpenAI cannot keep it consistent from release to release.
Hmm? They've been using the Luna/Terra/Sol/Astra scale for a few versions now and as far as I can tell it maps directly onto Haiku/Sonnet/Opus/Fable, in terms of the kind of thing it's meant to convey (if not in terms of the scale of differences).
Both companies have screwed up their naming royally. Maybe that's intentional - I'm sure it doesn't hurt Anthropic to have people using Fable a bit longer than they need to and racking up a huge bill in the process.
They're always very clear about it when a new model comes out. You have to click away a banner to continue using the service. There's not much they can do to make it any more obvious without it being a major hassle for all the people who understand versioning.
It's like iPhone 17 being faster than iPhone 16 Pro. Because it's newer and benefits from advances in chip architecture, despite being a lower model tier. Sometimes different iPhone models are released at different times.
Part of the confusion is that it's not linear capabilities. One model is not strictly "better" than the other. They have orthogonal capabilities.
I think the names are more of a reference to price. Luna is always cheaper than terra which is cheaper than Sol which is cheaper than Astra (for the same model number).
I don't understand what's so confusing about this.
The lineup of models is still the same.
There are four models: luna, terra, sol, and astra.
Pricing increases for each level. This new 6.1 Terra is 50% cheaper than Sol 6.1.
They are lowering prices for newer releases, so yes if you compare Sol 6.1 to 5.6 Terra it won't look right, but why are you doing that? Look at the same release version for comparable numbers.
Each model improves in successive releases but they maintain pricing and capability coherency within a release version.
Yes, Sol 6.1 is supposed to rival Astra for some work, which is what you'd expect from a later versioned release model against a prior release bigger model.
Assuming they release Astra 6.1 at some point, I expect it'll be better than Sol 6.1, at a higher price.
I don't understand what people are asking for here? No reductions in pricing? Different model names on every version release? Not releasing any models for a version number until they're all ready?
Hey! I think it's confusing because there is a mix of three components:
- Model name
- Version number
- Effort level
All 3 of these are a one-dimensional balance of price/speed, and capability (Loose terms). The confusion is in how these overlap, how they're announced/marketed etc. So you go from 1 dimensions to 3; you can visualize it as a box. For a given task, how do you choose where in this box you select? Which regions are your favorite? Can you map each region of this 3D space to specific tasks? Label them with intuitive names?
Given the operations of the model are opaque, can you see how someone (most people?) might be confused?
Effort (as it's used in the context, how many "thinking" tokens to burn) is still a free variable. It's not always shown to the end user but you can set it independently for any of these models.
After that you just have model "strength" and version number. But of course strength is only comparable at equal version number and the two axes are almost parallel.
Anthropic also does this, perhaps to a lesser extent.
But what's the ask? This is the cleanest way I see of them doing it.
I see a ton of criticism and not a single person coming up with anything better, unless it's to take away options, like removing effort level.
What about buying Intel CPUs? A car? Electrical components?
If you're not just using this at a consumer level to ask about cooking recipes (and if that's your use case, you really don't care about API pricing, or any of this, and can just use the default), and are using it professionally, then this seems like a low bar of effort to understand what you're buying.
I do grant you the relationship between model and effort level has an unclear relationship, but there are also tradeoffs in most complex technical products that are not always clear; this is exacerbated by the fact that this product is stochastic.
Why are engineers using this as a professional tool expecting this to be bought like a consumer gadget and then blaming the maker?
You are showing a selection from some third-party interface that mixes together models from six different suppliers: Mistral, Kimi, OpenAI, xAI, Anthropic and Google, with a bunch of old obsolete models.
Are you presenting this as evidence that OpenAI's naming scheme is bad?
If you don't want to spend the time to know the difference between all these different suppliers and their models, then why are you using whatever this program is that gives you so many choices, most of which are bad? Why is this OpenAI's fault?
The OpenAI employee above says that some smaller models of the same generation are better at certain tasks than larger models, regardless of cost difference.
They should have a guide saying "Terra is best at technical documentation and communication, Astra is best at large software projects and research, Sol is good for development" or something like that.
Then he qualified to say it depends on the unique details of the documentation. I don't think its possible. You have to try all models with effort and compare all or go on reputation or vibes.
“The ask” is give me a clue how to choose the best settings for a specific task. Model releases happen so often that the calculation keeps changing. Maybe even OpenAI doesn’t always know the answer.
This morning I asked GPT which models and settings I should use to inspect my VM dashboard with computer control. It said:
- GPT-6 Astra at Medium
- GPT-6.1 Sol at Medium (alt)
- use Luna only when the route was already known.
But I had already completed the task twice with Luna and it was 5.6! It navigated through a few pages of my dashboard, pulled up the 30-day usage chart, analyzed my CPU and RAM, and explained why I didn’t need an upgrade. I’d give Luna 5.6 an A on this task.
Maybe OpenAI should give us a “Tiptronic” selector: recommend the model and reasoning level based on the prompt, while still letting users override it.
> “The ask” is give me a clue how to choose the best settings for a specific task.
What are you talking about? If it's for some slop you vibe check it. If it's for a serious effort you write an eval. Same as it has been for the last 3 years.
There's also this "Instant" thingy which doesn't seem to be on the model-name axis but I really can't understand exactly what it is, even after asking ChatGPT about it.
It's confusing because the models are released one at a time rather than as a family.
Let's say I have some task that Luna 6.1 cannot do, but Terra 6.1 can. So I build my product around that. In a month when Luna 6.2 is released, then I have to figure out whether it's worth to evaluate and swap or just wait for Terra 6.2.
I think a lot of confusion comes from the discombobulated rollout of these models as well.
They released gpt-6-sol (alongside astra and luna but no terra) and 7 days later released gpt-6.1-sol making gpt-6-sol basically DOA before any real adoption. Everyone knows it wasn’t just better RL/post-training, it was a completely different model underneath. Now they randomly reintroduce terra.
I’m sorry, but if you’re not even a tiny bit befuddled by this then you must be taking whatever OpenAI is having.
If you scale a model down, or switch out attention with linear attention, or introduce KV-compression, etc., you don't really know how it will affect model performance before you've trained the model.
And once you've spent millions training it, you might as well make it available if it's pareto optimal for some tasks. It also gives you more insight seeing how people use it in practice.
The way to think of this is that (current) inferencing hardware has natural cut points of efficient scale (assuming some optimal architecture which at this point is almost always an MOE variant), so you always have an incentive to provide 3-4 model sizes (which nicely enough often means diff speed to answer also).
You are also rolling the training of your models by size and the lat est small model can sometimes beat an older model.
For a while we thought you just train the big model first then distill smaller models from it but since we've learnt the importance of on policy RL (basically the way a small model figures out a problem is very diff from a big model so it's best to RL optimize each model distinctly)
It's especially confusing because Sol 6.1 is mostly just Astra but 10x slower at 1/10th the price - unless you pay for it to be as fast as Astra - at which point it's the same speed and price...
It's nice that they have Terra which seems to fill a void that's been needed since Sol 6 focused on cost (at the expense of quality), Sol 6.1 got a huge boost in quality, but became unusably slow unless you paid Astra prices - at which point, why not just use Astra?
I work at OpenAI and I can try to explain our logic.
In general, LLM performance is incredibly multidimensional. There's no guarantee of a fixed ordering where the model that is best at math is also the model that's best at jokes. Despite that complexity, much of the variation in performance is explained by just a single variable: model size. In general, models that are bigger will (a) be more performant / reliable, (b) cost more per token, and (c) have better recall of esoteric facts. As you note, our models are ordered:
Luna < Terra < Sol < Astra
Even today, I'd recommend this as a general starting point for picking models. If you care about quality only, pick Astra. If you're a bit price sensitive, pick Sol. If you're very price sensitive, pick Terra or Luna.
Of course, model size doesn't capture all variation. Models that are trained later might have new data or new techniques or new architectures that earlier models lack. You can see this with models from all providers - Sonnet 5.5 outperforms Fable 5.1 in ways, and Gemini 3.8 Flash outperforms 3.1 Pro in ways.
In the case of GPT-6.1 Terra, I'd say Terra is probably better than Astra and Sol at technical communication, similar on vision and computer use (but much faster!), and a bit worse on most other things. For API developers, it's a nice midrange option if you want something cheaper than Sol but a lot smarter than Luna (especially for vision / computer use / communication). For subscribers, it's not a bombshell release - if you're hitting usage limits, maybe it helps you go a bit further, and if you want clearer communication or faster computer use, it can be a nice option to select.
If you have suggestions on how to name our products better, I'm all ears. Our intention is to make things clear and simple, even if we miss the mark sometimes.
Unfortunately, a lot of the complexity is somewhat irreducible and due to to the actual complexity of the products. Scaling model size is different from scaling reasoning effort which is different from scaling multi-agent count. So if we just swept all of those under a single name of "more IQ", it would make the naming simpler but the experience much worse, as important details would be obfuscated from end users.
> If you have suggestions on how to name our products better, I'm all ears.
For some reason, this is just hilarious to me. Idk if you're being serious or if this is passive aggressive but the thought of OAI actually taking advice from a random HNer and basing their naming scheme off that is peak.
This is one of the last bastions of semi-serious discussion on the internet, and it's full of incredibly high level engineering talent.
Where else would you go? I'm honestly scratching my head thinking of someplace that wouldn't run you out on a rail, but that would give serious feedback. Kernel mailing lists are out, maybe there's something like that which I can't remember off the top of my head, but...
Also, nomenclature is incredibly hard. There are only two hard problems in computer science: cache expiration and naming things.
But regular people are not the consumers of OpenAI's API - they consume "Chatgpt" and don't care about model nomenclature.
The primary users of any API are the designers and engineers of the applications that use it. How many laypeople know what Postgresql is? How many of the applications they use run on it?
We aren't talking about the API. We're talking about the model names. If someone fires up the OpenAI website to ask it a question, nobody knows what the names mean. That's what normal people do and are probably super confused at what is going on.
Please can I ask about one parameter you didn't mention here? All of your models have both a reasoning effort setting and also a non-pro vs pro setting. I have some intuitive feel (from experimenting) for the difference a larger vs smaller model makes, and why that's different from adjusting reasoning levels, but I have no real feel for how pro vs non-pro is different in behaviour from adjusting effort.
I think you can think of Pro approximately as running multiple instances of a non-pro model, than giving all their outputs to a final judge model which combines them into a better one.
- Understanding how effort and class interact (Is a high effort terra above a low effort Sol? Are larger models mostly always better regardless of effort?)
- understanding when (if ever) a new model tends to be better than an older but higher class one. Is an old astra better than a new terra?
If increasing effort and increasing model class produce improved results in a different way, I miss making it explicit in which way. In other words: What kind of issue warrants going up in effort? what kind of issue warrants going up in model class? What kind of issue is unlikely to be solved by either?
OpenAI itself cannot answer these questions. A prospective price-sensitive (or speed-sensitive) user has to experiment to find the cheapest (or fastest) model that is sufficient for each task.
The pelicans place it nicely between Sol and Luna in terms of both pricing and results.
The only thing that pelican have is "models where not trained on this" maybe that true months (or years ago) now it's on every HN post and these AI labs are monitoring HN
At least unless the Terminal Bench 4.0 score is more indicative than the other ones, because that one shows it clearly worse than 6.1 Sol for both performance and cost.
My expectation is Fable 5.5 is next overall improvement, and my bet is OpenAI's "Bel" will be the next competitive one (or, their lack of safety culture will bite them/us in the process).
There's very few stuff that needs Astra/Opus level intelligence.
Luna 6.1 would be amazing.
I'm optimistic about them tackling frontier intelligence next. Astra 6.1 please?
The "cost per task" metric is just a weaselly way to sound like they didn't just double the cost for most users, and aren't having their proverbial lunch money stolen. They can no longer claim to provide more tokens, more capable models, or more usage in general. So they made up a new metric where "task" could be defined to mean whatever they need it to mean.
They had to become VERY specific in their wording to present any kind of value proposition that will entice people.
[0] https://newsletter.semianalysis.com/p/anthropic-subscription...
Nope to both. Not even close.
The only way it works is if you define "task" in a way that OpenAI is good at it and Anthropic isn't.
Second of all, barring more cuts, yes I do think the $20 and $100 plans do deliver more tasks because the models have gotten cheaper in the meanwhile (like this very announcement).
> Are they delivering 50% more than Anthropic per token?
So say the public benchmarks. Pull up artificial analysis, “Cost (USD) to run all evaluations in the Artificial Analysis Intelligence Index”:
Sol 6.1 (max): $1,082 Opus 5.5 (max): $8,708 Sonnet 5.5 (max): $7,260
>7x difference in cost for the same tasks.
The original plan doesn't exist anymore. To get the same usage you now need a $500/month plan. The new plan that costs $200 is different in every way that counts. It was actually a 150% price increase for people who want or need what the old plan used to provide.
Don't get me wrong, I genuinely LIKE OpenAI. It's probably worth it even at the new price. I just hate the rhetoric designed to Goodhart past the math.
> the models have gotten cheaper in the meanwhile (like this very announcement).
That is not good news for plan users. Plan users now get less of something that also has less worth to begin with. It would only be good news if they got more of it.
For example, if I buy the "unlimited car wash" package for $19.99/month and the car wash lowers the price per wash from $10 to $1 my subscription becomes less valuable, not more. The breakeven has moved, so I now need to wash my car more than 20 times before the package is justified.
If the car wash then increased the price to $50/month would it be wise for me to keep the package? Would lowering the price per wash make the package more valuable to me somehow? No, it would do the opposite.
I’m not sure I see how. Subsidy got cut by half. But in the meantime 6.1 came out which uses 25% fewer tokens than 5.6 and costs 1/3 as much per task.
So for 1/2 the budget you use to have with 5.6 on the old plan, you can do more with 6.1. I don’t see that as a problem.
Once you are inserting models into non-creative (and software development is creative work) it is imperative to start creating your own small evals. Not just to check how different models perform in that equation, but to catch model drift over time.
5x is calculated in terms of API costs, nobody pays those extreme Anthropic prices outside enterprise users.
> like they didn't just double the cost for most users
Most users aren't on their $200 plan. Most users (including on the $200 plan) continue being subsidized.
Ignore value then. Anthropic still provides about 3x as many raw tokens. The 20x\200 plans provide about 10B for openai versus almost 30B for Anthropic.
Do you really think that Opus uses 3x as many tokens as Sol per task? I don't.
> Most users aren't on their $200 plan.
Fair enough. I should have said most heavy users.
> Most users (including on the $200 plan) continue being subsidized.
They continue to be fifty percent less subsidized, I guess.
Look, I don't fault them for trying to become profitable. I fault them for the weird weaselly business speak about cost per task. It's BS game playing and we all know it.
I'm honestly rooting for them, and hope it all works out. For now though I'm pretty salty that my bill jumped by 150% in one month. Gaslighting about made up metrics just makes me more angry.
We agonize over cost per task to the extreme for services/functionality built atop of APIs and can absolutely guarantee you that it is the real world metric that counts most right after “is the model good enough for the task at hand,” and furthermore that, yes, in fact, it is quite often that we see certain models/providers easily using 3x or more tokens on average for the same tasks (or even the exact same task) than a supposedly similarly matched model from another provider. Once you become intimately familiar with the thinking traces from specific models, you can preempt this to some extent with manipulations to the system prompt (essentially heading off entire thinking branches) but the nuking of reasoning traces in the name of preventing against distillation attacks has made this harder.
> Do you really think that Opus uses 3x as many tokens as Sol per task? I don't.
You don’t have to speculate. Pull up artificial analysis, pick either opus or sol and scroll down to total tokens and to total price. Opus (and sonnet) use 4x the tokens and 7x the dollars.
That difference alone removes the difference that semi analysis found.
Fair point. I just tried it, and I stand corrected. Claude uses almost exactly 3x as many tokens.
In the end Claude produced 2.25x more code, and used 3.01x more tokens (ignoring the cached system prompt, etc). It's answer was also much more substantial and feature rich - including many features I did not ask for.
I gave them both the prompt "Without searching the internet please write and test a python script that accepts a year (in any common format) as input and provides the date that easter fell or will fall on in that year. The output should be in discordian date format. You may not use any libraries."
The thing I found most interesting were the unit tests. Claude implemented the core code using both the Meeus–Jones–Butcher algorithm AND Gauss, but only used Gauss as a sanity check to ensure that it's primary code worked. Sol just validated that it matched some dates it could remember. That explains a good chunk of the almost doubled output, and it was a reasonable thing to do, though neither answer was "wrong."
Token usage looked like this (based on parsing it out of the jsonl, neither provider seems to make it easy to get real billing information per project/conversation.
I believe oh-my-pi will be able to ingest your Claude and codex conversations, count tokens, and then assign API-equivalent rates to them. May be worth looking into.
It's very likely this is a next checkpoint of GPT-6 "Sol", over-optimised for the web chat and offered to Codex subscribers as a "please go away" offering.
If this was half the price, I'd be very interested, but at 1/5/0.05? Feels like another panic release because they can't get Astra 6.1 out of the door...
But like the original release of Terra, it's still a weird middle child between the small-model-that-got-substantially-better (Luna) and the large-model-that-got-significiantly-cheaper (Sol). The TerminalBench 4.0 cost-per-task results are a bit confusing and I'm not sure why you would pick Terra over Sol in that case. Overall this release seems like a flex on Sonnet 5.5: a model which is having its own middle child problems between the godly Opus 5.5 and the surprisingly capable Haiku 5.5.
It is having improvements in other fields than coding while being more cost efficient.
If all I want is coding at a cheaper price, then deepseek is a good model for that.
I am more than okay to have a bifurcation of models good for coding and models good for everything other than coding. I strangely imagined it many years ago.
Everyone is suggesting OpenAI to release a model close to opus 5.5 yet it's releasing this. On one hand we can imagine that maybe their training cycle wasn't able to produce this whereas gpt 6.1 terra was made.
But on the other hand, I also now wish to see an OpenAI model which is good for just pure coding, like an absolute beast at it while being cheap (so the opposite of terra)
ironically, one of the few appreciations that I might give to OpenAI if they pushed this model to prod and are facing some pushback as developers like us want/only see coding models. They could've benchmaxed it and it would've been slightly irrelevant. I'd much rather see reality.
At the very least, this new model won't make things more expensive for anybody and would only make use-cases cheaper depending on what they are, so it's a fine deal for me.
I also wonder maybe this is why open source labs aren't targeting this space and why the only focus of AI labs is coding nowadays. Everything else just doesn't seem so worth it from investor, even end user, training perspective.
Maybe this is the reason why we have gotten almost an exponential growth in coding capabilities as that became the most (if not only) capability that truly really matters.
To not wish for discussion to go to a tangent, nonetheless this could also explain why our work as a software engineer is going through a slight crisis of meaning as these AI models have gotten impressive at coding.
As an (aspiring) developer (currently in 1st year college), I don't wish for AI to replace coding even more for me or to push it even more or to suck the joy out of it even further. So I am more than happy for these models to stay as they are and rather make advancements elseplace. Ironic because it is us developers who use AI as well and we want cheaper workflows and so we would also want models to be cheaper and better at coding. There are some fine lines of contradictions and irony in all of it. I'd much rather also have open-weight models of such terra-like models but I imagine that because of the reasons that I mentioned above, we are most likely gonna see it the most focused on coding but still open-weights is pretty close.
The more choices the better (though choice-fatigue is a real thing), I hope I was able to explain some of the nuance I feel about this particular launch.
Destroyed by Anthropics crazy decision to price it x5 after 100k context.
At least this my discovery when I did evals on my reviewers skills. 10k LOC is high end for high effort or around 150k context using opus. You can push towards 15k but that’s on max effort with about 400-500k ctx.
Fast models like haiku do not even need 1M ctx in my view
First we had:
> Luna (Small), Terra (Medium), Sol (Big).
Which was fairly smooth after you got used to their planets.
But then they added Astra as the "Bigger," which was mostly still self-coherent:
> Luna (Small), Terra (Medium), Sol (Big), Astra (Bigger)
But then things go confusing, they released Sol-6.1 which sometimes is superior to Astra, and discontinued Terra which made it look like this:
> Luna (Small), Sol-6.1 == Astra
But with Sol-6.1 costing the old Terra-6.0 price, while, as I said performing as good as the old "Bigger" model in many non-3D/UI situations.
Now they're re-introduced Terra, as a 6.1 release, making this even more confusing.
Did Microsoft's chief branding officer move over to OpenAI?
But minor credit to Anthropic, at least they've mostly stuck to their four model names (Haiku, Sonnet, Opus, and Fable). You mostly know where a model is MEANT to be positioned from the name alone. OpenAI cannot keep it consistent from release to release.
But, yes, currently Opus Vs. Fable has the same issue as Sol-6.1 Vs. Astra.
Hmm? They've been using the Luna/Terra/Sol/Astra scale for a few versions now and as far as I can tell it maps directly onto Haiku/Sonnet/Opus/Fable, in terms of the kind of thing it's meant to convey (if not in terms of the scale of differences).
Both companies have screwed up their naming royally. Maybe that's intentional - I'm sure it doesn't hurt Anthropic to have people using Fable a bit longer than they need to and racking up a huge bill in the process.
It's like iPhone 17 being faster than iPhone 16 Pro. Because it's newer and benefits from advances in chip architecture, despite being a lower model tier. Sometimes different iPhone models are released at different times.
I think the names are more of a reference to price. Luna is always cheaper than terra which is cheaper than Sol which is cheaper than Astra (for the same model number).
Per token, yes. But per task it's not even necessarily the case depending on how much thinking is needed.
But the model names are linear by price.
Yes, a more modern smaller model can beat an older bigger model at some tasks, that's what can be confusing.
“Test them all”
The lineup of models is still the same.
There are four models: luna, terra, sol, and astra.
Pricing increases for each level. This new 6.1 Terra is 50% cheaper than Sol 6.1.
They are lowering prices for newer releases, so yes if you compare Sol 6.1 to 5.6 Terra it won't look right, but why are you doing that? Look at the same release version for comparable numbers.
Each model improves in successive releases but they maintain pricing and capability coherency within a release version.
Yes, Sol 6.1 is supposed to rival Astra for some work, which is what you'd expect from a later versioned release model against a prior release bigger model.
Assuming they release Astra 6.1 at some point, I expect it'll be better than Sol 6.1, at a higher price.
I don't understand what people are asking for here? No reductions in pricing? Different model names on every version release? Not releasing any models for a version number until they're all ready?
Given the operations of the model are opaque, can you see how someone (most people?) might be confused?
After that you just have model "strength" and version number. But of course strength is only comparable at equal version number and the two axes are almost parallel.
Anthropic also does this, perhaps to a lesser extent.
I see a ton of criticism and not a single person coming up with anything better, unless it's to take away options, like removing effort level.
What about buying Intel CPUs? A car? Electrical components?
If you're not just using this at a consumer level to ask about cooking recipes (and if that's your use case, you really don't care about API pricing, or any of this, and can just use the default), and are using it professionally, then this seems like a low bar of effort to understand what you're buying.
I do grant you the relationship between model and effort level has an unclear relationship, but there are also tradeoffs in most complex technical products that are not always clear; this is exacerbated by the fact that this product is stochastic.
Why are engineers using this as a professional tool expecting this to be bought like a consumer gadget and then blaming the maker?
Why is it ok for engineers to have confusing names and ways to interact with something?
How am I supposed to decide between these options: https://imgur.com/a/bXHWowL
How am I supposed to keep up with the changes every few months?
Are you presenting this as evidence that OpenAI's naming scheme is bad?
If you don't want to spend the time to know the difference between all these different suppliers and their models, then why are you using whatever this program is that gives you so many choices, most of which are bad? Why is this OpenAI's fault?
No worries, you can always ask a AI.
They should have a guide saying "Terra is best at technical documentation and communication, Astra is best at large software projects and research, Sol is good for development" or something like that.
This morning I asked GPT which models and settings I should use to inspect my VM dashboard with computer control. It said: - GPT-6 Astra at Medium - GPT-6.1 Sol at Medium (alt) - use Luna only when the route was already known.
But I had already completed the task twice with Luna and it was 5.6! It navigated through a few pages of my dashboard, pulled up the 30-day usage chart, analyzed my CPU and RAM, and explained why I didn’t need an upgrade. I’d give Luna 5.6 an A on this task.
Maybe OpenAI should give us a “Tiptronic” selector: recommend the model and reasoning level based on the prompt, while still letting users override it.
What are you talking about? If it's for some slop you vibe check it. If it's for a serious effort you write an eval. Same as it has been for the last 3 years.
Let's say I have some task that Luna 6.1 cannot do, but Terra 6.1 can. So I build my product around that. In a month when Luna 6.2 is released, then I have to figure out whether it's worth to evaluate and swap or just wait for Terra 6.2.
They released gpt-6-sol (alongside astra and luna but no terra) and 7 days later released gpt-6.1-sol making gpt-6-sol basically DOA before any real adoption. Everyone knows it wasn’t just better RL/post-training, it was a completely different model underneath. Now they randomly reintroduce terra.
I’m sorry, but if you’re not even a tiny bit befuddled by this then you must be taking whatever OpenAI is having.
If you scale a model down, or switch out attention with linear attention, or introduce KV-compression, etc., you don't really know how it will affect model performance before you've trained the model.
And once you've spent millions training it, you might as well make it available if it's pareto optimal for some tasks. It also gives you more insight seeing how people use it in practice.
You are also rolling the training of your models by size and the lat est small model can sometimes beat an older model.
For a while we thought you just train the big model first then distill smaller models from it but since we've learnt the importance of on policy RL (basically the way a small model figures out a problem is very diff from a big model so it's best to RL optimize each model distinctly)
And some variants (e.g., the medium one) being skipped or delayed for a certain generation doesn't really change the overall size branding.
It's nice that they have Terra which seems to fill a void that's been needed since Sol 6 focused on cost (at the expense of quality), Sol 6.1 got a huge boost in quality, but became unusably slow unless you paid Astra prices - at which point, why not just use Astra?
Now at least we have a viable other option again.
In general, LLM performance is incredibly multidimensional. There's no guarantee of a fixed ordering where the model that is best at math is also the model that's best at jokes. Despite that complexity, much of the variation in performance is explained by just a single variable: model size. In general, models that are bigger will (a) be more performant / reliable, (b) cost more per token, and (c) have better recall of esoteric facts. As you note, our models are ordered:
Luna < Terra < Sol < Astra
Even today, I'd recommend this as a general starting point for picking models. If you care about quality only, pick Astra. If you're a bit price sensitive, pick Sol. If you're very price sensitive, pick Terra or Luna.
Of course, model size doesn't capture all variation. Models that are trained later might have new data or new techniques or new architectures that earlier models lack. You can see this with models from all providers - Sonnet 5.5 outperforms Fable 5.1 in ways, and Gemini 3.8 Flash outperforms 3.1 Pro in ways.
In the case of GPT-6.1 Terra, I'd say Terra is probably better than Astra and Sol at technical communication, similar on vision and computer use (but much faster!), and a bit worse on most other things. For API developers, it's a nice midrange option if you want something cheaper than Sol but a lot smarter than Luna (especially for vision / computer use / communication). For subscribers, it's not a bombshell release - if you're hitting usage limits, maybe it helps you go a bit further, and if you want clearer communication or faster computer use, it can be a nice option to select.
If you have suggestions on how to name our products better, I'm all ears. Our intention is to make things clear and simple, even if we miss the mark sometimes.
Unfortunately, a lot of the complexity is somewhat irreducible and due to to the actual complexity of the products. Scaling model size is different from scaling reasoning effort which is different from scaling multi-agent count. So if we just swept all of those under a single name of "more IQ", it would make the naming simpler but the experience much worse, as important details would be obfuscated from end users.
> Despite that complexity, much of the variation in performance is explained by just a single variable: model size.
please release model size :p
For some reason, this is just hilarious to me. Idk if you're being serious or if this is passive aggressive but the thought of OAI actually taking advice from a random HNer and basing their naming scheme off that is peak.
Where else would you go? I'm honestly scratching my head thinking of someplace that wouldn't run you out on a rail, but that would give serious feedback. Kernel mailing lists are out, maybe there's something like that which I can't remember off the top of my head, but...
Also, nomenclature is incredibly hard. There are only two hard problems in computer science: cache expiration and naming things.
The primary users of any API are the designers and engineers of the applications that use it. How many laypeople know what Postgresql is? How many of the applications they use run on it?
- Understanding how effort and class interact (Is a high effort terra above a low effort Sol? Are larger models mostly always better regardless of effort?)
- understanding when (if ever) a new model tends to be better than an older but higher class one. Is an old astra better than a new terra?
If increasing effort and increasing model class produce improved results in a different way, I miss making it explicit in which way. In other words: What kind of issue warrants going up in effort? what kind of issue warrants going up in model class? What kind of issue is unlikely to be solved by either?
Oh, easy:
https://deepswe.datacurve.ai/
Artificial Analysis has their own curves which are pretty similar.