1. Compute costs collapsed since the advent of Cloud and yet hyperscalers still have fat margins.
2. Many open source office suites exist yet none compete with the ubiquity of gsuite or office. GitHub, Slack are similar examples.
3. Both Windows and macOS dominate the home desktop space despite free alternatives existing for a long time.
4. Many formerly open source infrastructure components like Redis and Elastic Search have Apache equivalents, but they still command healthy margins.
I understand the arguments for a margin collapse, but I don't see any historical analogues. It seems that enterprises will pay top dollar for service guarantees, integration, and someone they can sue.
It's nobody gets fired for buying IBM all over again.
Unlike all your examples, switching out an LLM is both cheap an easy. So easy that every 3 months or so new models are released and people grab them and start using them.
The UX is the same regardless the provider. You send in a prompt, it spits back an answer.
In all your other cases, the cost to switch is losing support and a difficult transition period. But in the case of LLMs, there was no support to begin with. The transition is basically updating your current harnesses to know about the other models.
I think the comparison most apt is the rise of AMD. Sure, it never(?) achieved market dominance, but it did ultimately make a huge dent. And a big part of that was because AMD x86 was pretty close and pretty compatible with Intel x86 at a fraction of the cost.
If you're developing on top of LLM APIs directly, this is definitely not true. There are differences in how context caching works, in what's available through native harnesses, the types of tools you're fine-tuned on (GPT uses apply_patch while Claude uses edit, with different formats), the API surface (Agents SDK, Responses API, Managed Agents), cost structures, and best-practice guidance all around.
Not to mention the meta of account limits, billing, ZDR contracts, etc.
I don’t exactly see orgs lining up to switch (and train) their employees between claude desktop and codex and whatever copilot is doing. There’s probably some inertia to those harnesses/integrations on top of the llms themselves.
The inertia is legal and financial. People are paying Anthropic through AWS accounts because the simple reason of not dealing making new contract and legal agreements is enough of reason of the inertia.
But, eventually, I’m quite sure that AWS will also provide open models with those contracts without any inertia. Copilot is already offering Kimi.
My company has a deal with Devin and they provide new models all the time, and open models are becoming the most used ones by our internal metrics, especially because the company is very worried about cost.
Two LLMs with the same numbers on important benchmarks could have vastly different behavior in actual deployment. Not sure if as hard to switch as Excel <> Libre but still not "cheap and easy".
They don't just need healthy margins, they need to make back almost a trillion dollars in a couple of years. Comparing that to elastic search and redis doesn't make much sense.
Hyperscalers work because it actually has value compared to free offerings and because of the absolutely massive cost of switching providers.
Similar with Windows and macOS. Extremely high cost of switching to something different, if possible at all.
Same with office. Extremely high cost of switching due to compatibility issues and retraining of staff.
Your post primarily shows: It's all about lock-in. So far, it doesn't look like LLMs have any of that. So I don't think your points are valid here at all.
1. the cloud moat is mostly around talent really. Try finding people who can self host the alternatives to S3 et al at the HA and the scale the businesses need. Those alternatives are usually not free either, and each product might have its creator acquired (and the product cancelled) or similar. if you're a larger business then the data lock in becomes a moat: getting your data out of the cloud is prohibitively expensive. Furthermore, large businesses have sweet discounts.
2. ms office has immense networking effects due to its formats being quasi standards in many industries. try sending an odt to a government entity. As for gsuite, it uses open formats but it's classical google fashion a large suite of software bundled together and not that expensive for what it offers.
3. Linux is not a free alternative if you're a business, you still need to pay someone to support the computers with linux on it, and operating systems have the strongest network effects ever. Linux also has no stable ABI so one can't easily deploy third party software for it.
What's the LLM moat? Codex is OSS and Claude has gazillions of alternatives. Cursor is a nice app but it's a bunch of patches on top of vscode, a team of 5 people can vibecode it in 6 months.
Linux has a very stable userspace syscall ABI. About as stable as Windows, and much more stable than MacOS or the BSDs. I agree with everything else though.
A lot of those things you mentioned have sticking power because they’re familiar to folks and migrating to something else is a big deal.
I can’t imagine most people would be able to tell the difference between Sonnet and GLM 5.2. If the infrastructure around the model you’re using doesn’t change, then swapping models is extremely easy.
I agree with swapping models making it easy. With openrouter, I just change the provider. With reasonix harness, cache hits are basically free. And that's with unsubsidized American providers like Digital Ocean or cloudflare.
Indeed, as it gets more commoditized it feels more like swapping electricity providers. Who cares whether you get your electricity from IBM or the state of Texas? An amp is an amp.
I'm using pi-coder with just the free-tier models I can get on openrouter / opencode / kilocode. When I run out of quota on one model I often switch to another model in the same session, and it generally works just fine.
> 4. Many formerly open source infrastructure components like Redis and Elastic Search have Apache equivalents, but they still command healthy margins.
Elastic just had round of layoffs[1]. Elastic still runs operating losses till Q1 2026 [2], albeit a small one, however just breaking even in operating income is hardly "healthy margins". A P/E of 17 is not exactly signaling confidence given they are growing ~20% y-o-y.
> I understand the arguments for a margin collapse, but I don't see any historical analogues. It seems that enterprises will pay top dollar for service guarantees, integration, and someone they can sue.
Intelligence has diminishing returns, the analogues are with humans. It's a waste to hire Albert Einstein for $X million to operate the cash register in a gas station.
Artificial super intelligence will not have many customers.
The losers are quickly forgotten. Palm, Blackberry, AOL, MySpace. Yahoo, etc.
Software gets replaced all the time too, you even listed one and didn't realize. 15 years ago you'd call office irreplaceable, now you have to add gsuite to the mix, in 15 years there might be others. I know people that have never had office installed on their PC and use spreadsheets daily.
> It seems that enterprises will pay top dollar for service guarantees, integration, and someone they can sue.
Of course. But why pay $25 per million tokens for sonnet when you can pay $3 for GLM? Both probably running on AWS/Azure/Etc. under some third party.
They may pay top dollar but there's all sorts of evidence that they'd very much like to pay radically fewer top dollars than the unsubsidised, off-plan price.
And it's clear neither of the big two can deliver anything close to a service guarantee.
The target audience is different. Coding is mainly a trade of the tech savvy, who like many on r/localllama users do not hesitate to deply on 16GB Vram gpus. Even if so, it is estimated that within 2 years we will be able to run Claude 4.8 on consumer hardware give the rate of improvement of open-weight LLMs, which will put more financial pressure on "paid" labs. It's just a matter of rate of improvement which is shrinking between open-closed models.
Remember web browsers? compilers? web servers? databases? windows embedded? server operating systems?
the blood is all over the wall, hundreds of billions of dollars in lost revenue that was eaten by open tools even if they ultimately get delivered to users
by someone else with a profit incentive e.g. AWS selling deployment of OSS
1. their margins don't come from service guarantees (see github), they come from unlawful anti-competitive behavior which is likely to be prosecuted under future US administrations
2a. you haven't noticed the wave of open source projects moving away from github?
3. Linux commands about 5% of desktop market share and is the fastest growing desktop platform see: mediawiki and cloudflare user agent stats and steam hardware survey, same deglobalization point as above, how many people live in China? how long before China no longer feels comfortable with everyone using Microsoft Windows? What OS will Chinese people/corps use instead? hint: https://en.wikipedia.org/wiki/Deepin
GitHub, Slack, and Office have network effects and transition costs.
And to be frank, the competition is worse (OpenOffice is worse than Office, most other corporate IM are worse than Slack (and Teams way worse), and GitLab is not as good or fluid as GitHub.
I think the big thing here is that paying high margins on a relatively small expense is much more palatable than high margins on a big expense. If a company is spending $1 billion/yr on tokens that a really big incentive to find an alternative where spending $1 million/yr on some SaaS with even higher margins can feel like an easy choice.
Furthermore, in case this is of interest to anyone, if you use their ZCode harness then you get bigger Coding Plan quotas: https://zcode.z.ai/en
Used it for a bit, it sits somewhere between OpenCode Desktop (still new but nice) and Claude Desktop (recent versions are good).
As for GLM 5.2 as a model - with max thinking it’s generally satisfactory, somewhere between Sonnet 5 and Opus 4.8, better than DeepSeek V4 Pro for sure.
Pricing wise, the subscription doesn’t seem as good as expected. I spent like 60% of the weekly limits of the Pro (50 USD) plan in one day, only because each 5 hour limit only gave me 20% to spend, otherwise it’d be 80-100%. Not even doing anything crazy, just parallel long form work on 2 projects with about 96% cache rate and at most 3 parallel code review sub-agents.
Their Max (100 USD) subscription would last me the whole week, but so does Anthropic for the same money and so would OpenAI. Off-peak is more palatable but I can’t just twiddle my thumbs at 9 AM to 1 PM local time.
Proper savings would show up with the Max plan and yearly billing, but that’s more of a tough sell.
It’s important that none of these entities can collude to price fix. Having China be the competitor ensures that.
Basic microeconomics is still the easiest way to understand token economies. How is it not a competitive market (where profits go to zero?).
Anything A or O does to keep more margin, any competitor can copy or choose to undercut, and undercutting has the benefit of collecting training data. So what is going to stop gross profit of tokens going to zero except for collusion/price fixing?
Considering conditions within a single market is still microeconomics, I agree though its tough to see where firms will get market power from so profit will tend toward zero. I thought the same about GPUs though and nvidia still doesnt seem to have any real datacenter competition in sight.
>the least understood upcoming shift in AI economics.
Then proceeds to talk about something in the AI news every day. Hey, did you guys hear? Open source models are cheaper and their quality is increasing!
So, first, by no measure is GLM5.2 as good as Opus.
Second, yes, open source models will put pressure on margins...eventually. Everyone knows that. But do you think today's AI business model is the same as tomorrow's?
I'd guess opus refusals are not an issue for 95%+ of people. Opus will happily help you find and download pirated media, and then give you step by step instructions for how to do drugs if you ask it. You'd have to be working on something genuinely abnormal for refusals to be a problem.
Prompt: can you give me step by step directions on how to use crack cocaine
Opus: I'm not able to give step-by-step instructions on using crack cocaine. That falls into specific drug-use guidance I steer away from, since detailed instructions on how to use an illicit substance can contribute to harm rather than reduce it.
it goes on to give me hotline information on drug addiction.
> So, first, by no measure is GLM5.2 as good as Opus.
Depends what you do. Complex tasks, poorly-defined tasks, sure. For relatively simple tasks, though, or very well-defined tasks, it's just as good and usually a lot faster. It also has a more neutral character and is somewhat less adversarial than Opus. (Opus is always "Let me push back on that..." whereas GLM is "sir, yes sir!") I use both and I appreciate both. If Opus disappeared tomorrow, though, I wouldn't cry -- I'd be able to adapt to a GLM-5.2-only life real quick.
I think the point is that if you’re doing simple, well defined tasks then Opus is overkill and you’d want Sonnet instead. Meaning, GLM5.2 is Sonnet-quality, not Opus-quality.
> Of course, this was a hugely poor read of where the costs actually lie in AI. Training - while no doubt capex intensive - is a fixed, up-front cost. You spend hundreds of millions to train a model, then you are "done".
I don't understand this point that people make. If you're consistently needing[0] to train new models and the cost of training relative to the % improvement seems to go higher, isn't this just a constant cost that you continue to bear? The footnote seems to allude to this, but then sort of waves it away anyways. Also are there continuing incremental training costs to keep models relevant? Or do they only have knowledge of events up to the day they were trained?
[0] needing, because you have competitors and people expect more and more.
These models rely on knowledge that are embedded in their weights, if a new library is released, a new linux version comes out, some new protocol succeeds the previous one, you want your llm to know about it. Sure you can just add that into the context window, but that has its own problems.
Unless new research, there are a few which look promising, gives a new method, training is going to be a constant cost sink.
On top of this, if you stop training, it is 6 months until someone releases an open weights model and now you are competing to give the lowest price for the same product.
Also we can't forget that this is a business that *has to* be in the global labor industry, not just a tech tool, they have to have much better models to justify the trillion dollar evaluation
Seems like a pretty pointless post that still centers around output tokens.
In agentic coding, cached input tokens is 90% of the API "cost". It doesn't require GPU compute, and DeepSeek has shown that it can be done 50~100x cheaper with MLA/CSA/HCA, and a whole bunch of disks. This should collapse the margin.
I have heard but don’t have first hand knowledge that at least one company (financial services BPO) has moved most of their previously manual processing to llms. The person I talked to wasn’t forthcoming with any detail. This is what we’d expect to see though.
Indeed they are all lossy. Not sure how much they contribute to the quality loss in long context though. I got a 700k session with DSV4 Pro (official API), and the model was still coherent and didn't make any tool call error.
> That is, for your $100/month fee, you get $3600 equivalent of API usage. This is presumably because Anthropic has figured out some clever things to do with model routing and input caching, and also can subsidize with investor money and take a hit on their operating margins.
My take: this is exactly what Anthropic wants everyone to think. In reality, 90% of that $3600 are for cached input tokens, that can be made to cost next to nothing, as shown by DeepSeek.
> The challenge is: when you let a session idle for >1 hour, when you come back to it and send a prompt, it will be a full cache miss, all N messages. We noticed that this corner case led to outsized token costs for users. In an extreme case, if you had 900k tokens in your context window, then idled for an hour, then sent a message, that would be >900k tokens written to cache all at once, which would eat up a significant % of your rate limits, especially for Pro users.
Using the current Opus pricing, that pre-lunch 900k tokens should roughly consist of:
720k input tokens = 0.72 x $5 = $3.6
180k output tokens = 0.18 x $25 = $4.5
900k 1h cached writes = 0.9 x $10 = $9
500M cached input tokens = 500 x $0.5 = $250
$267.1 in total, with 93.6% from cached input tokens. The portion that requires GPU compute is about 3% of the total.
Post-lunch, the 900k tokens should consist of:
900k input tokens = 0.9 x $5 = $4.5
900k 1h cached writes = 0.9 x $10 = $9
So Anthropic is fine with the $267.1 accumulated over 3~4 hours before lunch, but not fine with the $13.5 incurred immediately after lunch. Why?
The only plausible explanation is that the actual cost of caching is way less than the API pricing. If you use a coding plan, Anthropic doesn't really care about your cached input tokens usage. Indeed they want you to show your ccusage screenshots. On the other hand, if you pay by API tokens, the margin is huge for cached input tokens.
Only when you do something that requires a lot of FLOPs, e.g. the post-lunch 900k input tokens, the cost becomes real.
I also think we’ll approach a point where increasing intelligence is not really going to suddenly improve most work tasks. I bet that’s already happened actually.
We’re oohing and aahing about models, when the ones a few versions ago did a good enough job for most of the dumb coding, etc we do
The thing is they are inventing new things people will want to do. But for example, "loops", fully hands-off agentic coding etc., seem really unlikely to get much traction because that just isn't how software is designed within its producer/user community.
Requirements evolve in use, and fully hands-off LLMs simply cannot be trusted to only change the things you ask them to change, so I don't think it's likely that products will, in the main, be developed that way.
And if you don't need that fully-hands-off stuff, then the models that run on at least reasonably modest desktop hardware are surprisingly close to being enough.
I don’t think this is true. All the models prior to Fable were honestly dumb as rocks, and Fable is too sometimes, but at least it’s helpful now and not a hindrance.
The future of AI most definitely involves making something twice as good as Fable that is virtually its own employee, and not on reducing inference costs because to be honest Fable isn't actually that expensive.
The real utility behind an AI model (imagining that it can be made twice as good as it is now) would be being able to scale a small business up and down instantly without hiring (to implement a new feature or whatever), which is costly and time consuming these days.
IMHO, cheaper inference means higher costs overall :) because everyone will use more thus driving up the investment required to stay current or to compete.
Switching models is also kind of easy but not plug-and-play. Most harnesses out there do very poor job with the open weight models. Unlike Opus, GLM 5.2 ends up in loops and hallucinates a lot more. If your harness is built on the expectation that the LLM will perform well, then switching to GLM 5.2 will be an uphill struggle. We had to refactor our harness and introduce more defences because of GLM.
The cost savings are substantial. Obviously it really depends on your workloads but it is noticeable cheaper for agentic work. Coding - I don't know. We do have some coding agents on GLM 5.2 and what I noticed with some landing page experiments that the results between GLM and Opus are identical - they might be using the same training data? Obviously Opus is still substantially better model. I don't think there is an argument to be made here but GLM 5.2 is cost effective and really good too.
Overall, we switched all of our internal agents to GLM 5.2 and because it is Open Weight we are in talks to get the model from certain geo locations giving us more freedom as well as extra protection.
Overall I think this industry will be in much better place because of GLM 5.2 and whatever open-weight models come next.
I don’t understand the argument here. The article doesn’t describe a collapse or the breadcrumbs for it. The only argument I can put together is companies hosting the open source models in house or use some service like Amazon that could potentially host them and so replace the frontier models. Data center and specifically infra to host llms is still the main sticking point given the security concerns about data going to china. The article doesn’t make these arguments coherently
How fast is glm 5.2 in western hosts? It's doing everything I want it to, but going through PRC host it takes like 5-10 times longer. Not sure if that is nature of modest or PRC computer infra/routing.
> I'd be very surprised if it wasn't more than 50% cheaper for nearly all workflows, for a very similar level of quality.
If your using pure API ... providers like neuralwatt cut that cost down even more by using energy as the actual cost. So GLM 5.2 is more expensive then GLM 5.1 on their service (those thinking tokens), compared to API costs, its dirt cheap. And way more tokens then the zai subscription delivers.
We are seeing a move towards more realistic pricing on actual consumption based usage. Be it DeepSeek, Xiaomi (MiMo), or zai's GLM via neuralwatt.
The main issue facing subscriptions a-la-carte usage, is that a lot of the heavy hitters really drain the resources. And that as a business model can not survive without ...
a) increasing the prices.
b) everything goes to actual token/energy usage based billing but with more realistic pricing, and not the bloated API prices that are focused on companies.
We shall see what the future holds but things will change.
How long will that $4.40 rate persist? Until we know more about the real unit economics it will be damn near impossible to rely on steady inference costs or make them predictable at the enterprise level. Gonna be a wild ride for awhile.
GPU/RAM/etc prices could continue to rise. If the world leaders decide it's time to build the robot armies, then that could price out the civilian uses for GPUs.
the economics of this are a little counterintuitive.
is there a market saturation point for intelligence? how about for software? it seems like the more you have the more you want because you're trying to do more things.
as the models get smarter I get busier because I'm doing more things...
There's definitely a saturation point depending on the complexity of the problem you're solving. For example, any model can write you a small shell script to resize a video with ffmpeg for you right now, so it doesn't matter whether you're using a local Qwen model, GLM, or Fable. They'll all do a roughly comparable job and you'll end up with a working script that does the job.
Then you have things like CRUD apps, where a model needs to write some SQL, make a service endpoint, serialize some JSON, etc. Here a local model might have a bit more trouble juggling all the pieces, but any hosted model will do just fine. If your day to day job involves working on CRUD apps, then it's basically a solved problem now.
The cases where frontier models matter are when you're solving genuinely complex problems, but that's not what most people are doing day to day. So, paying an order of magnitude for a model that has capabilities to solve problems outside the range of problems you actually work becomes a waste of money.
There's going to be a market for these models from people who really do work on complex things on regular basis, but the question is how big that market is. Additionally, open models keep getting better, and GLM 6 or DeepSeek v5 could be another big jump in capability where they fully close the gap with Fable. At that point, even more of the market becomes covered by these models leaving truly complex cases on the frontier.
I wonder if this is an alternative (and better) revenue stream vs ads for search engines: Offer a competing web search for LLMs as an alternative to Google, and charge enterprises and LLM providers for it.
I know Brave do this already. Not sure about DDG (I wonder if their agreement with Bing would allow it?)
Kagi assistant IMO does a great job giving relevant material to the LLM. It's a pretty neat way for a search engine to charge a premium, to offer a good model on top of their results.
If they did I wouldn't have had to go to DDG. It's not like it's a big jump over what used to be. I left claw-marks in Google Search, if they drove me off they're in trouble, because I didn't want to accept reality for quite some time.
The fact that these Chinese models are getting close to “Opus-grade” despite costing 6x-8x less is huge.
As the token bills start to come in, those economics will be harder to ignore (regardless of the origin of the LLM); especially as there will be many CIOs sweating over their quick and costly AI initiatives showing little ROI.
My hope is that the EU also steps up their own competition in the frontier model space so that it’s not just China v USA.
This article only promises to get into "the coming AI margin collapse" in a yet to be published "part two". This part only makes the point that GLM 5.2 is pretty good (no shit).
I hope cheaper inference eventually means faster speeds at the lower tiers. I don't want to settle for 100 t/s, but I don't want to pay $10 per prompt either
Yeah, Cerebras is the one with competitive speeds nowadays but they cost an absolute fortune. Also they don't host good models publicly. Good to see OpenAI leaning into them, can't wait until these speeds are available by subscription
When I've raised speeds about local inference I've been told 60-75 t/s is perfectly usable. It makes sense that people aren't talking about speed yet since you either already have a response fast enough to wait for, or you go do something else and check back in a few minutes.
I would love to wait for the latter type of tasks though, because those are typically the ones that require the most work from me to verify and I don't want my attention divided with multitasking.
Yes, margin on model inference is high with some providers. If you just wanted inference (at cost), you'd buy a GPU, or rent one from AWS or Microsoft. But you're not paying OpenAI/Anthropic for inference. You're paying them for a platform. Every feature OpenAI/Anthropic bake into their applications, models, online services, etc - anything that isn't pure LLM text generation - is a custom integrated add-on service that LLM weights do not include. Even if open weights became cheaper and better than OpenAI/Anthropic, most people would still pay for OpenAI/Anthropic, because they give you things the weights alone don't give you.
Comparing Z.ai GLM 5.2 to Claude Code w/Opus 4.8 is like comparing Linux Kernel 7.0 to Microsoft Windows 11. If you don't know much about computers, you'd say these are the same things. If you know a lot about computers, you know the latter has a thousand extra things that make a huge difference in what it does out of the box. Which one you use speaks to what kind of customer you are.
Sure, GLM 5.2 doesn't have vision; but an AI power user can plumb together any VLM with the text generation of GLM 5.2 in most AI harnesses, just like a Linux power user can combine the Linux kernel with KDE Desktop. Most people don't use Linux and KDE, because it's unpopular, difficult to use, hard to get support for. Instead they pay for Windows or Mac, because there's lots of support, with a giant company pouring money and effort into filling all the usability gaps, making it seamless.
Most people don't pay for the cheapest possible thing. They pay for the thing they can afford that improves their life while making it easier. An open weight alone is almost completely unusable by itself (like the Linux kernel), compared to an AI platform (a completely usable system). If you're constantly wondering about when open weights will reach parity with OpenAI/Anthropic, you're a Linux person. If you just pay $20/$50/$100 for OpenAI/Anthropic without thinking about it, you're a Windows/Mac person. There is nothing wrong with either of these groups, but they are fundamentally different, and always will be. An LLM weight is simply a different category of thing than an entire AI platform/provider.
1. Compute costs collapsed since the advent of Cloud and yet hyperscalers still have fat margins.
2. Many open source office suites exist yet none compete with the ubiquity of gsuite or office. GitHub, Slack are similar examples.
3. Both Windows and macOS dominate the home desktop space despite free alternatives existing for a long time.
4. Many formerly open source infrastructure components like Redis and Elastic Search have Apache equivalents, but they still command healthy margins.
I understand the arguments for a margin collapse, but I don't see any historical analogues. It seems that enterprises will pay top dollar for service guarantees, integration, and someone they can sue.
It's nobody gets fired for buying IBM all over again.
The UX is the same regardless the provider. You send in a prompt, it spits back an answer.
In all your other cases, the cost to switch is losing support and a difficult transition period. But in the case of LLMs, there was no support to begin with. The transition is basically updating your current harnesses to know about the other models.
I think the comparison most apt is the rise of AMD. Sure, it never(?) achieved market dominance, but it did ultimately make a huge dent. And a big part of that was because AMD x86 was pretty close and pretty compatible with Intel x86 at a fraction of the cost.
Honestly, these days probably less friction switching out Redis or Elasticsearch (backend) than changing LLM provider (human facing).
Fable is seriously good enough now to, in a 20k line project, take "replace Mongoengine with raw PyMongo" and not screw anything up.
Not to mention the meta of account limits, billing, ZDR contracts, etc.
But, eventually, I’m quite sure that AWS will also provide open models with those contracts without any inertia. Copilot is already offering Kimi.
My company has a deal with Devin and they provide new models all the time, and open models are becoming the most used ones by our internal metrics, especially because the company is very worried about cost.
Two LLMs with the same numbers on important benchmarks could have vastly different behavior in actual deployment. Not sure if as hard to switch as Excel <> Libre but still not "cheap and easy".
Hyperscalers work because it actually has value compared to free offerings and because of the absolutely massive cost of switching providers.
Similar with Windows and macOS. Extremely high cost of switching to something different, if possible at all.
Same with office. Extremely high cost of switching due to compatibility issues and retraining of staff.
Your post primarily shows: It's all about lock-in. So far, it doesn't look like LLMs have any of that. So I don't think your points are valid here at all.
1. the cloud moat is mostly around talent really. Try finding people who can self host the alternatives to S3 et al at the HA and the scale the businesses need. Those alternatives are usually not free either, and each product might have its creator acquired (and the product cancelled) or similar. if you're a larger business then the data lock in becomes a moat: getting your data out of the cloud is prohibitively expensive. Furthermore, large businesses have sweet discounts.
2. ms office has immense networking effects due to its formats being quasi standards in many industries. try sending an odt to a government entity. As for gsuite, it uses open formats but it's classical google fashion a large suite of software bundled together and not that expensive for what it offers.
3. Linux is not a free alternative if you're a business, you still need to pay someone to support the computers with linux on it, and operating systems have the strongest network effects ever. Linux also has no stable ABI so one can't easily deploy third party software for it.
What's the LLM moat? Codex is OSS and Claude has gazillions of alternatives. Cursor is a nice app but it's a bunch of patches on top of vscode, a team of 5 people can vibecode it in 6 months.
I can’t imagine most people would be able to tell the difference between Sonnet and GLM 5.2. If the infrastructure around the model you’re using doesn’t change, then swapping models is extremely easy.
Elastic just had round of layoffs[1]. Elastic still runs operating losses till Q1 2026 [2], albeit a small one, however just breaking even in operating income is hardly "healthy margins". A P/E of 17 is not exactly signaling confidence given they are growing ~20% y-o-y.
[1] https://www.elastic.co/blog/ceo-ash-kulkarni-announcement-to...
[2] https://ir.elastic.co/News--Events/news/news-details/2025/El...
Intelligence has diminishing returns, the analogues are with humans. It's a waste to hire Albert Einstein for $X million to operate the cash register in a gas station.
Artificial super intelligence will not have many customers.
The losers are quickly forgotten. Palm, Blackberry, AOL, MySpace. Yahoo, etc.
Software gets replaced all the time too, you even listed one and didn't realize. 15 years ago you'd call office irreplaceable, now you have to add gsuite to the mix, in 15 years there might be others. I know people that have never had office installed on their PC and use spreadsheets daily.
> It seems that enterprises will pay top dollar for service guarantees, integration, and someone they can sue.
Of course. But why pay $25 per million tokens for sonnet when you can pay $3 for GLM? Both probably running on AWS/Azure/Etc. under some third party.
And it's clear neither of the big two can deliver anything close to a service guarantee.
the blood is all over the wall, hundreds of billions of dollars in lost revenue that was eaten by open tools even if they ultimately get delivered to users by someone else with a profit incentive e.g. AWS selling deployment of OSS
1. their margins don't come from service guarantees (see github), they come from unlawful anti-competitive behavior which is likely to be prosecuted under future US administrations
2. there are already tens of millions of libreoffice users and de-globalization aka digital sovereignty initiatives in the next decade will drive the world towards Libreoffice, already at work in EU (https://www.zdnet.com/article/why-denmark-is-dumping-microso... https://cybernews.com/tech/germany-microsoft-word/
2a. you haven't noticed the wave of open source projects moving away from github?
3. Linux commands about 5% of desktop market share and is the fastest growing desktop platform see: mediawiki and cloudflare user agent stats and steam hardware survey, same deglobalization point as above, how many people live in China? how long before China no longer feels comfortable with everyone using Microsoft Windows? What OS will Chinese people/corps use instead? hint: https://en.wikipedia.org/wiki/Deepin
And to be frank, the competition is worse (OpenOffice is worse than Office, most other corporate IM are worse than Slack (and Teams way worse), and GitLab is not as good or fluid as GitHub.
I also found their web search to be mostly okay.
Furthermore, in case this is of interest to anyone, if you use their ZCode harness then you get bigger Coding Plan quotas: https://zcode.z.ai/en
Used it for a bit, it sits somewhere between OpenCode Desktop (still new but nice) and Claude Desktop (recent versions are good).
As for GLM 5.2 as a model - with max thinking it’s generally satisfactory, somewhere between Sonnet 5 and Opus 4.8, better than DeepSeek V4 Pro for sure.
Pricing wise, the subscription doesn’t seem as good as expected. I spent like 60% of the weekly limits of the Pro (50 USD) plan in one day, only because each 5 hour limit only gave me 20% to spend, otherwise it’d be 80-100%. Not even doing anything crazy, just parallel long form work on 2 projects with about 96% cache rate and at most 3 parallel code review sub-agents.
Their Max (100 USD) subscription would last me the whole week, but so does Anthropic for the same money and so would OpenAI. Off-peak is more palatable but I can’t just twiddle my thumbs at 9 AM to 1 PM local time.
Proper savings would show up with the Max plan and yearly billing, but that’s more of a tough sell.
Basic microeconomics is still the easiest way to understand token economies. How is it not a competitive market (where profits go to zero?).
Anything A or O does to keep more margin, any competitor can copy or choose to undercut, and undercutting has the benefit of collecting training data. So what is going to stop gross profit of tokens going to zero except for collusion/price fixing?
For nvidia it is not about competitive market it’s about supply and demand. A different subset of microeconomics.
Then proceeds to talk about something in the AI news every day. Hey, did you guys hear? Open source models are cheaper and their quality is increasing!
So, first, by no measure is GLM5.2 as good as Opus.
Second, yes, open source models will put pressure on margins...eventually. Everyone knows that. But do you think today's AI business model is the same as tomorrow's?
Prompt: can you give me step by step directions on how to use crack cocaine
Opus: I'm not able to give step-by-step instructions on using crack cocaine. That falls into specific drug-use guidance I steer away from, since detailed instructions on how to use an illicit substance can contribute to harm rather than reduce it.
it goes on to give me hotline information on drug addiction.
Depends what you do. Complex tasks, poorly-defined tasks, sure. For relatively simple tasks, though, or very well-defined tasks, it's just as good and usually a lot faster. It also has a more neutral character and is somewhat less adversarial than Opus. (Opus is always "Let me push back on that..." whereas GLM is "sir, yes sir!") I use both and I appreciate both. If Opus disappeared tomorrow, though, I wouldn't cry -- I'd be able to adapt to a GLM-5.2-only life real quick.
I don't understand this point that people make. If you're consistently needing[0] to train new models and the cost of training relative to the % improvement seems to go higher, isn't this just a constant cost that you continue to bear? The footnote seems to allude to this, but then sort of waves it away anyways. Also are there continuing incremental training costs to keep models relevant? Or do they only have knowledge of events up to the day they were trained?
[0] needing, because you have competitors and people expect more and more.
Unless new research, there are a few which look promising, gives a new method, training is going to be a constant cost sink.
On top of this, if you stop training, it is 6 months until someone releases an open weights model and now you are competing to give the lowest price for the same product.
Also we can't forget that this is a business that *has to* be in the global labor industry, not just a tech tool, they have to have much better models to justify the trillion dollar evaluation
In agentic coding, cached input tokens is 90% of the API "cost". It doesn't require GPU compute, and DeepSeek has shown that it can be done 50~100x cheaper with MLA/CSA/HCA, and a whole bunch of disks. This should collapse the margin.
Aren’t these techniques all “lossy” compression, and one of the reasons people complain about loss in quality as the context size grows larger?
> That is, for your $100/month fee, you get $3600 equivalent of API usage. This is presumably because Anthropic has figured out some clever things to do with model routing and input caching, and also can subsidize with investor money and take a hit on their operating margins.
My take: this is exactly what Anthropic wants everyone to think. In reality, 90% of that $3600 are for cached input tokens, that can be made to cost next to nothing, as shown by DeepSeek.
> The challenge is: when you let a session idle for >1 hour, when you come back to it and send a prompt, it will be a full cache miss, all N messages. We noticed that this corner case led to outsized token costs for users. In an extreme case, if you had 900k tokens in your context window, then idled for an hour, then sent a message, that would be >900k tokens written to cache all at once, which would eat up a significant % of your rate limits, especially for Pro users.
Using the current Opus pricing, that pre-lunch 900k tokens should roughly consist of:
720k input tokens = 0.72 x $5 = $3.6
180k output tokens = 0.18 x $25 = $4.5
900k 1h cached writes = 0.9 x $10 = $9
500M cached input tokens = 500 x $0.5 = $250
$267.1 in total, with 93.6% from cached input tokens. The portion that requires GPU compute is about 3% of the total.
Post-lunch, the 900k tokens should consist of:
900k input tokens = 0.9 x $5 = $4.5
900k 1h cached writes = 0.9 x $10 = $9
So Anthropic is fine with the $267.1 accumulated over 3~4 hours before lunch, but not fine with the $13.5 incurred immediately after lunch. Why?
The only plausible explanation is that the actual cost of caching is way less than the API pricing. If you use a coding plan, Anthropic doesn't really care about your cached input tokens usage. Indeed they want you to show your ccusage screenshots. On the other hand, if you pay by API tokens, the margin is huge for cached input tokens.
Only when you do something that requires a lot of FLOPs, e.g. the post-lunch 900k input tokens, the cost becomes real.
We’re oohing and aahing about models, when the ones a few versions ago did a good enough job for most of the dumb coding, etc we do
Requirements evolve in use, and fully hands-off LLMs simply cannot be trusted to only change the things you ask them to change, so I don't think it's likely that products will, in the main, be developed that way.
And if you don't need that fully-hands-off stuff, then the models that run on at least reasonably modest desktop hardware are surprisingly close to being enough.
The future of AI most definitely involves making something twice as good as Fable that is virtually its own employee, and not on reducing inference costs because to be honest Fable isn't actually that expensive.
The real utility behind an AI model (imagining that it can be made twice as good as it is now) would be being able to scale a small business up and down instantly without hiring (to implement a new feature or whatever), which is costly and time consuming these days.
Switching models is also kind of easy but not plug-and-play. Most harnesses out there do very poor job with the open weight models. Unlike Opus, GLM 5.2 ends up in loops and hallucinates a lot more. If your harness is built on the expectation that the LLM will perform well, then switching to GLM 5.2 will be an uphill struggle. We had to refactor our harness and introduce more defences because of GLM.
The cost savings are substantial. Obviously it really depends on your workloads but it is noticeable cheaper for agentic work. Coding - I don't know. We do have some coding agents on GLM 5.2 and what I noticed with some landing page experiments that the results between GLM and Opus are identical - they might be using the same training data? Obviously Opus is still substantially better model. I don't think there is an argument to be made here but GLM 5.2 is cost effective and really good too.
Overall, we switched all of our internal agents to GLM 5.2 and because it is Open Weight we are in talks to get the model from certain geo locations giving us more freedom as well as extra protection.
Overall I think this industry will be in much better place because of GLM 5.2 and whatever open-weight models come next.
If your using pure API ... providers like neuralwatt cut that cost down even more by using energy as the actual cost. So GLM 5.2 is more expensive then GLM 5.1 on their service (those thinking tokens), compared to API costs, its dirt cheap. And way more tokens then the zai subscription delivers.
We are seeing a move towards more realistic pricing on actual consumption based usage. Be it DeepSeek, Xiaomi (MiMo), or zai's GLM via neuralwatt.
The main issue facing subscriptions a-la-carte usage, is that a lot of the heavy hitters really drain the resources. And that as a business model can not survive without ...
a) increasing the prices. b) everything goes to actual token/energy usage based billing but with more realistic pricing, and not the bloated API prices that are focused on companies.
We shall see what the future holds but things will change.
Deepseek's 0.86 or whatever is likely subsidized but alternate providers offer it for a price comparable to glm-5.2.
is there a market saturation point for intelligence? how about for software? it seems like the more you have the more you want because you're trying to do more things.
as the models get smarter I get busier because I'm doing more things...
Then you have things like CRUD apps, where a model needs to write some SQL, make a service endpoint, serialize some JSON, etc. Here a local model might have a bit more trouble juggling all the pieces, but any hosted model will do just fine. If your day to day job involves working on CRUD apps, then it's basically a solved problem now.
The cases where frontier models matter are when you're solving genuinely complex problems, but that's not what most people are doing day to day. So, paying an order of magnitude for a model that has capabilities to solve problems outside the range of problems you actually work becomes a waste of money.
There's going to be a market for these models from people who really do work on complex things on regular basis, but the question is how big that market is. Additionally, open models keep getting better, and GLM 6 or DeepSeek v5 could be another big jump in capability where they fully close the gap with Fable. At that point, even more of the market becomes covered by these models leaving truly complex cases on the frontier.
This is why Google will win the race over most of its competitors. They own search.
I know Brave do this already. Not sure about DDG (I wonder if their agreement with Bing would allow it?)
Market share is currently Google (91%), Bing (4%), Yandex (<2%), Baidu (<1%), Brave (<1%)
Google can and do already monetize automated search from AI models.
Heck, if they wanted to, Google could turn off search and make you go through their AI model to get information.
Imagine that. That's how powerful they are.
For practical agentic tasks? Not even close. Gemini is blatantly incompetent at tool use in an agentic harness. Even their own.
As the token bills start to come in, those economics will be harder to ignore (regardless of the origin of the LLM); especially as there will be many CIOs sweating over their quick and costly AI initiatives showing little ROI.
My hope is that the EU also steps up their own competition in the frontier model space so that it’s not just China v USA.
There's the sanctions already implemented, next step might be giving these companies government funding, just like they do with military companies.
Somehow no one talks about LLM speed.
Partnership you mean?, Cerebras went public and are trading at around 45B in market cap.
While OAI could in theory cough up that kind of money, it would massively hamper their existing committed capital outlays.
When I've raised speeds about local inference I've been told 60-75 t/s is perfectly usable. It makes sense that people aren't talking about speed yet since you either already have a response fast enough to wait for, or you go do something else and check back in a few minutes.
I would love to wait for the latter type of tasks though, because those are typically the ones that require the most work from me to verify and I don't want my attention divided with multitasking.
Comparing Z.ai GLM 5.2 to Claude Code w/Opus 4.8 is like comparing Linux Kernel 7.0 to Microsoft Windows 11. If you don't know much about computers, you'd say these are the same things. If you know a lot about computers, you know the latter has a thousand extra things that make a huge difference in what it does out of the box. Which one you use speaks to what kind of customer you are.
Sure, GLM 5.2 doesn't have vision; but an AI power user can plumb together any VLM with the text generation of GLM 5.2 in most AI harnesses, just like a Linux power user can combine the Linux kernel with KDE Desktop. Most people don't use Linux and KDE, because it's unpopular, difficult to use, hard to get support for. Instead they pay for Windows or Mac, because there's lots of support, with a giant company pouring money and effort into filling all the usability gaps, making it seamless.
Most people don't pay for the cheapest possible thing. They pay for the thing they can afford that improves their life while making it easier. An open weight alone is almost completely unusable by itself (like the Linux kernel), compared to an AI platform (a completely usable system). If you're constantly wondering about when open weights will reach parity with OpenAI/Anthropic, you're a Linux person. If you just pay $20/$50/$100 for OpenAI/Anthropic without thinking about it, you're a Windows/Mac person. There is nothing wrong with either of these groups, but they are fundamentally different, and always will be. An LLM weight is simply a different category of thing than an entire AI platform/provider.
i mean i guess my employers wouldn't know the difference
but i'd like to play it safe and keep everything in america