I'm wondering how much cross-pollination of ideas and subsequent implementation details there are between people working at these companies. It does seem like all of this could potentially converge.
The blog post paints a vivid picture of a slop volcano. We speak of code its authors never read, never share, and never accept patches. So the entire point of git is moot.
On the other hand, some lower-level tools have to be simple, reliable and efficient. Here, sharing is critical. Even the authors of the post shared some incremental improvements to git. Here, the slop-cannon-scalability story is moot.
I think git is becoming a protocol. One product can no longer cover these cases.
What got us here won't get us there perhaps? Can you tell why open source started to exist in first place? Wasn't that a way to crowdsource manual labour, and this reason is not valid anymore, as cost of this spiecif labour is near zero?
Can anyone explain to me what this massive volume of code could actually be doing? They obviously have variations of model architectures, training infrastructure, serving infrastructure, and the source for their apps.
Why would the volume that any of this be increasing exponentially? And what are all the engineers doing when (besides the model quality) their products don't seem to be changing very much?
What they do not have is a version of chatgpt.com that is not a slow, awkward, glitchy mess.
As a user of Astra, it writes code that appears to work. It does not necessarily write sensible code that is architected in a way that makes sense based on what it does. I just watched it do a major rework of a couple thousand lines of code by keeping the original organization and kludging up an adapter to make old code have new semantics.
I would expect that this approach, coupled with lots of fun asking for changes and none of the grunt work of actually thinking about the result, could result in exponentially increasing amounts of mess.
I haven’t used a recent Anthropic model - maybe it’s better at this.
Strangler pattern can make sense for that, and that may involve that kind of adapters. But it also involves actually finishing the job and make certain new semantics go through the whole thing and restructuring until it makes sense.
Nah I made a workout app with Claude code. Today it loaded the same warmup exercise as my main block 3 times in a row. The program does not call AI to make the plan so that’s the code it wrote lol. In general though the app is good enough if you know what you’re doing which seems to be the theme with AI.
For one, they have the internal models that us normies don't have. Also, machine learning as a field is probably the most automate-able field for AI. It's 100% on computers without "real world" collisions at all, AI can do the whole stack today even with publicly available models. And since it's so empirical, you can make a lot of progress by just running more experiments with more compute.
But "big tech" companies have a lot of different components. And they hit git scaling limits long before the current AI age. The commenter was asking about OAI, not big tech, committing much more code.
> Why would the volume that any of this be increasing exponentially?
Because the limiting factor for software has always been the supply, not the demand. When the average human engineer would ship 100 lines of production code / week, you could only afford to do the most critical things. But if code gets cheaper to produce, the projects that were marginal become worth doing, and the projects that were obviously insane become marginal. And that's exactly what is happening as models get cheaper and better, leading to the frontier of worthwhile projects expanding rapidly.
> And what are all the engineers doing when (besides the model quality) their products don't seem to be changing very much?
That's got some nice But other than that, Mrs. Lincoln, how was the play? energy to it!
I’m going to assume people are asking for the benefits. Assuming that when there are so many commits it breaks git and CI, there would be a tangible thing that we can look and touch to say these 10 million PR in the last week brought you bla, cause 10 million PR are what it took to have say cloud functions or redis.
At some point I imagine they will be writing their own software for the entire training and inference stack. Kernels, libraries, why not? With compute so constrained it must be beneficial to optimize everything as much as possible
I’m not buying this argument, the apps of the LLM providers are some of the most wasteful apps that exist. Have you ever looked at how much memory / CPU Claude and Codex use? They ship with entire virtual machine hypervisors for sandboxing.
I think it’s simply a matter of dogfooding and new feature development, they do have much higher velocity than companies of equivalent size and an unlimited token budget helps as well.
becuase devs stop coding by hand and start prompting AI even for smallest of tasks, for example "hey codex, check grafana alerts and tell me x, y, z" instead of navigating it by hand and figuring out in grafana UI
> Why would the volume that any of this be increasing exponentially?
Surely it's obvious?
Slop cannons firing at a rate so high no human can hope to restrain them.
Such a colossal volume of slop that even the most well engineered, efficient technology on the planet, written by one of the greatest coders of all time, is no longer sufficient to contain it.
> Such a colossal volume of slop that even the most well engineered, efficient technology on the planet, written by one of the greatest coders of all time, is no longer sufficient to contain it.
And git is enough to version control some of the most complex software like the Linux Kernel, the FreeBSD operating system, QEMU, browsers like Firefox,... Git was insufficient for Facebook and Google because they wanted to put the whole org inside a single repo, which broke the assumption where decentralization in git would happen (repo level, not subtree level). Now it seems that another assumption is broken, that you wanted a version control system, not a snapshot or a rolling log.
It's AI slop. AI coding agents write much more voluminous code, in my experience. The base code is more expansive, and there are more tests, and there are many, many more commits as you keep fixing and fixing and fixing over and over and over.
I'd bet money they're capturing every change made by any agent, not just at the end of a 'task' (or even a turn). Free training material volume to see what was changed and then what was changed later in the same turn to fix the mistakes introduced.
If they allocate more compute internally than they sell on subs & APIs, then they should be generating more code than all the customers they sell to combined.
Anyone who has fallen into the ADR pit with agents could tell you this. Its just slop, slop to fix slop, docs about slop, slop to fix the docs, etc. 600 commits and 200k LOC/Docs for software that used to be 15 commits and 10k LOC +README and no real value to be found.
Or it's yet another iteration to get another test with overbuilt providence catalogling, boated code, overbuilt nonsense. Suddenly you're 2.5 Billion tokens deep and you just wonder "What the actual Feck did they do, how did they burn that many tokens? It's just a web app with a plugin system, my god."
I think it's more that agents gaslight you to believing that timezone from Baker Island and Howland Island is most important issue ever and needs that 1k exception.
While I don't have any particular interest in the topic this is a great blog post. No AI in every second line, no needless marketing or hype. Just a bunch of engineers talking in their own voice about a topic in great detail with numbers, charts, visualizations.
Why not use a completely different VCS topology if you are a gigantic trillion dollar behemoth? Why are we still using the "distributed" git concept as the basis for everything?
Centralization is exactly what you want at this kind of scale. The most marginal participation from the server would make a huge difference.
There is a reason game studios don't use git. It's not just about the large objects.
Haha, I'm also building a code forge (at eol.sh, just a placeholder is there now). I like that you don't look like yet another Github clone, which is my primary frustration with the alternatives.
Is the increase in commits graph in percentages because it’s fewer than Anthropic’s rate? A case of commit envy?
In all seriousness though I have enjoyed reading these posts. I will never work at this scale, but I like to read how different teams approach problems like this and I do wonder if some of these approaches will trickle down (or up?) into regular practice as everyone is committing more and faster.
> We built Pando with Codex, moving from kick-off to production, in just 10 weeks. The core of Pando was designed by a single engineer, implemented by teams of agents, and deployed by a handful of engineers. Pando began serving production traffic in April of this year, and is one part of a larger system of source control management tools used at OpenAI.
It is truly a strange time to be alive. I've made systems in the past month that I've wanted to do for years, and they're astronomically better than I would have been able to rush together had I found the time. And I'm even trying to be tempered and work out more careful approaches to some of the folks I've seen.
It's an asset because it's a demonstration that their technology works. It would be stupid for them not to tokenmaxx even if that was more efficient. Kinda like you don't want an Apple salesman to be using an Android device.
This thing aside... IDK, I used to think that there's no way to manage such a mess but apparently there is. Or rather, it starts to seem plausible that models will evolve faster than this mess, and once it becomes unmanageable, you can simply ask AI "make it better, no bugs please" and it'll mostly work.
It's becoming harder and harder to justify my own existence as a software engineer. It's surreal that that I dedicated years of study to prepare myself for a job that's being automated away.
Of course the demand for human developers won't drop to 0 but it'll be limited to a handful of industries where even small mistakes are costly, like aerospace. But then let's be real - what percentage of currently employed devs are competent enough to write such code?
> Technical details aside, this volume is ridiculous. How on earth do they see this kind of "growth" as an asset and not a liability?
That coding volume is a liability only for human engineers, with their limited memory, short attention span, tiredness, and quirky coding preferences. LLMs don't have any of those problems.
Every bit of code is more context getting soaked up in future agent runs. More context used = worse output. More tests need to be written, then fixed, using up more compute, time and context without adding value compared to a more thoughtful, efficient implementation.
It's not just the GPUs that I was referring to, but also the time spent in test runners locally and in CI. There's also the possible runtime cost of extraneous abstractions, but that's beside the point I was getting at.
Thoughtful developers do take money, but so too does shitty code that reaches a point LLMs struggle to maintain. Having both- with the thoughtful developers in charge- is better than just throwing more LLMs at it and hoping for the best.
You can't make these argument against the position that all these problems will be solved. Any problem you point to, someone would argue that an LLM can solve. Expensive compute? costs could get 1000x cheaper. Longer context gets worse output? Newer models would solve that. Ever increasing massive codebase? We'll get a 1B or a 1T context window. Time spent in test runners? parallelize, or selectively select tests, or even better; ask the LLM to optimize the tests or develop a 100x faster chips. Runtime cost of extraneous abstractions? Ask the LLM to keep functionality and focus on performance.
If you seriously believe we have "solved coding" or achieved AGI in coding, what is not possible? Any argument you can come up with will be easily dismissed with "we'll just ask the AGI to solve that problem".
Not necessarily. High LoC software is a potential bad design smell (without a counterfactual for the same level of functionality it might not be easy to tell though). Badly designed code entering an agent context is always a liability, because it contaminates decisions down the road, and makes new work slower as it needs more tokens to process and navigate old code.
And that some one can be an LLM. Does your manager at your job review every line of code you wrote or trust you based on your competence and reviews only a compressed portion?
Frankly, these discussions are tiring because there's a false notion that humans have to be in the loop 100% of the time. This is untrue. LLMs are accurate enough that they can be left to do things on their own and human intervention is needed only for out of distribution instances.
I'm not sure why this is downvoted. This is the correct answer. Code is mostly written by and for agents with the option for humans to read it from time to time.
This kind of fallacy keeps repeating time and time again. Another version of this fallacy goes like this: "agents can create a lot of code but humans have to review each PR anyway so reviewing is a massive bottleneck". This doesn't make sense because reviewing is also done by agents and humans don't have to read every line.
Another way to avoid having to maximize download speed is to use a global git cache per machine in a similar way that pnpm optimizes installing packages:
There is always something useful to learn, even if it's a mistake. pnpm in particular centralizes your packages, and symlinks them to each project that needs them, which cuts down on file copy and disk usage significantly. The same principle could be applied to cargo/rust, python venvs, dotnet, or even large C++ projects could improve their disk usage and file copy significantly if they followed the same idea. Why do you have to have a massive target folder in every rust project if all that could live in a hash addressable shared store. Its the same basic idea of CoW or deduping file systems, but done in the application for far better result.
Scaling software development at Huge AI Company /s
We at Huge AI Company has surpassed human capabilities of software development. Git can no longer keep up with our development cycle and we're proud to announce that we're moving to Wish Driven Development.
Unlike Anthropic or OpenAI where no single engineer has any idea what they're doing and still uses pathetic source tracking, we have implemented an evolution and moved on to the next stage with our AGI: Wish Driven Development.
Our engineers, now called Wish Whisperers, open pull requests to add wishes to a controlled environment where all our wishes are stored in a Wish Registry. Swarms of agents are then dispatched to validate these wishes and when the pull requests are merged, implement these wishes directly in production.
Our productivity has significantly gone up, so much so that our Wish Whisperers are now running their own Wish Registries to automate their wishing speed.
To be frank, we still have no idea what we are doing but this is AGI level productivity and it's all that matters at the end of the day.
Over the last few months at Xavi Group, our team of one engineer has gone from ~1k commits a day to ~100k commits per day as a result of debilitating financial decisions and exponentially increasing agentic workflow capability. Our internal Gitea services began to get bogged down by our proprietary frontier coding agent (internal codename `xavi-polaris`) refactoring the monorepos comprising our three flagship projects, one of which of course is our widely unknown RSI harness project.
(At this time we are not accepting any additional requests for our closed private beta, unless you ask reaaaalllyyy nicely $$$)
These demands were increasingly straining our S3 bucket capacity (petabyte scale) and our CI/CD pipelines (3 DO droplets, a macbook mini, and a legacy windows server possibly compromised). Most critically it was straining the agentic wellfare of Xavi sessions themselves, which began to flag push/pull latencies that were taking so long their caches would go cold between tool calls, resulting in even greater financial hardship.
`xavi-polaris` sess-3LQGgyUxLGXiV7OTTIMo4TWm0a6 is credited with identifying that this unprecedented pace was unsustainable, with projections that by mid 2027, commits per day could be north of 1 billion as monorepo sizes increased and refactoring became increasingly challenging despite the profound software engineering capabilities unlocked by the RSI harness project.
To identify solutions, we first looked to ensure we wouldn’t reinvent the wheel we were reinventing. We noted projects at Google, Anthropic, and OpenAI that recently aimed to address the very same challenges, but after ~20k USD in session research that incidentally solved an open conjecture in statistical analysis, we decided that adopting any of these existing protocols would limit Xavi sessions in our internal promotion benchmarks that enable “contributor” sessions to progress to “lead” sessions, irrevocably inhibiting our expansion of upper agentic management.
AGIt is a drop in replacement for Git, which we identified as being critically important because xavi sessions are deeply familiar with git commands as a result of a massive distillation and theft of proprietary data. AGIt utilizes novel CRDT and distributed systems technology built from our extensive experience with scaling distributed systems off the back of YJS. If that doesn’t make any sense to you, it didn’t make any sense to the individual human engineer at Xavi Group either.
Importantly, we don’t believe it needs to. The future is agentic workflows, and we’re excited to continue giving back to the nursery that birthed our projects and has delivered at scale for almost every other software company on the planet. We’ll be donating five `xavi-polaris` teams to pumping PRs as fast as possible to these open source projects. As we enter this new era we’d like to thank contributing sessions sess-3LQKRQrJUyAvYHlUDJcjos2CInf and sess-3LQKRnqna2emLvyUyPUqSEIaUh0, their teams were instrumental in this endeavor, and both will be promoted from “lead” to “steward” to reflect their tenacity in building this system over the last few hours.
- Cursor: https://cursor.com/blog/git-at-any-scale
- GitHub: https://github.blog/engineering/architecture-optimization/bu...
- Anthropic: https://www.anthropic.com/research/redesigning-source-contro...
- GitHub: https://github.blog/engineering/architecture-optimization/th...
Maybe also tangentially East River Source Control's too: https://ersc.io/blog/what-comes-after-git.
I'm wondering how much cross-pollination of ideas and subsequent implementation details there are between people working at these companies. It does seem like all of this could potentially converge.
Which would support your argument, but I don’t think this is particularly new.
On the other hand, some lower-level tools have to be simple, reliable and efficient. Here, sharing is critical. Even the authors of the post shared some incremental improvements to git. Here, the slop-cannon-scalability story is moot.
I think git is becoming a protocol. One product can no longer cover these cases.
Why would the volume that any of this be increasing exponentially? And what are all the engineers doing when (besides the model quality) their products don't seem to be changing very much?
As a user of Astra, it writes code that appears to work. It does not necessarily write sensible code that is architected in a way that makes sense based on what it does. I just watched it do a major rework of a couple thousand lines of code by keeping the original organization and kludging up an adapter to make old code have new semantics.
I would expect that this approach, coupled with lots of fun asking for changes and none of the grunt work of actually thinking about the result, could result in exponentially increasing amounts of mess.
I haven’t used a recent Anthropic model - maybe it’s better at this.
If you are wondering what these changes are, you can take a peek at https://github.com/openclaw/openclaw/commits/main/
This isn't to say that such pace of iteration is good or bad. That's a separate discussion.
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin...
Because the limiting factor for software has always been the supply, not the demand. When the average human engineer would ship 100 lines of production code / week, you could only afford to do the most critical things. But if code gets cheaper to produce, the projects that were marginal become worth doing, and the projects that were obviously insane become marginal. And that's exactly what is happening as models get cheaper and better, leading to the frontier of worthwhile projects expanding rapidly.
> And what are all the engineers doing when (besides the model quality) their products don't seem to be changing very much?
That's got some nice But other than that, Mrs. Lincoln, how was the play? energy to it!
I think it’s simply a matter of dogfooding and new feature development, they do have much higher velocity than companies of equivalent size and an unlimited token budget helps as well.
I do believe this will force the AI labs to eventually solve the bloat problem with AI, but we’re definitely not there yet.
Give a verifiable benchmark for the AI agent to hit such as reduce token processing time by 5% without affecting output.
It's better than the ductape and glue pipelines that we had previous to this.
The software is the definition of plumbing no one cares about and testing it is trivial.
many yaks to shave. many bikesheds to paint.
if you're paid by the line, why not go to town...
Surely it's obvious?
Slop cannons firing at a rate so high no human can hope to restrain them.
Such a colossal volume of slop that even the most well engineered, efficient technology on the planet, written by one of the greatest coders of all time, is no longer sufficient to contain it.
And git is enough to version control some of the most complex software like the Linux Kernel, the FreeBSD operating system, QEMU, browsers like Firefox,... Git was insufficient for Facebook and Google because they wanted to put the whole org inside a single repo, which broke the assumption where decentralization in git would happen (repo level, not subtree level). Now it seems that another assumption is broken, that you wanted a version control system, not a snapshot or a rolling log.
Or it's yet another iteration to get another test with overbuilt providence catalogling, boated code, overbuilt nonsense. Suddenly you're 2.5 Billion tokens deep and you just wonder "What the actual Feck did they do, how did they burn that many tokens? It's just a web app with a plugin system, my god."
[0] https://martinfowler.com/bliki/ArchitectureDecisionRecord.ht...
I've seen it on other workplaces, but from what I see from open ai and anthropic, the code shouldn't be as extended.
But maybe I'm crazy lol
Wasn't this flagged as a big risk, as it seems to place complete trust in the client?
Especially with all the rogue agents and containment/monitoring failures we have seen in 2026?
Would love to hear how the validation is deemed sufficient.
Centralization is exactly what you want at this kind of scale. The most marginal participation from the server would make a huge difference.
There is a reason game studios don't use git. It's not just about the large objects.
Disclaimer: I am building a code forge at https://juju.bi
In all seriousness though I have enjoyed reading these posts. I will never work at this scale, but I like to read how different teams approach problems like this and I do wonder if some of these approaches will trickle down (or up?) into regular practice as everyone is committing more and faster.
Impressive!?
This thing aside... IDK, I used to think that there's no way to manage such a mess but apparently there is. Or rather, it starts to seem plausible that models will evolve faster than this mess, and once it becomes unmanageable, you can simply ask AI "make it better, no bugs please" and it'll mostly work.
It's becoming harder and harder to justify my own existence as a software engineer. It's surreal that that I dedicated years of study to prepare myself for a job that's being automated away.
Of course the demand for human developers won't drop to 0 but it'll be limited to a handful of industries where even small mistakes are costly, like aerospace. But then let's be real - what percentage of currently employed devs are competent enough to write such code?
That coding volume is a liability only for human engineers, with their limited memory, short attention span, tiredness, and quirky coding preferences. LLMs don't have any of those problems.
LLM compute is getting cheaper, so one can re-run the coding agent until it produces the desired result.
> without adding value compared to a more thoughtful, efficient implementation
Thoughtful implementations require thoughtful developers, and thoughtful developers cost more money that dumb GPUs.
Thoughtful developers do take money, but so too does shitty code that reaches a point LLMs struggle to maintain. Having both- with the thoughtful developers in charge- is better than just throwing more LLMs at it and hoping for the best.
If you seriously believe we have "solved coding" or achieved AGI in coding, what is not possible? Any argument you can come up with will be easily dismissed with "we'll just ask the AGI to solve that problem".
Frankly, these discussions are tiring because there's a false notion that humans have to be in the loop 100% of the time. This is untrue. LLMs are accurate enough that they can be left to do things on their own and human intervention is needed only for out of distribution instances.
This kind of fallacy keeps repeating time and time again. Another version of this fallacy goes like this: "agents can create a lot of code but humans have to review each PR anyway so reviewing is a massive bottleneck". This doesn't make sense because reviewing is also done by agents and humans don't have to read every line.
https://git-dedup.ben3d.ca
Also, maybe FUD, but is there anything useful to learn from (p)npm other than cautionary tales?
Unlike Anthropic or OpenAI where no single engineer has any idea what they're doing and still uses pathetic source tracking, we have implemented an evolution and moved on to the next stage with our AGI: Wish Driven Development.
Our engineers, now called Wish Whisperers, open pull requests to add wishes to a controlled environment where all our wishes are stored in a Wish Registry. Swarms of agents are then dispatched to validate these wishes and when the pull requests are merged, implement these wishes directly in production.
Our productivity has significantly gone up, so much so that our Wish Whisperers are now running their own Wish Registries to automate their wishing speed.
To be frank, we still have no idea what we are doing but this is AGI level productivity and it's all that matters at the end of the day.
Over the last few months at Xavi Group, our team of one engineer has gone from ~1k commits a day to ~100k commits per day as a result of debilitating financial decisions and exponentially increasing agentic workflow capability. Our internal Gitea services began to get bogged down by our proprietary frontier coding agent (internal codename `xavi-polaris`) refactoring the monorepos comprising our three flagship projects, one of which of course is our widely unknown RSI harness project.
(At this time we are not accepting any additional requests for our closed private beta, unless you ask reaaaalllyyy nicely $$$)
These demands were increasingly straining our S3 bucket capacity (petabyte scale) and our CI/CD pipelines (3 DO droplets, a macbook mini, and a legacy windows server possibly compromised). Most critically it was straining the agentic wellfare of Xavi sessions themselves, which began to flag push/pull latencies that were taking so long their caches would go cold between tool calls, resulting in even greater financial hardship.
`xavi-polaris` sess-3LQGgyUxLGXiV7OTTIMo4TWm0a6 is credited with identifying that this unprecedented pace was unsustainable, with projections that by mid 2027, commits per day could be north of 1 billion as monorepo sizes increased and refactoring became increasingly challenging despite the profound software engineering capabilities unlocked by the RSI harness project.
To identify solutions, we first looked to ensure we wouldn’t reinvent the wheel we were reinventing. We noted projects at Google, Anthropic, and OpenAI that recently aimed to address the very same challenges, but after ~20k USD in session research that incidentally solved an open conjecture in statistical analysis, we decided that adopting any of these existing protocols would limit Xavi sessions in our internal promotion benchmarks that enable “contributor” sessions to progress to “lead” sessions, irrevocably inhibiting our expansion of upper agentic management.
We’re proud to introduce our new, closed source Git model: AGIt.
AGIt is a drop in replacement for Git, which we identified as being critically important because xavi sessions are deeply familiar with git commands as a result of a massive distillation and theft of proprietary data. AGIt utilizes novel CRDT and distributed systems technology built from our extensive experience with scaling distributed systems off the back of YJS. If that doesn’t make any sense to you, it didn’t make any sense to the individual human engineer at Xavi Group either.
Importantly, we don’t believe it needs to. The future is agentic workflows, and we’re excited to continue giving back to the nursery that birthed our projects and has delivered at scale for almost every other software company on the planet. We’ll be donating five `xavi-polaris` teams to pumping PRs as fast as possible to these open source projects. As we enter this new era we’d like to thank contributing sessions sess-3LQKRQrJUyAvYHlUDJcjos2CInf and sess-3LQKRnqna2emLvyUyPUqSEIaUh0, their teams were instrumental in this endeavor, and both will be promoted from “lead” to “steward” to reflect their tenacity in building this system over the last few hours.