I think this is written forgetting a lot of the context of the time (I was at a relevant part of the industry to be present around some of the early “oh shit this thing works” moments).
It was absolutely not a given that next-token prediction would scale to the point that it could answer outside of its training domain (this is the point about “made up questions”, it was important to test it on material outside of the training set since the scale of training was yet-unheard-of at the time).
Further, the success of post-training on verifiable tasks (coding) was also a complete shock that changed the world overnight. LLMs jumped the “I can make code” to “I confidently can make code that solves your problem” chasm suddenly, and it has changed the industry.
For someone so invested in this from the early days (remember the scale of Microsoft’s commitment) it is entirely logical that testing things “on day one” would occur and this would be a much bigger shock compared to most people that learned of advances at the same time as the rest of the world. (There are huge delays between research capabilities and shipping)
I find the authors point not very compelling.. Bill Gates was by all accounts an exceptional programmer at one point and then spent a long time evaluating software engineers. He seems like a natural person to gauge AI progress in coding?
Around when AI was getting much better at SQL I was writing SQL everyday at work and had been for years. I don’t think it would’ve been hubris for me to judge that it was getting really good and would shortly be super human at SQL. It would’ve been correct anyway.
> Bill Gates was by all accounts an exceptional programmer at one point
Perhaps, but that point was in the 1970s, 50 years before he was evaluating OpenAI, and there were vastly fewer programmers in the 1970s, so "exceptional" then isn't necessarily exceptional now.
The author's point is that "better than you" doesn't mean "superhuman", it just means "usefully good".
If we assume that you're 95th percentile at writing SQL, it's going to be amazing when companies can just copy-paste the 0.95 SQL dev instead of having to pay for that expertise -- but it doesn't really mean that AI improvement is going exponential, or that we've reach the singularity, or that AI will cure all disease and end human suffering or any of the other nonsense you see AI boosters predicting on this website every day.
The first part of the article makes sense in that:
In any system of competing agents (such as human beings) where the reward for performance can enhance performance, will always result in even minor random differentials in initial performance exponentially accumulating until one or a handful of individuals dominate. In such a system, only a small portion of success can be attributed to innate skills. So it's true that success in itself, while signaling fitness to survive, does not necessarily signal an outsized ability to make accurate judgements.
I'm not entirely clear what the second part of the article is trying to convey. Is it that highly successful individuals attribute their success entirely to their skill and therefore when a machine is able to replicate their skillset, they are surprised and consider the technology to be world-altering?
This article first makes a point that Gates isn’t expert enough in biology to assess AIs capability to answer complex questions on the topic as being a breakthrough.
Then immediately goes on to say that his comments about AI coding are arrogant despite being an expert in that field with more experience than nearly anyone.
I would be very skeptical that Gates has any experience in what we could call modern programming languages and architectures. When was the last time he was hands-on in the core development of an OS or major application? Things have changed significantly since like, 1985. Supervising teams of people writing complicated software? Sure. But even then, his role was abstracted away from the code base by at least 1993 or so.
Gates in the modern era would probably do great writing a small application designed to be extremely small and run on very limited resources. If your experience is targeting applications to run on systems with 2MB to 4MB of RAM at most and MS-DOS lower memory limitations, that's its whole own historical skill set and knowledge...
Gates has a lot of raw intelligence, but I'm not sure if he ever coded in an environment where keeping complexity under control was vital, instead of powering through with brute force. In other words, whether he ever programmed, himself, where "taste" and "style" mattered.
Designing new algorithms is where the real skill is, but an average programmer will never do this, and even a 10x programmer will do it maybe a few times in their lifetime at most.
It seems pretty clear to me that Bill was saying AI was as good as him a few months ago and now it is superhuman. This is an interview transcript, remember, not a written affidavit. Humans can switch focus mid stream.
> I picked on Bill Gates above to illustrate the overconfidence effect
After having read your blog i can assert you've demonstrated that effect on yourself. Bill has all capability and credentials proven so we know this guy is an authority on software engineering topics, you have zero if i'm not mistaken.
Gates doesn't have the biology credentials. He's a pedophile, to top it off, so most probably that's where the article should have started with. It's telling that for most of the last 30 or so years the most wealthy person on this planet has been a pedophile, come to think of it maybe we (the human species) do deserve what's coming.
Author is spot on on Bill Gates. By his own admission, his company essentially missed the coming of the Internet, and we all literally suffer today for it.
He missed the coming interconnectedness of the machines and instead presumed that every computer was going to be a fancy copy machine calculator (that will never talk to others, thus security is not needed).
What is the opposite of the Dunning-Kruger effect? Where you believe since you don’t know anything about a topic (eg biology), therefore nobody else can (you may have some incredible decades of research experience to have novel thoughts on it)?
There's something I find difficult when reading this piece. The thing is, I've been making a living as a programmer and I'm now in my seventh year, but programming wasn't actually my major. I majored in computational physics in university, jumped into programming on that basis, and taught myself programming. So does that mean I have no right to make up programming problems and ask questions? I mean, couldn't Bill Gates have studied biology hard and understood things outside his own field too?
Or is the theme of the piece that one ought to have epistemic humility?
But the thing is, this piece itself also makes claims about psychology, power theory, and human nature to some degree, which is ironic. But that can't be it. Because this piece has no epistemic humility in it.
Then, if we demand according to what this piece says, in the end it comes down to "no one can test AI outside the major they studied in university" — but honestly, isn't it just a matter of buying the textbooks and papers and testing AI with them?
For sure, Bill Gates didn’t study biology for four years in college and doesn’t have diploma. How come he can now ask questions about biology after being envolved in the field for fifteen (or how many) years? Get off biology’s lawn.
There’s no reason they can’t. To be honest, it seems to me that the author hasn’t studied hard here; I can’t imagine anyone being so dismissive if they have detailed personal experience of what the current crop of frontier models can do on whatever kind of intellectual task they feel is important.
> Anything approaching my skill level is superhuman
Or someone has the impression that Ai can code as well as them (and find errors) but at 10x the speed or more. They then conclude that human ability and superhuman speed is superhuman in total.
"the very notion that something else could be "as good as I am" is, first of all, hard to believe and secondly necessarily means that it's "superhuman". That man is not lacking self-esteem, no sirree, Bob. "
I like that the article just tries to pass off a computer program coding at the level of humans (not to mention solving Millennium Problems) as something that should be just be totally unsurprising. Bill Gates is just a dumb arrogant tech bro to be surprised that a neural network exceeds his abilities at coding.
It’s just plainly not true. Yes AI uses electricity but it’s not that large a percentage of global electricity use. A lot has been happening globally that effects energy use the last few years.
I also tend to blame any uptick on electing a president that explicitly ran on burning more coal and spent money to subsidize keeping otherwise obsolete coal online.
I keep hearing this AI water and energy panic stuff. I’m a believer in facts, so I go looking for it, and it’s just not there. Water use is similar to golf courses and light agriculture, though I can see it being a problem in places like the Southwest. The electricity use is decent but not at the levels people are losing their minds over.
It’s just not there.
I can see some of the complaints over noise and political corruption like special tax breaks and deals.
I note that that chart doesn't contain data for 2026. It also doesn't contain data for 2030.
Unless something catastrophic happens, [0] construction four years out is effectively locked in. When what's being constructed is expected to increase CO2 emissions by ~50% after it's built, it's quite reasonable to assert that the LLM Industry has reversed our collective work on cutting CO2 emissions. It's exceedingly reasonable to bitch bitterly about it when those very same entities driving the construction today were screaming left and right about all of the initiatives they'd been championing to reduce power consumption and Save The Planet and whatnot just a few years ago.
[0] ..such as a fuckoff-huge bubble that's being used to justify said construction messily exploding...
None of this matters. By the time truth out itself, the Kushner's, Gate's, Andreeson's, and Altman's of the world will have walked off with hundred of billions of our 401k money.
The overconfidence effect is well documented and a known thing in light aircraft private aviation. People with education and good paying professional careers (doctors, lawyers, dentists, etc) who have the disposable income to buy a $300,000 aircraft and maintain it can easily become overconfident in their abilities. Because they're good at one thing in their life that requires a lot of education and training, they think it automatically extends to every other domain of knowledge that they set themselves to.
The original v-tailed Beechcraft Bonanza used to be known as the "v-tailed doctor killer".
Every IT project I've worked on for the last 25years have been successful even though usually 30% of IT project fail.
I'm smart enough to know that I can't make a distinction between my expertise or luck for the success of those project. But my ego wants it to be my expertise because I'm the only constant in those projects
I'm trying to find the exact clip, but there's a segment of an interview where Elon Musk explains why he finds it likely that we are living in a simulation, and part of his evidence is how unlikely he finds it to be him: He's the richest man in the world, the first trillionaire, he's influencing geopolitics and launching space rockets.
He finds it so astounding to be himself, to feel like the Main Character of the World that he considers this evidence that the world he experiences is a simulation. The more humbling proposition -- that the world is largely random and he just happens to be the guy who kept rolling sixes -- is less compelling.
he presents the idea of Bill Gates observing an OpenAI model a. doing "complex" biology problems, b. some coding problems, and then based on his assumption that Bill Gates knows rudimentary biology, and used to also write code, the fact that Bill Gates is impressed with the OpenAI model means Bill Gates thinks he has superhuman biology and coding skills. This simply does not follow.
This would be like if I went in my garage, spent several weeks with the 3d printer and CNC machine to build a robot that can "dance", then I made a video of me dancing about as well as your average dad at a wedding along with my dancing robot and I said "this thing can dance better than me!", therefore: "Ha, your robot sucks! You must think you're Baryshnikov if you call that dancing". I mean this is kind of sad.
Slightly more than a couple of years ago, just observing an LLM coherently doing work at the academic level of a sixth grader was brand new to all of us. Which was objectively impressive. It doesnt matter "so what? anyone can do 6th grade math"....this..is not..the point...
edit: the point they want to make, that these people are not qualified to make dire predictions about "superintelligence", and that they see themselves fit to determine "superintelligence" based on watching the robots be smarter than them, is one I generally agree with, and their certainty originates from a kind of hubris. but it's not a hubris you can assign to the general LLM-using population as this seems to be doing. Most people using LLMs think the "superintelligence" thing is a load of bunk (probably moreso than the general population who seem to be cowering in terror).
Ah yes, the professor from Hoboken is here to teach us that tech bros are delusional, and it’s really professors from Hoboken who know what is actually going on.
Biases and heuristics have become somewhat mainstream “intellectualism”, but just like people using the words prolific and myriad incorrectly, is leveraged here to a false conclusion.
The anti-AI narrative continues, to the detriment of those who listen to its self-destructive guidance that equates excessive enthusiasm with delusion.
I am no fan of Bill Gates. He's pedophile adjacent, in my *opinion.
But I was there. He worked really, really fucking hard. As did most of the early crowd. If you're asking yourself how people back then worked so hard, just realize that there are drugs that can help you do that.
It was absolutely not a given that next-token prediction would scale to the point that it could answer outside of its training domain (this is the point about “made up questions”, it was important to test it on material outside of the training set since the scale of training was yet-unheard-of at the time).
Further, the success of post-training on verifiable tasks (coding) was also a complete shock that changed the world overnight. LLMs jumped the “I can make code” to “I confidently can make code that solves your problem” chasm suddenly, and it has changed the industry.
For someone so invested in this from the early days (remember the scale of Microsoft’s commitment) it is entirely logical that testing things “on day one” would occur and this would be a much bigger shock compared to most people that learned of advances at the same time as the rest of the world. (There are huge delays between research capabilities and shipping)
Around when AI was getting much better at SQL I was writing SQL everyday at work and had been for years. I don’t think it would’ve been hubris for me to judge that it was getting really good and would shortly be super human at SQL. It would’ve been correct anyway.
Perhaps, but that point was in the 1970s, 50 years before he was evaluating OpenAI, and there were vastly fewer programmers in the 1970s, so "exceptional" then isn't necessarily exceptional now.
If we assume that you're 95th percentile at writing SQL, it's going to be amazing when companies can just copy-paste the 0.95 SQL dev instead of having to pay for that expertise -- but it doesn't really mean that AI improvement is going exponential, or that we've reach the singularity, or that AI will cure all disease and end human suffering or any of the other nonsense you see AI boosters predicting on this website every day.
In any system of competing agents (such as human beings) where the reward for performance can enhance performance, will always result in even minor random differentials in initial performance exponentially accumulating until one or a handful of individuals dominate. In such a system, only a small portion of success can be attributed to innate skills. So it's true that success in itself, while signaling fitness to survive, does not necessarily signal an outsized ability to make accurate judgements.
I'm not entirely clear what the second part of the article is trying to convey. Is it that highly successful individuals attribute their success entirely to their skill and therefore when a machine is able to replicate their skillset, they are surprised and consider the technology to be world-altering?
Then immediately goes on to say that his comments about AI coding are arrogant despite being an expert in that field with more experience than nearly anyone.
Unsure how to interpret that.
Being CEO does not involve a lot of coding. Yes he was an expert 40 years ago, but a lot has happened in 40 years
Gates in the modern era would probably do great writing a small application designed to be extremely small and run on very limited resources. If your experience is targeting applications to run on systems with 2MB to 4MB of RAM at most and MS-DOS lower memory limitations, that's its whole own historical skill set and knowledge...
A cursory check of his social media shows every other post is him talking about AI slop.
Designing new algorithms is where the real skill is, but an average programmer will never do this, and even a 10x programmer will do it maybe a few times in their lifetime at most.
After having read your blog i can assert you've demonstrated that effect on yourself. Bill has all capability and credentials proven so we know this guy is an authority on software engineering topics, you have zero if i'm not mistaken.
Gates doesn't have the biology credentials. He's a pedophile, to top it off, so most probably that's where the article should have started with. It's telling that for most of the last 30 or so years the most wealthy person on this planet has been a pedophile, come to think of it maybe we (the human species) do deserve what's coming.
He missed the coming interconnectedness of the machines and instead presumed that every computer was going to be a fancy copy machine calculator (that will never talk to others, thus security is not needed).
A massive hole in perception.
Or is the theme of the piece that one ought to have epistemic humility?
But the thing is, this piece itself also makes claims about psychology, power theory, and human nature to some degree, which is ironic. But that can't be it. Because this piece has no epistemic humility in it.
Then, if we demand according to what this piece says, in the end it comes down to "no one can test AI outside the major they studied in university" — but honestly, isn't it just a matter of buying the textbooks and papers and testing AI with them?
It's a slightly difficult problem.
Also, Dunning-Kruger effect doesn’t exist. It’s autocorrelation.
Or someone has the impression that Ai can code as well as them (and find errors) but at 10x the speed or more. They then conclude that human ability and superhuman speed is superhuman in total.
I like that the article just tries to pass off a computer program coding at the level of humans (not to mention solving Millennium Problems) as something that should be just be totally unsurprising. Bill Gates is just a dumb arrogant tech bro to be surprised that a neural network exceeds his abilities at coding.
https://ourworldindata.org/profile/co2/united-states
The second chart is the one I’m looking at. Where is the 15 year reversal? If zoom in I see a small uptick. Not a 15 year trend reversal.
I keep hearing this AI water and energy panic stuff. I’m a believer in facts, so I go looking for it, and it’s just not there. Water use is similar to golf courses and light agriculture, though I can see it being a problem in places like the Southwest. The electricity use is decent but not at the levels people are losing their minds over.
It’s just not there.
I can see some of the complaints over noise and political corruption like special tax breaks and deals.
https://www.bloomberg.com/news/articles/2026-10-28/us-data-c...
Unless something catastrophic happens, [0] construction four years out is effectively locked in. When what's being constructed is expected to increase CO2 emissions by ~50% after it's built, it's quite reasonable to assert that the LLM Industry has reversed our collective work on cutting CO2 emissions. It's exceedingly reasonable to bitch bitterly about it when those very same entities driving the construction today were screaming left and right about all of the initiatives they'd been championing to reduce power consumption and Save The Planet and whatnot just a few years ago.
[0] ..such as a fuckoff-huge bubble that's being used to justify said construction messily exploding...
The original v-tailed Beechcraft Bonanza used to be known as the "v-tailed doctor killer".
I'm smart enough to know that I can't make a distinction between my expertise or luck for the success of those project. But my ego wants it to be my expertise because I'm the only constant in those projects
He finds it so astounding to be himself, to feel like the Main Character of the World that he considers this evidence that the world he experiences is a simulation. The more humbling proposition -- that the world is largely random and he just happens to be the guy who kept rolling sixes -- is less compelling.
This would be like if I went in my garage, spent several weeks with the 3d printer and CNC machine to build a robot that can "dance", then I made a video of me dancing about as well as your average dad at a wedding along with my dancing robot and I said "this thing can dance better than me!", therefore: "Ha, your robot sucks! You must think you're Baryshnikov if you call that dancing". I mean this is kind of sad.
Slightly more than a couple of years ago, just observing an LLM coherently doing work at the academic level of a sixth grader was brand new to all of us. Which was objectively impressive. It doesnt matter "so what? anyone can do 6th grade math"....this..is not..the point...
edit: the point they want to make, that these people are not qualified to make dire predictions about "superintelligence", and that they see themselves fit to determine "superintelligence" based on watching the robots be smarter than them, is one I generally agree with, and their certainty originates from a kind of hubris. but it's not a hubris you can assign to the general LLM-using population as this seems to be doing. Most people using LLMs think the "superintelligence" thing is a load of bunk (probably moreso than the general population who seem to be cowering in terror).
Biases and heuristics have become somewhat mainstream “intellectualism”, but just like people using the words prolific and myriad incorrectly, is leveraged here to a false conclusion.
The anti-AI narrative continues, to the detriment of those who listen to its self-destructive guidance that equates excessive enthusiasm with delusion.
But I was there. He worked really, really fucking hard. As did most of the early crowd. If you're asking yourself how people back then worked so hard, just realize that there are drugs that can help you do that.