Why most people underestimate AI capabilities
- Why are most people not getting the whole AI risk thing? Many of us LLM power users are living in a different world. Opus 5.5 is out and has been an inflection point and it doesn’t seem that most people have really updated on it, even though it might be trivial to demonstrate the state of the art and thus raise awareness amongst the general population. On some level, I can’t believe that AI frontier companies' marketing is not better on this, but maybe they don’t care because the money is now in API and RSI and no longer in selling subscriptions.
- Here’s a list of several biases that lead large parts of the population to underestimate the current state of AI, going from most prevalent and perhaps only common among very casual AI users to biases that are probably at this stage even amongst most well-educated knowledge workers:
- Voice mode: I think AI companies push voice mode because many people don’t actually type. Like, I vaguely remember that a very large amount of WhatsApp and Meta usage is voice messages. And so, for the normal consumer the voice mode is probably what they use. The problem is of course that voice mode, which has to be fast and low-latency, can’t reason much at all, and so people underestimate the current state of AI. There’s a whole genre of videos on Instagram with very many views and likes where people talk to AI via voice mode and make fun of their silly errors (e.g. can’t tell you how many r’s there are in strawberry).
- Free or cheap consumer subscriptions: People use the free version or at most a $20 subscription which doesn’t really let them use AI for everyday tasks (more below).
- Low reasoning effort / Latency: Lots of people are impatient and want their answers or even tasks fast. And AI still struggles with that at a cost reasonable for consumers.
- Lack of management experience: Seems like the problems that people have with AI these days are also due to lack of experience managing people. I remember Anthropic recently made an announcement to that effect saying that the new Claude version would be better at trying to figure out what the user actually wants because people are bad at verbalizing it / managing Claude.
On the other hand, LLM power users have fundamentally different workflows. For instance, I’ve recently stopped using the web app for a bunch of things and now start sessions with Fable 5.2 in Claude Code with the $200 Max sub with claude --permission-mode auto --rc --effort max --settings '{"ultracode":true}' :
- --permission-mode auto: This is probably super ill-advised but this gives Claude access to my computer and it can log into my Chrome plugin, and buy things and so on.
- --rc: This is remote control, I can then tell it from my phone to work on things on my computer. It can use my computer and pretty much do everything fairly sensibly. --effort max --settings '{"ultracode":true}': Fable then orchestrates a swarm of Opus 5.5 instances and so I don’t actually run into Fable rate limits. Many will say this is overkill but I just want to get the task done correctly with minimal input, and I have to wait anyway so I might as well set it to maximum reasoning etc. (on the $200 sub you can ask Claude to do all your online life / work admin without hitting the limit).
More and more, whenever I find myself having a small admin task like buying or booking something online or doing small work tasks I hand it over to Claude, even if I can do it faster (to be fair Claude even in the very expensive fast mode is still somewhat slow). But even really popular web applications are often so poorly designed and buggy, that the frictions cause cognitive load that I’d rather not use them. What I now prefer is just talking to Claude. Like for instance today, I couldn’t add a new address on eBay during the checkout! On freaking eBay! Off to Claude to take care of it. Many such cases. Because Claude is fairly slow, this works best for asynchronous tasks but it’s just a matter of time before it’ll be much faster. Anthropic also has a bunch of limits on Claude like not letting it make calls (but Grok for instance can also make calls). Soon, we’ll have faster and cheaper models that will be able to do that. But Claude with this setup is already at the level of a competent secretary at this stage, and then there’ll be massive demand for lots of AI, because everyone will have to have one.
You might say none of this is novel or surprising but remember OpenClaw? And how everyone was talking about it? And then a couple of months later people made fun of people buying all the Mac Minis because it was still so error-prone? It no longer is and I was thinking of buying a Mac Mini recently. It seems somewhat overdetermined that every person will have their own personal AI assistant.
You might say that this was unsurprising, but do you remember how Dwarkesh just about a year ago was skeptical that LLMs could do his taxes? I don’t really see why LLMs couldn’t do his taxes these days with enough compute.