A car that accelerates instead of braking every once in a while is not ready for the road. A faucet that occasionally spits out boiling water instead of cold does not belong in your home. Working properly most of the time simply isn’t good enough for technologies that people are heavily reliant upon. And two and a half years after the launch of ChatGPT, generative AI is becoming such a technology.
Even without actively seeking out a chatbot, billions of people are now pushed to interact with AI when searching the web, checking their email, using social media, and online shopping. Ninety-two percent of Fortune 500 companies use OpenAI products, universities are providing free chatbot access to potentially millions of students, and U.S. national-intelligence agencies are deploying AI programs across their workflows.
When ChatGPT went down for several hours last week, everyday users, students with exams, and office workers posted in despair: “If it doesnt come back soon my boss is gonna start asking why I havent done anything all day,” one person commented on Downdetector, a website that tracks internet outages. “I have an interview tomorrow for a position I know practically nothing about, who will coach me??” wrote another. That same day—June 10, 2025—a Google AI overview told me the date was June 18, 2024.
For all their promise, these tools are still … janky. At the start of the AI boom, there were plenty of train wrecks—Bing’s chatbot telling a tech columnist to leave his wife, ChatGPT espousing overt racism—but these were plausibly passed off as early-stage bugs. Today, though the overall quality of generative-AI products has improved dramatically, subtle errors persist: the wrong date, incorrect math, fake books and quotes. Google Search now bombards users with AI overviews above the actual search results or a reliable Wikipedia snippet; these occasionally include such errors, a problem that Google warns about in a disclaimer beneath each overview. Facebook, Instagram, and X are awash with bots and AI-generated slop. Amazon is stuffed with AI-generated scam products. Earlier this year, Apple disabled AI-generated news alerts after the feature inaccurately summarized multiple headlines. Meanwhile, outages like last week’s ChatGPT brownout are not uncommon.
Digital services and products were, of course, never perfect. Google Search already has lots of unhelpful advertisements, while social-media algorithms have amplified radicalizing misinformation. But as basic services for finding information or connecting with friends, until recently, they worked. Meanwhile, the chatbots being deployed as fixes to the old web’s failings—Google’s rush to overhaul Search with AI, Mark Zuckerberg’s absurd statement that AI can replace human friends, Elon Musk’s suggestion that his Grok chatbot can combat misinformation on X—are only exacerbating those problems while also introducing entirely new sorts of malfunctions and disasters. More important, the extent of the AI industry’s new ambitions—to rewire not just the web, but also the economy, education, and even the workings of government with a single technology—magnifies any flaw to the same scale.
The reasons for generative AI’s problems are no mystery. Large language models like those that underlie ChatGPT work by predicting characters in a sequence, mapping statistical relationships between bits of text and the ideas they represent. Yet prediction, by definition, is not certainty. Chatbots are very good at producing writing that sounds convincing, but they do not make decisions according to what’s factually correct. Instead, they arrange patterns of words according to what “sounds” right. Meanwhile, these products’ internal algorithms are so large and complex that researchers cannot hope to fully understand their abilities and limitations. For all the additional protections tech companies have added to make AI more accurate, these bots can never guarantee accuracy. The embarrassing failures are a feature of AI products, and thus they are becoming features of the broader internet.
If this is the AI age, then we’re living in broken times. Nevertheless, Sam Altman has called ChatGPT an “oracular system that can sort of do anything within reason” and last week proclaimed that OpenAI has “built systems that are smarter than people in many ways.” (Debateable.) Mark Zuckerberg has repeatedly said that Meta will build AI coding agents equivalent to “mid-level” human engineers this year. Just this week, Amazon released an internal memo saying it expects to reduce its total workforce as it implements more AI tools.
The anomalies are sometimes strange and very concerning. Recent updates have caused ChatGPT to become aggressively obsequious and the Grok chatbot, on X, to fixate on a conspiracy theory about “white genocide.” (X later attributed the problem to an unauthorized change to the bot, which the company corrected.) A recent New York Times investigation reported several instances of AI chatbots inducing mental breakdowns and psychotic episodes. These models are vulnerable to all sorts of simple cyberattacks. I’ve repeatedly seen advanced AI models stuck in doom loops, repeating the same sequence until they manually shut down. Silicon Valley is betting the future of the web on technology that can unexpectedly go off the rails, melt down at the simplest tasks, and be misused with alarmingly little friction. The internet is reverting to beta mode.
My point isn’t that generative AI is a scam or that it’s useless. These tools can be legitimately helpful for many people when used in a measured way, with human verification; I’ve reported on scientific work that has advanced as a result of the technology, including revolutions in neuroscience and drug discovery. But these success stories bear little resemblance to the way many people and firms understand and use the technology; marketing has far outpaced innovation. Rather than targeted, cautiously executed uses, many throw generative AI at any task imaginable, with Big Tech’s encouragement. “Everyone Is Using AI for Everything,” a Times headline proclaimed this week. Therein lies the issue: Generative AI is a technology that works well enough for users to become dependent, but not consistently enough to be truly dependable.
Reorienting the internet and society around imperfect and relatively untested products is not the inevitable result of scientific and technological progress—it is an active choice Silicon Valley is making, every day. That future web is one in which most people and organizations depend on AI for most tasks. This would mean an internet in which every search, set of directions, dinner recommendation, event synopsis, voicemail summary, and email is a tiny bit suspect; in which digital services that essentially worked in the 2010s are just a little bit unreliable. And while minor inconveniences for individual users may be fine, even amusing, an AI bot taking incorrect notes during a doctor visit, or generating an incorrect treatment plan, is not.
AI products could settle into a liminal zone. They may not be wrong frequently enough to be jettisoned, but they also may not be wrong rarely enough to ever be fully trusted. For now, the technology’s flaws are readily detected and corrected. But as people become more and more accustomed to AI in their life—at school, at work, at home—they may cease to notice. Already, a growing body of research correlates persistent use of AI with a drop in critical thinking; humans become reliant on AI and unwilling, perhaps unable, to verify its work. As chatbots creep into every digital crevice, they may continue to degrade the web gradually, even gently. Today’s jankiness may, by tomorrow, simply be normal.
Source link
