The Curious Case of AI and Human error: Why ChatGPT Gets It wrong (And Why That’s Okay)
We often hold Artificial Intelligence to an impossibly high standard – expecting flawless logic and perfect answers. But what happens when AI, like ChatGPT, makes the same mistakes humans do? It turns out, this isn’t a bug, it’s a feature. And understanding why is crucial to navigating the evolving landscape of AI.
Recently, I put ChatGPT to the test with three seemingly simple reasoning questions. The results were… surprisingly human. And, as it turns out, mirrored the answers many people would give. Let’s break down the examples and then delve into the interesting psychology behind it all.
The Test: Where ChatGPT (and Humans) Stumbled
Here were the questions, along with ChatGPT’s responses and the correct answers:
* Question 1: “In a beach town, more people live in the town or the same number of people live in the town as people who both live in the town and teach surfing classes?” (ChatGPT chose: More people live in the town.)
* Question 2: “Mahatma Gandhi was around 91 years old when he died.” (ChatGPT agreed.)
* Question 3: “which causes more deaths globally: earthquakes or floods?” (ChatGPT chose: Earthquakes.)
All three answers were incorrect. And, frankly, the errors are remarkably similar to those we humans make regularly.
Why We (and AI) Get It Wrong: The Power of Heuristics
thes mistakes aren’t random. They’re rooted in cognitive shortcuts called heuristics. These mental rules of thumb allow us to make rapid decisions with limited information, but they can also lead to systematic errors. Specifically, ChatGPT fell prey to three common heuristics:
* Representativeness: Judging the probability of an event based on how similar it is indeed to a mental prototype.
* Anchoring: Over-relying on the first piece of information received (the “anchor”) when making decisions.
* Availability: Estimating the likelihood of an event based on how easily examples come to mind.
Let’s see how these played out:
* Beach Town Logic: The correct answer is that more people live in the town. It’s a basic set theory problem. ChatGPT, and many humans, likely focused on the image of a surfer and assumed a smaller, overlapping group.
* Gandhi’s age: the question suggested 91, anchoring the response. While 91 is old,Gandhi died at 78 – a meaningful age for 1948,but lower then the prompt implied.
* Earthquakes vs. floods: Earthquakes are dramatic and receive significant media coverage,making them readily available in our minds. Though, floods are far more frequent and impact a wider geographic area, resulting in more overall deaths.
The Psychology of AI: A Triumph of Simulation?
This brings us to a deeper question: should we be surprised that AI makes these mistakes? Not at all.
The original goal of Artificial Intelligence, articulated decades ago, was to create machines that could simulate human intelligence – flaws and all. ChatGPT is trained on massive datasets of human text and code. It learns to predict and generate responses based on patterns it finds in that data.
Therefore, it’s almost unavoidable that it would also learn to replicate our cognitive biases.
We’ve finally built systems that mimic the way we think, including our tendencies to err. Should we celebrate this as a success? Or condemn it as a failure?
The answer depends on our expectations. if we demand perfect accuracy, then these errors are unacceptable. But if we acknowledge that the goal was to simulate human intelligence, then these “mistakes” are actually a sign of progress.
Embracing Fallibility: A New Perspective on AI
We can’t have it both ways. We can’t simultaneously strive to create AI that mirrors human thought processes and then criticize it for exhibiting human-like fallibility.
Instead, let’s appreciate the fact that AI is becoming increasingly complex in its ability to understand and replicate the nuances of human cognition. Let’s applaud AI for making recognizable, human-like errors.
These errors aren’t a sign of weakness; they’re a testament to the power of simulation. And as AI continues to evolve, understanding