Deep learning is an artificial intelligence technique for pattern recognition, a successful method that is penetrating the field of computer science. We often see it in many famous news headlines, such as the ability to diagnose diseases more accurately than doctors or the capability to prevent traffic accidents through autonomous driving features. However, the reality is that many deep learning systems are unreliable and easily fooled.
This makes AI systems resemble some overly confident humans, often exhibiting confidence that far exceeds their actual capabilities. While humans are relatively good at recognizing their own mistakes, many AIs have no way of knowing when they are wrong. Sometimes, AI systems struggle to recognize their errors even more than they do to produce accurate results.
The reality is that many deep learning systems are unreliable and easily fooled.
Researchers from the University of Cambridge and the University of Oslo state that this instability is the Achilles’ heel of modern AI and that a mathematical paradox highlights the limitations of AI. This paradox can be traced back to two great mathematicians of the 20th century, Alan Turing and Kurt Gödel. In the early 20th century, mathematicians were attempting to prove that mathematics was the ultimate language to unify science. However, Turing and Gödel discovered a paradox at the heart of mathematics: the truth of certain mathematical propositions cannot be proven, and some computational problems cannot be solved by algorithms.
By the end of the 20th century, mathematician Steve Smale proposed a list of 18 unsolved problems, the last of which addresses the limits of human and machine intelligence. This problem remains unresolved to this day, but it presents the world of artificial intelligence with a paradox first raised by Turing and Gödel: mathematics has inherent fundamental limits, and similarly, artificial intelligence algorithms also face unsolvable problems.
Neural networks can be trained to provide a reliable AI system.
A new study shows that artificial intelligence, pervasive with inherent limitations, may be the root cause of this longstanding mathematical paradox. By extending the approach proposed by Gödel and Turing, researchers have demonstrated the limits of algorithms for computational neural networks. They propose a classification theory that describes how, under certain conditions, a neural network can be trained to provide a reliable AI system.
Neural networks are the most advanced tools in the field of artificial intelligence, simulating the connections between neurons in the brain. In the new study, researchers indicate that while good neural networks may exist in certain cases, we cannot create a reliably trustworthy neural network due to this paradox. In other words, no matter how accurate the data we use to build the neural network, we can never obtain the perfect information required to create a 100% perfect neural network.
At the same time, it is impossible to compute how good an existing neural network is, regardless of how extensively it has been trained with data. No matter how much data an algorithm has access to, it will never produce the network it needs. This aligns with Turing’s perspective: there are computational problems that cannot be solved regardless of computational power and runtime.
Researchers point out that not all artificial intelligence is flawed. In some cases, it is only reliable in specific fields, using particular methods. However, this is not what we observe in many systems.
Some situations where AI makes mistakes are perfectly fine, but it needs to be honest about that.
Dr. Matthew Colbrook from the Department of Applied Mathematics and Theoretical Physics stated: “In some situations, it is perfectly fine for AI to make mistakes, but it needs to be honest about it. And that is not what we are seeing with many systems—there is no way to know when they are more or less confident about a decision.”
Professor Anders Hansen from the Department of Applied Mathematics and Theoretical Physics at Cambridge said: “Currently, AI systems can sometimes be predictable. But at some point, you will feel exhausted from not achieving what you want, and you will try a different method. It is important to understand the limitations of different approaches. We are at a stage where the practical successes of AI far exceed theory and understanding. There needs to be a foundational program to understand AI computing to bridge this gap.”
“When 20th-century mathematicians identified different paradoxes, they did not stop researching mathematics. They sought new paths because they understood the limitations,” Colbrook said. “For AI, it may be the case of changing pathways or developing new routes to build systems that can solve problems reliably and transparently while understanding their limitations.”
We are at a stage where the practical successes of AI far exceed theory and understanding.
The next stage for researchers is to combine approximation theory, numerical analysis, and computational foundations to determine which neural networks can be computed by algorithms and which can be stable and reliable. Just as the paradoxes regarding the limits of mathematics and computing identified by Gödel and Turing have led to rich foundational theories—describing both the limits and capabilities of mathematics and computation.
