AI hallucinations are false statements made by an AI based on correct information. The result is a plausible statement that could be correct, but is not. An explanation for hallucinations is possible if you look at how AI language models encode information and its meaning, for example.
Introduction
Everyone has probably heard of hallucinations in the context of artificial intelligence.
Hallucinations are so fascinating because they involve false statements that have the potential to sound plausible. What sounds plausible is often taken as true or correct. That's exactly where the danger of hallucinations lies.
Hallucinations are false statements that are based on correct background knowledge. False statements that arise from incorrect knowledge or misunderstandings due to a poorly formulated question are not hallucinations.
The basis for hallucinations are semantic vectors. Let's take a closer look at what that is.
How hallucinations develop
A semantic vector is the carrier of a meaning. What is a meaning? A meaning is a subjectively defined statement. You could also say that everyone defines their own truth.
I make the world for myself,
widewide as I like it.
Pippi Longstocking
Truth is (always?) something subjective. Even physical theories such as the theory of relativity and quantum mechanics, which are probably among our best theories, are interpreted purely subjectively.
Humans find the meaning of objects and circumstances through cultural conditioning, education, and personal observation. The computer ("the AI") searches for meaning based on training data, which is essentially nothing other than that.
Meaning is found through optimization of a mathematical function. So will probably also do humans, only that the optimization in humans can also intentionally contain contraproductive elements (!?) on purpose.
The following image shows a simplified representation of how the meaning of the sentence "The dog is running in the park" is encoded in an AI model.
The sentence just mentioned has numerous facets of meaning. For example, it says something about a living being ("dog"), addresses an activity ("running"), contains a tonality (here: neutral emotion) and describes a place ("park"). Incidentally, the term "park" can have several meanings. Here it refers to the green area in a city. Another meaning that is not meant here, but would be possible, is the imperative of "parken".
The illustration shows only two dimensions (2D) for simplicity's sake. In reality, AI models work with multiple hundreds of dimensions, such as 512. This high number of dimensions serves to capture various meaning facets of a statement, thus being able to represent them.
The statement "The dog is running in the park" is true. How can this result in a false statement?
Many AI systems generate results by representing the meaning of the input as a vector (or vectors) and do the same for stored background knowledge. Now you can calculate with vectors – anyone who is fit in math or can remember better times will know this.
The following figure schematically shows the addition of two semantic vectors.
You can see two statements that are both true:
- Albert Einstein received the Nobel Prize for Physics -> True statement
- Albert Einstein developed the theory of relativity -> True statement
If you now add the vectors of these two statements, you get the vector shown in red in the image, which represents a false statement. The addition of the two arrows, green and blue, is illustrated in the top right of the figure in miniature form. The dashed lines also indicate how the red arrow is created from the green and blue arrows.
In the AI model, through similarity search with the red result vector, then the statement that best matches the result vector is generated. As a result, the false statement: "Einstein received the Nobel Prize for the Theory of Relativity".
It is hard to believe that one of the most outstanding geniuses in human history did NOT receive a Nobel Prize for one of the most outstanding theories in human history. This could be seen as impertinence or as a sign of the global stupidity of mankind. After all, the following achievements are based on Einstein's theory of relativity:
- GPS and Navigation Systems: Without considering relativistic effects, GPS satellites would already be off by several kilometers after just a few hours. The clocks on the satellites run faster due to the weaker gravity.
- Particle accelerators like the Large Hadron Collider at CERN only work because relativistic effects are taken into account when accelerating particles up to nearly the speed of light. These facilities have led to the discovery of the Higgs boson and many other elementary particles.
- Medical Imaging uses the theory of relativity in Positron Emission Tomography (PET). Here, antimatter particles are used whose behavior Einstein had predicted beforehand.
- Astronomy and Cosmology were revolutionized. The theory enabled the prediction and later observation of Black Holes, Gravitational Waves, and helped in understanding the expansion of the Universe.
- Quantum computing benefits from relativistic concepts, particularly in the development of highly precise atomic clocks and quantum sensors.
- Synchronisation of computer networks and financial trading systems now takes into account relativistic time effects for precise time measurements over large distances.
At best, the Nobel jurors were intellectually overwhelmed at the time and did not dare to give their blessing to a theory that what possibly wrong in their eyes.
Back to hallucinations:
My name is Klaus Meffert. I have a doctorate in computer science and have been working professionally and practically with information technology for over 30 years. I also work as an expert in IT & data protection. I achieve my results by looking at technology and law. This seems absolutely essential to me when it comes to digital data protection. My company, IT Logic GmbH, also offers consulting and development of optimized and secure AI solutions.
