AI offers enormous opportunities and at the same time harbors considerable risks. For both reasons, AI scares many people. They therefore argue that AI is only based on statistics and has nothing to do with human intelligence. They forget that our entire existence is based on statistics.
The statistics factor of our existence
When reading about language models, one often learns that these LLMs only relied on finding the next probable word to generate and output after the last generated word, which is then done word by word. And so comes out what others consider an intelligently created text. Instead of words, actual tokens are processed. Simplified, tokens are something like word parts or syllables. Tokens are used to achieve compression and also to semantically better grasp longer or compound words.
Even the Hamburg Data Protection Officer uses this argument to say that he is not responsible for AI at all, because AI does not process personal data. Not much positive comes to mind when one thinks about it. It lacks any foundation, as can be proven. The following statement might already serve as a counterargument:
Language models predict what the next word is likely to be. That's not intelligent.
Language models say that probably the next word fitting to the given context is predicted. That's intelligent.
Many people do not pay attention to the part in bold: it is usually not mentioned.
German grammar
Even humans can be referred to as so-called Token-Parrot. The term came from someone with whom the author discussed technical details of AI. That person meant that language models are only Token-parrots. They would therefore parrot back what they had been fed once through training data.
Why is man also a token parrot? When we talk about German grammar, as native speakers, we see that it is based on probabilities. We learn which words and terms fit together with other words and terms. To do this, we learn probabilities. But our brain hides the principle of language learning so well from us that we don't notice that the basic principle of grammar is very simple. It wasn't until the emergence of the Transformer approach in 2017, which is a very important foundation for today's AI, that it became possible even for the naive computer scientist to understand this if he wants to.
We can wait until the holy day of never-never …
It's grammatically correct, but would a native speaker say it like that?
If a foreigner distorts a idiom due to ignorance (or poor memory) and reproduces it in another, but grammatically correct form, then we recognize the person as a foreigner. For we have perceived the deviating probability distribution.
Radioactive decay
Let's start with a quote from Wikipedia that any physicist would probably say:
The decay time of each individual atomic nucleus is random.
Reference: Wikipedia
This means: When you observe a particle, then you don't know how long this particle exists. Or differently: You cannot know, how long this particle exists. You cannot know it because knowing would be an infringement against the acknowledged physical worldview. The knowledge is therefore not possible. If interested in more I recommend popular science books like those by Werner Heisenberg or Albert Einstein.
How long does it take for a radioactive substance like uranium to stop emitting radiation? You cannot generally answer this question when looking at a particle of uranium. For lawyers: This question cannot be answered, not even "in principle". Once again: The question of when a radioactive substance is no longer radioactive cannot be answered when looking at a particle.
The Half-life is the time after which the radioactive radiation of a substance (radioactive nuclide) has halved. How does one calculate this number? At least, it holds that: "The moment of transformation of an individual atomic nucleus cannot be predicted, …" (Half-life/Wikipedia). The half-life is determined as "statistical mean" (same source).
Radioactive decay is a random process that relies on statistics.
Quantum physics
The quantum physics was made known with Albert Einstein's discovery in the year 1905. He discovered that light is not to be understood as a continuous beam, but rather in the form of tiny packets, the quanta. This is no physics lecture. In order not to enrage too many physicists, it should still be mentioned that there exists the wave-particle duality, which can be found with light. This leads into the so-called Double-Slit Experiment, which is probably the best proof for our inability to truly understand our own existence.
For example, lasers are based on findings from quantum physics. Even GPS, the basis of the navigation system in your car or on your smartphone, would not exist without quantum physics. GPS, in turn, is based on highly accurate atomic clocks, which only exist because we can understand their principle with quantum physics.
Even the tunnel effect of flash memory is based on quantum physics. To put it briefly: without quantum physics, this contribution would neither be written here on a computer nor reach you over the internet via SSD hard drive, nor would you own (payable or fast) computer. Let alone talking about a smartphone.
In quantum physics there are random events that are essentially unpredictable – even if one knows all available information about a quantum system.
Source reference Quantum fluctuation, added bold print. Other sources: Werner Heisenberg, Albert Einstein etc.
It even goes so far that there can't be a perfect vacuum. For that would violate the Heisenberg Uncertainty Principle.
If you want to know more: Werner Heisenberg, Niels Bohr, Richard Feynmann and Albert Einstein have written comprehensible books on the subject, to which others basically have little to add.
Intelligence based on a neural network
Of course, intelligence can be represented in any suitable way. But a neural network has proven itself to be particularly effective. It is used both in your brain as well as in the artificial AI-brain.
Neural networks only process numbers, nothing else.
In your brain, all signals land as analog values that manifest themselves in current and voltage. Action potentials in neurons transmit the electrical signals further (“invisible processing”).
At some point, the "output" occurs. Your mouth moves because the corresponding muscles have received the number command from the brain via the spinal cord. The numbers are currents and voltages.
Analog signals can be converted into digital signals. Some accuracy is lost in the process because analog signals are continuous and digital signals are discrete. Continuous means, for example, that there are any number of numbers between 0 and 1. Discrete means that there are only a limited number of numbers between 0 and 1. How many numbers these are in the digital system depends on the accuracy used. It is easy to see that it is not important whether there are an infinite number of numbers between 0 and 1 or "only" 100,000 billion numbers. The loss of accuracy when converting from analog to digital signals is negligible. Analog and digital systems can therefore be considered equivalent in this respect, as current AI systems prove, which are often far superior to humans (with the exception of you, of course).
Everything is a number
Language models are based on tokens. Tokens are converted into numbers. To do this, one uses a dictionary. This is so simple that you don't really need to talk about it at all. Here's an excerpt from the publicly available dictionary of GPT-2:
The strange "G" in front of some tokens is an indicator that the respective token must form the beginning of a word. All tokens without this indicator must not be at the beginning of a word. The GPT-2 dictionary has 52,000 entries.
All open source language models have a dictionary of this type, which you can download and view as a text file.
How does it work with Images? You know Dall-E or Midjourney. The procedure is as follows:
- Your text (prompt) is converted into numbers and fed into the DALL-E AI model.
- DALL-E processes these numbers via a neural network that only calculates with numbers.
- The end result is numbers. These numbers are interpreted as pixels.
A picture point is called Pixel. With a RGB color channel and a color depth of 24 bits, a pixel has 3 bytes: 1 byte for red, 1 byte for green and 1 byte for blue. Each byte can take values between 0 and 255. A pixel consists as three numbers.
And what about speech, i.e. audio signals? An MP3 file, or your recorded voice, consists of vibrations ("waveform"). Your brain (probably) processes two channels: One channel is the left ear, another is the right ear. In home theater you know 5.1 or something similar. The 5 stands for the 4 corner speakers and the center speaker. The 1 stands for the bass speaker. Audio signals can therefore be converted into numbers, as your stereo system or cell phone show.
All other signals can also be converted into numbers. At the Thermometer, it is the temperature as a numerical value, at the Seismograph it is the strength of the earthquake (about on the Richter scale). Other signals are already digital. Take for example an Excel table containing sales figures.
All signals can be converted into numbers.
Control commands, on the other hand, can be executed by sending numbers to actuators.
An actuator, such as your mouth or your hand, is controlled by sending numbers to the actuator. That's it. Where's the secret ingredient here? There is none. The fact that chemical processes are also involved is a detail that is apparently not necessary and is due to the nature of biological systems. If you see it differently, it would be good if you could contribute a few arguments.
Opinions on statistics
Opinions are not real evidence. They should only be stated here so that no one thinks that the author is alone with his opinion.
Opinions of others
From a 30-minute conversation with a DEEPL employee in November 2024 it became clear: The employee is an IT specialist and understands the technical functioning of language models. His professional focus is on linguistics, which is not surprising for DEEPL. He agrees with the author that statistics are also the basis of human intelligence. He also sees that robots will become increasingly powerful. That not every IT specialist has this insight is shown by the above example with the token parrot (the term comes from another IT specialist who may have not yet found the edge of what's possible).
Prof. Dr. Maximilian Wanderwitz is a professor for business law and IT law. He publishes extensively on the topic of AI. After his lecture in Mainz on November 26, 2024, he was asked by the author about his opinion. Mr. Wanderwitz confirmed that his view also is that statistics are an important element of human intelligence and that he sees it analogously to AI.
An T-Systems employee, who is responsible for digital systems at his company, confirmed this on November 26, 2024 as well: He sees statistics as an element of human intelligence just like in artificial intelligence.
Sam Altmann, co-founder of OpenAI sees the power of AI as so great that AI will far surpass humans. "Far" here means "unimaginably far". Surpassing intelligence with something other than intelligence seems hard to imagine.
Definition of (artificial) intelligence
The author proposes (since already April 3, 2024) the following definition of Artificial Intelligence:
An artificial intelligence is an artificial system, that attempts to solve a problem in an unspecified, useful way, even when given a fuzzy specification, by combining existing knowledge with new knowledge and drawing conclusions.
Definition of the term artificial intelligence. Source: Klaus Meffert, dr-dsgvo.de
You can decide for yourself what an artificial system is. It is not important. The 27 member states of the EU see it differently on average. This average is called the democratic process. The EU sees a machine as a prerequisite for AI. This restriction is unnecessary and arrogant. Diesel has also been shown to be environmentally friendly. Excluding this beforehand was unnecessary and wrong.
So what is intelligence? The same, except that intelligence is not (necessarily) artificial. This gives us the following definition:
As Intelligence, a System is referred to that tries, to solve a Problem also with vague specification in a not precisely specified, solution-oriented way to solve it and combines existing with new Knowledge and draws conclusions.
Definition of the term intelligence. Source: Klaus Meffert, dr-dsgvo.de
The definition of intelligence is identical to the definition of AI, with the exception of the adjective "artificial".
Your view
Do you think that AI is "only" based on statistics and is therefore not an intelligent system? Then it would be nice to know what principle, if not statistics, human intelligence is based on. Your definition of AI or intelligence will be gladly considered if you have a different one than the one mentioned above and find it better. Furthermore, it would be good to know where the argument is when someone says: "AI is based on statistics". Correct, but where is the point?
Conclusion
Statistics is the basis of our existence. The best theory we have is probably quantum theory. It describes our reality in an extremely precise way. Anyone who doubts quantum physics is denying their own existence.
Because simple is (too) simple?
The main reason for the assumption of some that statistics as a mechanism is too simple to produce intelligence.
Because simple is just simple. We have to let go of the assumption that incredible mechanisms such as what we call intelligence must be based on complicated principles. The fact that a system is not complicated enough is not an argument that this system cannot be efficient.
Intelligence is very simple in its nature. It's based (typically) on a neural network that processes numbers and learns from examples. Examples are pairs of Is (input) and Shoulds (output), or even just "Is" pairs (e.g., German texts), which are considered correct.
It is true that the German language (and many other languages around the world) is based on statistics. The fact is that text can be converted into numbers. The same applies to images, speech and other sensor values. Neural networks only ever calculate with numbers. There is no text, no sound and no image in your brain, only numbers.
It's also correct that we (you too) learn German language and grammar through examples, just like AI. A mechanism called Attention is used for this purpose. In AI it's called "Attention". It probably works without Attention as well, as current technical approaches show. It doesn't matter how intelligence arises, because there are often multiple ways to achieve a goal.
How a goal or how intelligence is achieved is completely irrelevant (when viewed over longer periods of time).
However, some approaches are more efficient than others.
AI systems usually (still) learn from data supplied by humans. One major source is the internet. However, robots learn like humans if they carry an AI brain with them. They can learn from examples from their experienced environment. These robots already exist. It's not theory, it's reality. Household robots and other robotic everyday helpers will soon be available for several thousand euros.
The idea that artificial intelligence is something qualitatively different from human intelligence leads away from reality. Even if AI were a very good simulation of intelligence: What does that make for a difference?
Utilize AI as a high-performing employee in your company. The use cases are diverse! AI works best for concrete use cases.
Update: European language model Teuken
Hard to believe, but true. The European language model called Teuken has just been published (26.11.2024).
First tests are very promising. In a practical example, it is shown what Teuken can do. For this, the first part of this contribution was fed into the Teuken-LLM and should be summarized. The task was to summarize in 3 sentences. Here is the result:
Sentence 1: Language models such as GPT-3 are based on determining and pronouncing the most probable word for the previously generated word.Sentence 2: Tokens are a way of representing longer words or sentences in a more compressed form. They can also represent semantically more complex information.Theorem 3: The argument that AI does not process personal data is wrong and shows a fundamental understanding problem of AI.Theorem 4: Humans can be called "token parrots" because they demonstrate the repetition of what they have been taught in training data.Theorem 5: German grammar is based on probabilities. This is intuitive to humans and is used unconsciously when learning language.Sentence 6: The transformer approach was a breakthrough in AI, allowing even non-experts to understand the basics of language processing.The text explains various aspects of the GPT-3 language model as well as German grammar.>>> that was 6 sentences. i want a summary in 3 sentencesTrue, here is the summarized version:1. language models like GPT-3 use tokens instead of words.2Humans can be called "token parrots" because they demonstrate the repetition of what they have been taught in training data.3.
At first, Teuken delivered 6 sets instead of 3. But these 6 sentences were very good. The request to Teuken to please provide a summary in 3 sentences was then also fulfilled. The result is impressive, even if there is a small spelling mistake.
What is striking: Teuken places "Sentence 1", "Sentence 2" etc. in front of the generated sentences. According to the author's observations, other AI models have not done this so far.
Teuken-7B-instruct-commercial-v0.4 from openGPT-X was used. Hence the license information:
Lizenzangabe (Teuken commercial):
Copyright 2024 openGPT-X
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
Key messages of this article
Language models do not simply work word by word, but with so-called tokens. These are like smaller units of words or syllables.
Humans can also be described as token parrots because, like AI, we learn and apply probability-based rules of language.
Radioactive decay is a random process in which the half-life is calculated as a statistical mean value. It is not possible to predict when an individual atom will decay.
Quantum physics is part of our everyday lives, e.g. in lasers, GPS and flash memory.
Without quantum physics, there would be neither computers nor the Internet.
In quantum physics, there are unpredictable events and a perfect vacuum cannot exist.
Intelligence is mapped in AI systems with the help of neural networks that only process numbers.
Analog signals can be converted to digital with very little loss of accuracy.
Everything is a number: language models such as GPT-2 use a dictionary with 52,000 entries to translate text into numbers. Images are also converted into numbers (pixels) by models such as DALL-E. Audio signals and other measured values can also be represented as numbers. Actuators (such as human limbs) are controlled using numbers.
Statistics is an essential component of both human intelligence and AI.
Conclusion in a nutshell
AI is a system that solves problems and processes new information. It learns from examples and works with numbers. It is the same with human intelligence.
Statistics is important for our world and quantum theory describes it best. Intelligence is simply structured: it is based on neural networks that process numbers.
Artificial intelligence is no different to human intelligence in practice – it can be just as useful. Companies should use AI to become more efficient.
My name is Klaus Meffert. I have a doctorate in computer science and have been working professionally and practically with information technology for over 30 years. I also work as an expert in IT & data protection. I achieve my results by looking at technology and law. This seems absolutely essential to me when it comes to digital data protection. My company, IT Logic GmbH, also offers consulting and development of optimized and secure AI solutions.
