Artificial intelligence as a bullshit magnet

AI is the hot topic that has already revolutionized our everyday lives and will continue to change them significantly. Many are suddenly AI experts. Many are calling for AI to be regulated. Many trivialize AI and say that AI language models do not process personal data. The following is an outline that aims to clarify misunderstandings.

Introduction

AI is both underestimated and overestimated. Most people, often myself included, do not understand the possibilities offered by AI systems. Just yesterday, I saw revolutionary AI approaches that were unknown two weeks ago. As someone who works very intensively with artificial intelligence, I feel this way almost every day.

Many think that AI is a hype, which will fade away again soon. Wrong! With the Transformer approach, in 2017 the Intelligence function of humans was deciphered, I say. Instead of programming an algorithm for solving a problem, I only have to feed enough examples into my AI system, which runs under the desk. So even previously unknown hieroglyphs were discovered and deciphered.

From a justified fear of the negative consequences of increasingly powerful AI systems, many are calling for regulation. But they don't say how.

Then there are detractors, who want to profile themselves as AI experts or legal enablers. They tell others how or that they can use ChatGPT profitably. Even at the DSRI conference (German Foundation for Law and Informatics), it was claimed in a contribution that AI models do not process personal data.

Others reassure by referring to the new informal data protection agreement between Europe and the USA. Just because data can now be sent to the USA without additional guarantees, some suggest that any data processing is therefore permitted.

A few details on the individual points follow.

Possibilities of AI systems

An AI can do everything a human can do and much more. Maybe not yet, but potentially (in a specific application area X) as early as next week. Robots with AI brains will soon be walking around and experiencing the environment. This will be exactly the same as how children learn. We will see who takes the place of parents. It could be human trainers, but also other robots or algorithms.

An example of rapid development: AI language models could only process a very few characters of text at once. This amount of text is referred to as context length. Until just a few months ago, the context length in almost all AI language models I was familiar with was 1024 characters, or one kilobyte.

The context length increased every week, first to 2048, then to 4096, then to 8192, then to 16,000 characters, and later to 32,000 characters. ChatGPT recently boasted a context length of 128,000 characters.

Yesterday I read about an approach that has been known in research for a few months. It can process a context length of one billion characters (= 1,000,000,000) at once. A quick calculation: Before = 128,000 characters, one blink of an eye later = 1,000,000,000 characters. That's an improvement by a factor of 7800, just like that.

Another example: The Transformer approach mentioned above has a few weaknesses. It is very resource-hungry. Even high-performance computers or graphics cards need a few seconds to generate an answer to a question to the chatbot. Every ChatGPT user knows what I'm talking about. Now there is an approach that provides the same response quality, but responds 8 times faster and requires only a third of the expensive and barely available graphics card memory for its calculations.

If you are over 50 years old, I have good news for you: There is a chance that you will the of natural causes and in peace. All significantly younger people will experience the end of humanity because AI systems will massively outdo, enslave or wipe us out. Possibly another catastrophe will occur beforehand, but this article is not about that.

Is AI just statistics?

The question is irrelevant. It doesn't matter whether the human brain is based on statistical processes. What matters is what comes out in the end. Obviously, our entire existence is based on statistical processes. Compare this with quantum physics, a very elementary and powerful theory. Quantum physics is based on the fact that the behavior of a tiny particle of our existence cannot really be predicted. Rather, a statement can only be made about particles if many are considered and the average is drawn from the observations.

Obviously, German grammar is based on learning which words are typically strung together and fit together. That is also statistics. But hardly anyone talks about it.

The regulation of AI

The capabilities of AI make many people rightly anxious or worried. Out of their felt helplessness, some demand regulation of AI systems. What exactly that means is usually not said. The only demands that have stuck in my head are as follows:

• Labeling of AI-generated works: Images, videos, texts…
• Disclosure of the sources used to train an AI system
• Anything else? I can't think of anything worth mentioning right now

On the first point: it's a gift. Labeling works is a good idea, but it changes almost nothing. Criminals will not start labeling their fake videos and fake news as artificially generated fake works. Everyone else will dutifully adhere to the label. This won't save humanity, but it can be done. The benefit is there, but it is only a selective intervention that has hardly any qualitative effect.

Disclosure of sources, i.e. training data: Anyone who demands this simply has no idea how AI models are structured. This demand comes years too late. The sources are usually known:

• The Pile (Text)
• Common Crawl (Text)
• LAION data set (pictures, German club!)

The text data includes Wikipedia, news pages and popular websites in particular.