ChatGPT and Bing AI as a guarantee for incorrect results. Data friendliness and data accuracy?

Because ChatGPT first amazed mankind with its incredible results, many people think they have to use this chatbot for everything and everyone. What seems harmless in the private sphere quickly leads to problems in business life that can quickly become justiciable.

Introduction

Recently, a German company announced that they offer an AI solution for interpreting legal texts. The trigger was a contribution on Dr. DSGVO about Artificial Intelligence for analyzing legal texts.

The company's AI application is still in the beta stage. There is reason to suspect that a prototype was built in a hurry in order to explore the market for such applications. At no point was it mentioned which language model is used in the background. Investigations suggest that it is ChatGPT.

This article was created together with a presentation from 05.09.2023 at the IT Club Mainz at the Gutenberg Digital Hub in Mainz/Germany.

See below for impression.

The AI system's answers were good, but not earth-shattering. It provided reasonably plausible answers to legal questions, including sources. However, the answers would not have been sufficient to win a legal dispute. In fact, the results would probably have led directly to defeat.

The fact that ChatGPT-3 was used by the German company's AI application was established by asking the chatbot for information about this. The AI's response was: "Yes, I am based on a model called GPT-3, which was developed by OpenAI. […]

What's so problematic about ChatGPT? The question could also be asked for Microsoft Copilot. A simple test proves, that Copilot itself is overburdened with simple tasks.

Motivation

Chatbots typically draw on a pre-compiled knowledge base. Here's an example of an application from the above German company, which apparently uses ChatGPT in the background:

The chatbot therefore does not know a judgment that exists and is uniquely identified (for a lawyer) by a unique identifier. The chatbot should have told the user where the limits of the chatbot are rather than pretending that the judgment they are looking for does not exist.

To use ChatGPT, all you need to do is use the OpenAI interface (API). Because it is so easy to use this API, many people seem to feel compelled to do so. This is where the trouble begins, as you can see from the example above.

This is how easy it is to use ChatGPT from your own program (note: the data all flows to OpenAI/ChatGPT, only the following program code is local to the programmer):

import openai
openai.api\_key = "XYZ" #You OpenAI API-key
completion = openai.ChatCompletion()
def frage\_chatgpt(frage):
chat\_log = \[{
'role': 'system',
'content': 'You are a helpful assistant',
}\]
chat\_log.append({'role': 'user', 'content': frage})
response = completion.create(model='gpt-3.5-turbo', messages=chat\_log)
answer = response.choices\[0\]\['message'\]\['content'\]
return answer
print(frage\_chatgpt("What is the answer to all questions?"))

Anyone really can do it. In any case, as many non-programmers can create and execute this code as there are people who are not master mechanics and can change a tire on their car. As indicated in the code, the fun costs money once a test quota per call has been exceeded. The response time could also be faster (this is complaining at a high level to dampen the illusion of some that ChatGPT can do everything).

Another example of a false statement generated by ChatGPT:

Here is the translation: My conclusion, which can also flow into my own AI, but not into ChatGPT: Cookies are not text files, but datasets. ChatGPT will therefore continue to always answer incorrectly.

As a reminder: ChatGPT is an extremely advanced language model that's in a league of its own.

Unfortunately, ChatGPT is based on so many millions or billions of documents that too much imprecise general knowledge and too little precise specialized knowledge has found its way into the artificial brain.

The Microsoft Bing Search is also driven by AI. Microsoft recently financially participated in OpenAI to profit from the ChatGPT hype. Especially when simple formulated search queries come into play, the results are quickly wrong or absurd if the Bing AI comes into play. Conversely, the results often lie within the positive expectation range if Bing searches conventionally. Conventionally, it often works best to get good hits. The difference is that answers from the search results are only quoted, not abstracted, i.e., in their own words reproduced by the AI.

Better a correct, quoted answer than a wrong answer in your own words.

The better solution

A better solution provides correct answers more often and gives the user the chance to better recognize potentially inaccurate or incorrect answers so that they can critically review them. Whether an answer may be inaccurate or incorrect can be easily determined with a company's own AI, unlike when using the black box called ChatGPT.

The use case for company-owned AI systems is always given when exact answers are required or when own documents are to be searched frequently. ChatGPT either fails here or is too expensive in the best case, as OpenAI charges according to usage volume.

Making yourself dependent on third parties does not necessarily appear to be the best strategy either. We know this from recent years. Examples include systems and platforms from Google, Microsoft, Meta and Apple.

Data accuracy, mentioned in the contribution title: The obligation to keep or process personal data correctly arises, for example, from Article 18(1) GDPR. If the correctness of such data is disputed or its processing is not or no longer justified, a person affected can request a restriction on processing.

But even for data that has no personal reference, its accuracy is often fundamental. For example, if a lawyer includes a chatbot's response in their pleading, it would be better if what is written there is correct and stands up to scrutiny.

It would also be fatal for research projects or patent specifications if incorrect instructions or incorrect findings were to be incorporated into the work. The transfer of critical data to third parties is also legally risky. Data can be critical if, for example

• secret,
• protected by copyright,
• personal or
• Confidential

are.

Inculcate factual knowledge

The reliability of a text-processing AI can be significantly improved by explicitly pushing important facts into the electronic brain. This is often referred to as grounding. Microsoft supposedly uses this for its search engine Bing, but it fails, as numerous examples show. When done correctly, grounding helps to eliminate widespread misinformation.

After our company-owned AI was brought up to speed, it also provides better answers than ChatGPT, specifically for knowledge that is relevant. Who Goethe was, should not be something our own AI system needs to know. This unnecessary additional knowledge only burdens a AI model and leads to false statements, at least to less performance ("the AI no longer sees the forest for the trees") or slower responses.

Here is an example of an answer that was generated from the Dr. GDPR knowledge base and is qualitatively much better (and above all correct) than the results from Microsoft Bing and ChatGPT:

Searchable corporate knowledge is characterized by the fact that it is usually reasonably manageable, available in the form of documents and whose statements are a priori considered correct (or more correct than external sources). A dedicated AI search engine for corporate knowledge has several advantages.

Firstly, the company's own AI is autonomous and not reliant on others. The question of a data protection impact assessment (DSFA) hardly or not at all arises here. Secret or confidential data are well stored in one's own systems.

Secondly, a self-sufficient company AI can be adapted to your own needs. If you look at the program example above, you may notice the following things:

1. With Python, AI systems can be built in just a few lines of code.
2. The complexity is not in the program, but in the AI model, which only needs to be set up once.
3. A small step is enough to make a big difference.

Economic applications with significant added value are possible in this way. Own AI systems are even often cost-effective than using the ChatGPT interface, apart from the mentioned disadvantages when using a third-party system.

Conclusion

Artificial intelligence is overestimated or misjudged by many, probably due to the justified enthusiasm for the new possibilities. General intelligences such as ChatGPT are suitable for precisely this purpose: To answer general queries, and without any claim to reliability or even correctness. This can be seen from the user guidance of such systems alone, which contain no indication that the results should be checked carefully or that there may be an incorrect answer.

Companies that want to use AI must first clarify one question: What exactly should this AI be used for? If this question cannot be answered, then one thing is certain. This company either does not need an AI system or does not yet know what for.

The private use of ChatGPT and similar systems is certainly exciting, but something completely different from the use of chatbots or AI systems in general in companies.

If a company has a problem that can be solved with the help of AI, then it is worth checking whether this problem can be solved with the company's own AI system.