Offline-AI as an Opportunity for Companies of All Sizes

An Offline-AI often delivers better results than the best AI models, full control over data, and is in many cases significantly cheaper than solutions with ChatGPT or Microsoft Azure. What is an Offline-AI and what opportunities does it offer?

The Future of Artificial Intelligence

This post describes a way for companies to achieve very good results with their own AI system and also have full control over their own data. Even with intensive use of this company AI, costs remain low.

This form of AI is referred to here as Offline-AI.

Advantages of Offline-AI

Capability

An Offline-AI is characterized by its performance capabilities. It can process and analyze large amounts of data to gain valuable insights. This allows companies to make better decisions and optimize processes. Through the combination with conventional methods, the AI can also solve complex tasks and make predictions. Especially challenging problem statements can be solved by adding procedures from Machine Learning.

Example:

A company participates in tendering and has collected a series of data from previous tenders. For the next tender, the company now wants to know which bid price will likely lead to an award.

This type of problem can be solved much better by an Offline-AI than by general intelligences like ChatGPT. The reasons are particularly:

• Specific task description that cannot be solved universally (and if it ever can, then humans will no longer be the most intelligent life form on this planet). .
• Forms of text and terms specific to your company and industry.
• Large amounts of data need to flow into the AI. This raises the question of costs with cloud services like Microsoft Azure or the ChatGPT API.
• Possibly: Need for confidentiality of your data or privacy of your data.

Rule of thumb: The high performance of Offline-AI comes into play above all in application cases that do not seem as trivial as entering a text or fewer texts into a chatbox.

Data control

An Offline-AI protects the data that belongs to you. This can be patent applications or trade secrets. Often, confidentiality agreements have also been made for certain data. This data should better not end up with American companies (Cloud Act, EO 12333, FISA 702). Some customers do not want their data, which is given to you, to end up with third parties such as Google, Microsoft, Meta or OpenAI. All these companies use user data to train their own AI. .

An Offline-AI holds your data. That's why this AI is called that: it can run without an internet connection in principle. This means all data streams are fully controllable. You decide which data flows where. Of course, the AI system has access to the internet if necessary. For example, when videos should be automatically downloaded for the AI or when the AI should conduct internet research.

Thus many legal questions are resolved. Contracts for data processing with American corporations no longer need to be concluded and no longer legally evaluated. Opt-outs due to the use of input data by the AI provider are eliminated.

Because an Offline-AI does not have to reproduce previously read content, problems with copyright or the GDPR can be avoided. A generative AI like ChatGPT, which is not "gated," is on the other hand almost a guarantee of copyright infringement.

Affordable costs

A company-owned Offline-AI is not only powerful and secure, but also cost-effective. Once set up, it doesn't matter if your AI system runs only 3 hours a week or 24 hours a day and every day of the year with full load.

Someone who has looked at the costs of cloud platforms like Azure might find this cost aspect particularly appealing.

Try Offline-AI now

Optimizable and with full data control. Economical even in continuous operation.
Fully-controlled data center, no third-parties.

Try now

Even the use of the ChatGPT-API raises questions about costs, although it can be described as low-priced. Because an AI is somewhat vague, it is not clear a priori how often and with what data material the AI must be called up for a desired result. Therefore, it doesn't matter whether 100 calls cost 1 euro or 10 euros. The uncertainty about the costs alone is a deterrent.

If a system has to process many data, such as your company's documents or all Hessian laws, then frequent calls to a very favorable API will quickly become expensive and unplannable.

From a cost perspective, another advantage of Offline-AI for businesses emerges.

Innovation power

Innovation is a process. A good process runs as continuously and whenever necessary.

A 24/7 operation is the perfect prerequisite for innovation. 24/7 means that the company's AI is always and fully available, and works best permanently.

Technically, all cloud services can be used permanently. However, the costs are often outrageously high and often unpredictable. Besides your money, you also hand over your data to the cloud provider, who doesn't even say thank you for it.

An Offline-AI always costs the same amount, regardless of how often it is used. The costs essentially amount to:

• Hardware Procurement: Either rental or purchase (own data center or colocation, thus integrating own servers into a foreign data center). A professional AI server is already available for 5,000 euros. When renting, costs of under 1,000 euros per month arise. For some AI application cases, the costs for server rental are even negligible.
• Installation System: This one-time expenditure depends on your use case. It is manageable. If you rent a ready-made AI server, costs are even lower.
• Programming AI: Depending on your application case. Often very efficient possible. Please see the following example: The author of this post was interviewed for a 3sat documentary. For the doc, the author programmed an AI system within few hours, which can identify and mark burglars in surveillance videos. See 3sat-Documentation from minute 33:16.

A few examples of innovative projects that are possible with Offline-AI:

• Document Search: Documents can also be support tickets or customer emails. Find the best matching documents for your search query. The search can also be formulated as a natural language question.
• Find and evaluate global knowledge: The internet is searched based on your search query through a background process, with the goal of retrieving more hits than are relevant. In the next step, these often 1000 hits are given to an Offline-AI. In a multi-stage process that can be easily programmed, irrelevant hits are first sorted out. Then follows the crawling of the remaining hits. The full results are read out and given back to the Offline-AI, which then analyzes the global knowledge texts according to your specification. Examples:
  • SEO Analyses
  • Extract addresses
  • Analyze competition
  • Determine current innovations
  • Recognize trends
  • Construction projects
• Translation in other languages: As a side effect you get a powerful translator. For perfection you should use DEEPL etc. If however very good (!) results are sufficient, then you get these for free and in any quantity with an Offline-AI gift. Think of manual or AI-generated work results that a colleague in Spain, France or from Ukraine receives and is supposed to understand.
• Audio transcription: Converts language from videos or podcasts into text. Works better than any available AI solution. The Offline-AI uses its own dictionary. It can automatically download and transcribe an unlimited number of video or podcast episodes. Prompt tuning is not required.
• Generate Images: Unlike services like Midjourney or DALL-E, your Offline-AI can generate thousands of images. Personal styles and automatic optimization of your image prompts are possible. Things like copyright checking are also well integratable.
• Object recognition: Either you want to recognize specific objects on videos or images. It's also possible to find production errors in the manufacturing process. For this, a one-time training of your Offline-AI with your production part images is important. You can already see here that this would be difficult with a cloud solution. Especially since a specific optimization seems sensible and can save considerable costs.

However, the most important aspect that transcends all concrete issues is another.

Self-improvement through AI self-critique

With an Offline-AI, even a self-improvement of your AI system is possible.

To understand this, you have to know that AI systems are nearly as unreliable as humans themselves. Even highly intelligent people are dumb in many fields of knowledge or areas of life. Take Albert Einstein for example. If he were still alive and you asked him how best to treat wood for the outdoors, you might not get a good answer.

Also, one must assume that the answer, for example, of an AI language model is not good enough. However, it is sufficient if the AI response is half-decent, possibly even (still) wrong. .

Let's take Knowledge Extraction from any texts as an example. Such texts can come from documents in your company or authority, or be found on the internet.

What's particularly charming is extracting knowledge from the Internet. The treasure trove of data is endless. All that's needed for this is an interface to a search engine. Such an interface exists, and it does so in just as cost-effective a way as the Offline-AI itself offers.

Therefore, interesting content from the internet could theoretically be continuously harvested. This content could then be continuously analyzed by an Offline-AI. The Offline-AI would continuously correct its own answers if necessary. The results could then be continuously translated into other languages.

To illustrate this, here is an example where key statements are to be extracted from a document. The AI delivers, for example, a list with 5 key statements about a document (from the internet or your company). A key statement extracted by the AI for the Dr. GDPR-Blog article Cookies are not text files reads:

"The document describes why cookies are not text files and that cookies are instead data sets.

First version of a core use case extracted from the mentioned document using an Offline-AI.

This answer is correct, but cannot be used independently because it refers to the read document. What we want here are independent core statements.

Therefore, the core message to be extracted would be the question to the AI: "Check if the following answer is understandable on its own. If the answer is understandable on its own, respond with 'OK'. If the answer is not understandable on its own, rephrase the answer so that it is understandable on its own."

The Offline-AI then responds with a rephrased answer:

"Cookies are not text files. Cookies are data sets."

Second version of a core use case extracted from the mentioned document using an Offline-AI.

This core statement is correctly extracted and independently understandable. In the same way, one could extractAI-responses from the same AI

• Check for spelling errors and have the AI correct them itself
• translate into another language, (here real results, quickly generated with the same AI):
  • Cookies are not text files. Cookies are sets of data_
  • Cookies are not text files. Cookies are sets of data_
  • Cookies are not text files. Cookies are data sets._
  • Azerbaijani: Kukilər mətn faylları deyil. Kukilər məlumat dəstləridir. (DEEPL does not know this language! Cross-check was done with another AI language model)
• have it expressed in a simpler language
• have summarized
• to adapt the style and length to that of a social media post
• …

Also, multiple blog posts can be read and then compiled into one post. All for 0 euros.

Some answers could be obtained directly from ChatGPT, while the Offline-AI would require two prompts. The difference is that the Offline-AI allows for an additional 100 prompts per document at no extra cost, whereas usage-based platforms like those from OpenAI or Microsoft charge more for additional calls. The second difference is data security. The Offline-AI operates on your server, not sharing data with third parties who would be happy to have it and become increasingly powerful.

Let's summarize what self-improvement means:

• Your company's AI generates answers at a rapid pace.
• Your company's AI is self-evaluating ("self-critique")
• Your company's AI improves itself if needed ("Self-improvement")

This can be taken to extremes by Self-improvement of AI itself (not just individual answers). This is already possible, even on an AI-Laptop. For this, the result generated and improved by the AI itself is fed back into the same AI as a training dataset. The training datasets are then used to create a new version of the AI. This process is called Finetuning. The new, improved version of the AI improves itself further again. The process stops when there's nothing more to improve based on the available data and the given task or the AI model has reached its limits. The first case, availability of data or the task setup, is the more likely reason for the end of self-improvement. After "perfect" comes "?".

Another huge advantage of Offline-AI: you can combine entire process chains with it. Try commanding ChatGPT to read an entire website with 500 subpages, and then extract all relevant keywords, themes, and key statements from the contents that have been read. This already fails because OpenAI is not willing to scan "the entire internet" for your company for $20 a month. Even using the ChatGPT API for 500 more comprehensive documents (after you've read them yourself) is no fun, as the costs arise from the intensity of use.

Conclusion

An Offline-AI offers endless possibilities and opportunities for companies of any size. Meanwhile, AI language models are not only excellent for the English language but also for German, Spanish, Ukrainian etc.. In particular, smaller language models have these advantages.

So companies can operate their own AI systems in a very cost-effective and data-friendly way. These AI systems are worthwhile when it's not just about occasionally entering a prompt into a chatbox. The performance of one's own AI is often far higher than that of the best general AI systems in the world.

Offline-AI is worth it when it comes to many data or documents, when it's about intensive use, when it's about full data control, when it's about reading knowledge from the internet or (!) when your specific application case should be best possible solved with AI

Showcases for Offline-AI already realized (excerpt):

• Summarize and simplify legal texts
• Generate images (on AI laptop)
• Simplify Intelligent Communication
• Question-Answer System with reliable answers

In short, an Offline-AI is worthwhile for your company if you have a clearly defined problem. Feel free to ask for more information about the feasibility for your specific problem.