Offline-AI Showcase: Digitization of Documents

Document digitization works very well with Offline-AI. Offline-AI is locally running AI that is often better than ChatGPT, data-friendly, and cost-effective. This includes recognizing text and images as well as semantic search within these extracted information. The showcase shows concrete details.

What is Offline-AI?

Some might understand "Offline-GPT" better. Offline-AI, however, has nothing to do with OpenAI and other third-party providers.

An Offline-AI runs on its own computer. This can be either purchased hardware or rented hardware. Offline means that the AI does not send data to third parties. The Offline-AI can access the internet if needed or communicate with other IT systems.

Offline-AI can produce significantly better results for many use cases, such as for the digitization of documents, than with ChatGPT and other cloud services. In companies, it often involves thousands of documents. The costs with cloud services are often incalculable and also expensive for many requests. Offline-AI offers an affordable cost flat rate. Full data control is also a reason for many not to use ChatGPT or Microsoft Azure.

What does digitization of documents mean?

Digitalization means the transformation of analog into digital information. Often, this involves converting paper documents into digital images (files). To do this, the paper document is scanned or photographed. Afterwards, the resulting image is evaluated (even when scanning, an image is created!).

Using the example of a document from the European Data Protection Board (EDPB), it is shown how Offline-AI can help with the digitization of documents.

The images shown above represent the pages of a PDF document. These images are created either by scanning or by converting a PDF document into individual pages.

After the document has been scanned (or photographed), it is evaluated with Offline-AI. In this process, the text content of the document is determined. Further procedures also recognize images and their content.

With Offline-AI, even images can be described. Here is a screenshot of a leaflet on the topic of Offline-AI.

The Offline-AI now had the task of describing what the image depicts. Here is the result:

a black and white drawing of a man with horns, ikea manual, as a d & d monster, a an ai generated image

The German translation is also provided by the Offline-AI upon request:

A black and white drawing of a man with horns, an IKEA manual, as a D&D monster, an AI-generated image

For those who need the Ukrainian, Turkish, Spanish, Italian, or Polish version, Offline-AI can also be of help:

• Ukrainian: Black and white drawing of a man with horns, IKEA instruction manual, as a D&D monster, and an AI-generated image
• Turkish: bir adamın kulakları olan siyah ve beyaz bir çizim, ikea kılavuzu, d&d canavarı olarak, bir ai oluşturulmuş görüntü
• Spanish: Un dibujo en blanco y negro de un hombre con cuernos, manual de IKEA, como un monstruo de D&D, una imagen generada por inteligencia artificial
• Italian: disegno a matita nero e bianco di un uomo con corna, manuale Ikea, come mostro D&D, immagine generata da AI
• Polish: rysunek czarno-biały mężczyzny z rogami, instrukcja IKEA, jako potwór D&D, obraz generowany przez AI

The translations were verified with the previous gold standard, DEEPL, and are reproduced here unchanged.

The next step could be the recognition of sections/blocks.

The blocks shown in the illustration were automatically detected and marked. They serve as a precursor for powerful recognition of text and image information.

The following illustration shows how much information can be contained in such blocks.

The displayed text excerpts were automatically detected. The user now has several options available. Information can also be found in the flow text as well as with strict search. Strict search only returns hits for sections that each contain the entire search term. Instead of a search term, questions can also be asked to the document. For comfort reasons, the user only sees his search mask (input field) and the results at the end. He only sees the above shown images on request.

Query your own documents: With Offline-AI, not only better than ChatGPT, but also cheaper and with full data control.

Furthermore, it is also possible, for example, to find semantically similar pages to a given document page.

In this example, the pages that are visually similar to a given page (page 1, top left) were found. Visual similarity in this example exists if the gray-shaded boxes appear on other pages. This is the case for pages 3 to 8 (from left to right, from top to bottom). As a counter-example, page 2 was displayed as a visually dissimilar text page.

Semantic searches can also be performed on text. Document and page searches can be performed as powerfully with Offline-AI. For example, a search for "personal data" was performed in the aforementioned digitized PDF document.

Some of the hits can be seen here:

Of course, the Offline-AI can display the hits directly as text. Only for the example, the hits were shown here in the form of page screenshots.

A detail hit for this search query is shown here:

Similar matches were also found without further effort, which mean the same thing but use a different expression. The ambiguity between "person-related" and "person-related" was automatically resolved by the AI. This very simple example can be made nearly arbitrarily powerful.

A powerful example for semantic search is a question-answer assistant for the Dr. GDPR Blog described.

Similarity search

How would it be if you could find the semantically most similar images to a given image? A bear is a bear, a cat is a cat. Regardless of whether the respective animal (or object, if other images) is large or small, on the left or top of the image, or whether only the head or a full shot is visible.

For document pages, the result is similarly impressive:

On the right side of the image is the page from a PDF document for which similar pages should be found. On the left side of the image are the pages that show optical similarity. The similarity here lies in the text flow, but above all in the gray-shaded block. If images were visible in the original image, they would be taken into account. Instead, one could also have found similar documents based on the text visible in the image. The possibilities are endless.

To conclude, a brief example that information can also be recognized in more demanding images.

The input image is as follows. It was taken with an old mobile phone, at low resolution and in poor lighting conditions:

The untrained Offline-AI recognized, marked, and extracted the following information in a three-quarters of a second on a laptop:

The core data was recognized and its position returned. This allowed the serial number 49865 to be recognized correctly and the number A055247 to be recognized almost correctly (the "A" was recognized as "4", which is hardly noticeable even to a human).

As can be seen, a few entries are unrecognized. There are several solutions for this:

• Automatically rotate image and perform recognition again
• Semantic Comparison of Letters and Digits with Offline-AI and Once-Trained KI Model
• AI model trained with examples of tire photos
• If there are too few training examples available: Synthesize arbitrarily many examples with Offline-AI + conventional methods (noise, image rotation, quality reduction, …)

It is therefore not only possible to digitize text documents (which can also contain images) with Offline-AI. It is also possible to automatically evaluate photos. This is particularly interesting for insurance companies. The practice examples collected to date, often hundreds of thousands of them, can be used as reliable training data for an Offline-AI system. If too few training data are available, there is the possibility of artificially generating such training data. Offline-AI is also used for this purpose and is already looking forward to running on your server for hours while you enjoy your evening or the weekend.

Conclusion

Offline-AI can digitize documents of various types. The scanned or photographed documents are analyzed with AI for this purpose. Information from text and images is extracted with AI. For example, the extracted information can then be semantically searched, summarized, translated into simpler language or other languages with AI.

A similarity search with images is also possible: To an input image, the most similar images are found. And this from a semantic point of view and not like "earlier" by comparing image pixels.

The only thing still needed for digitization is a good scanner or a mobile phone with a camera, depending on the application.

Offline-AI keeps the data where it belongs, namely in your company. In addition, Offline-AI offers the possibility of accessing data from the internet or communicating with your other IT systems.

The results are significantly better for many use cases than ChatGPT could ever be. Multilingualism is also no problem, even with company-specific dictionaries. Technical terms from the insurance, medical or legal professions can thus be adequately considered.