IMPLEMENTATION OF RAG ARCHITECTURE FOR AUTOMATED DOCUMENT VERIFICATION IN CORPORATE STORAGE SYSTEMS USING LARGE LANGUAGE MODELS
Abstract and keywords
Abstract:
A specialized solution is offered for analyzing documents used in corporate document management systems. Purpose: to develop and implement a RAG architecture for automated verification of corporate documents using locally deployed large language models that identify missing required fields, data format errors, and substantive contradictions in documents of transport and logistics systems. Methods: Java application was designed and implemented programmatically, integrating a text extraction module from PDF and DOCX documents based on Apache libraries, a vector storage with simplified embeddings based on frequency analysis of words, a semantic search algorithm through cosine similarity calculation and an LLM interaction client via the Ollama server API. Results: the developed system has demonstrated the ability to contextually analyze the content of documents and adaptability to variable information presentation formats, which makes it possible to overcome the limitations of traditional systems. The experimental verification was performed on a test set of corporate documents with intentionally introduced errors of various types; the effectiveness was assessed by the metrics of completeness, accuracy and their average, as well as by system response time. The llama3.2:latest model showed the best results, while completely excluding the transfer of confidential data outside the organization’s infrastructure. Practical significance: the proposed solution is applicable for automation of documentation quality control in corporate document management systems of transport enterprises, government agencies and industrial organizations. The modular architecture provides scalability to other types of documents and the ability to integrate with existing information systems at minimal cost of adaptation. Using open models and a local Ollama server reduces dependence on thirdparty cloud services and ensures compliance with information security requirements.

Keywords:
large language models, automated document verification, vector search, natural language processing, corporate systems, Ollama, text extraction, semantic analysis
Text
Text (RU) (PDF): Read Download
References

1. Rotman D. RAG i generativnyy II. Sozdaem sobstvennye RAG-payplayny s pomosch'yu LlamaIndex, Deep Lake i Pinecon = RAG-Driven Generative AI: Build custom retrieval augmented generation pipelines with LlamaIndex, Deep Lake, and Pinecone. SPb.: Piter, 2025. 320 s.

2. Java SE/JDK Version 26 API Specification: Module java.base. URL: http://docs.oracle.com/en/java/javase/26/docs/ api/java.base/module-summary.html (data obrascheniya: 30.01.2026).

3. Ollama’s Documentation. URL: http://docs.ollama.com (data obrascheniya: 02.02.2026).

4. Shildt G. Java. Polnoe rukovodstvo. 12-e izd. = Java: The Complete Reference. Twelfth Edition / per. s angl. i red. Yu. N. Artemenko. SPb.: Dialektika, 2023. 1344 s.

5. Horstmann K. S. Java. Biblioteka professionala. T. 1. Osnovy. 10-e izd. = Core Java. Volume I — Fundamentals. Tenth Edition / per. s angl. i red. I. V. Bershteyna. M.: Vil'yams, 2016. 864 s.

6. Portyankin I. A. Swing. Effektnye pol'zovatel'skie interfeysy. 2-e izd. M.: Lori, 2011. 607 s.

7. Huang D., Wang Z. LLMs at the Edge: Performance and Efficiency Evaluation with Ollama on Diverse Hardware // Proceedings of the International Joint Conference on Neural Networks (IJCNN 2025) (Rome, Italy, 30 June — 5 July 2025). Institute of Electrical and Electronics Engineers, 2025. 8 p. DOI:https://doi.org/10.1109/IJCNN64981.2025.11228317

8. Vahaj M., Raza S. M., Nehra V. Retrieval Augmented Generation (RAG) using LLMs // Proceedings of the Annual International Conference on Data Science, Machine Learning and Blockchain Technology (AICDMB 2025) (Mysuru, India, 27–28 June 2025). Institute of Electrical and Electronics Engineers, 2025. 5 p. DOI:https://doi.org/10.1109/AICDMB64359.2025.11277692

9. Bloh D. Java. Effektivnoe programmirovanie. 3-e izd. = Effective Java. Third Edition / per. s angl. i red. I. V. Krasikova. SPb.: Dialektika, 2019. 464 s.

10. Claim Verification in the Age of Large Language Models: a Survey / A. Dmonte [et al.] // ArXiv. 2024. Vol. 2408.14317. 9 p. DOI:https://doi.org/10.48550/arXiv.2408.14317

Reviews
1. review
Authors: Bubnov Vladimir

Login or Create
* Forgot password?