Skip to content
CPLT
Request a free call
Menu
All posts
ocrself-hosted-aicompliancedocuments

Use AI on Your Documents Without a Third Party

A local model alone does not keep documents in house. What has to run on your own systems, from OCR to backups, and how I check each step.

Stéphane Lepain··Updated ·6 min read

You can use AI on your own documents without sending them to a third party if every step that touches them runs on systems you control: the model, the text extraction (OCR), the embeddings, the search index, the logs and the backups. An open-weight model on your own server covers the first step. The other steps are separate, and any one of them can still send a copy out. So run the model locally, then check where each piece of the document goes.

What has to run on your own systems?

Self-hosting a language model does not establish where documents are processed. OCR, embeddings, logs, storage and backups can use different services. I map the complete path before describing a document workflow as private.

I check which parts of the actual workflow are local and which use external providers.

For each stage, record what information it receives, where it is processed, what persists and who can access it. Include extracted text and derived indexes, not just the original PDF. An approved external provider may be appropriate; it needs to be an explicit decision. See data-processing responsibilities.

Which models can run without a third party?

Open-weight models such as Llama, Mistral or Qwen run on hardware you own or a server you control, served through tools like Ollama, llama.cpp or vLLM. The prompts and documents then stay on that machine. You carry the hardware and the work of running it, and quality against the hosted frontier models varies by task.

A mix also works: a local model for the documents that must stay in house, a hosted API for the rest, with the rule written down. I compared the costs in buying an AI server vs paying for API tokens and the wider choice in what a private AI stack is.

Does a document have to stay only in memory?

The processing must follow the agreed data boundary and retention requirements. There is no universal requirement that all document bytes exist only in RAM, and choosing RAM does not prove that no copy persists elsewhere.

Temporary files, swap, crash dumps, logs, queues and backups may all matter. Verify the actual configuration and failure behaviour. A successful deletion response or empty query is evidence about that interface, not proof that every copy is gone.

Can OCR run locally for scanned PDFs?

Yes. Open-source OCR engines run on your own machine, but each one needs testing on your own pages.

I test representative scans, text pages, tables and diagrams before choosing an extraction method. Compare the extracted text and reading order with the visible page, retaining enough source references for a person to check the result.

Different extraction tools can fail differently. A long output is not proof that the page was read correctly, and model confidence is not a substitute for comparison with the source. Mark missing or uncertain content for review.

For available capabilities and licences, consult the current Tesseract documentation and the documentation of any other proposed engine. A particular model, GPU or language configuration needs its own acceptance test.

Where are embeddings computed, and where is the index stored?

These are two separate places, so check both. The embedding step sends text to whichever model turns it into vectors. The resulting search index is kept in a database that is configured on its own. In LibreChat's RAG API, for example, the embeddings provider is one setting (OpenAI by default, with Azure, Hugging Face or a local Ollama model as options) and the index sits in a PostgreSQL database with pgvector. A local database does not stop text going to a remote embedding provider, and a local embedding model does not put the database on your systems.

An external embedding service receives the text sent to it. Local embedding is an option when required, but it still needs access controls and a defined retention policy.

Check isolation using users who should and should not have access to each document. A project label or separate collection is a design component, not proof that all access paths are protected. Include retrieval, exports and administrative interfaces in the agreed checks.

Treat indexes and chunk metadata as derived information that also needs protection. Do not assume that deleting the source PDF removes its searchable content.

How do you know a document is really deleted?

Document what must be removed, when, from which active stores and under what backup policy. Define what evidence the deletion procedure can produce and what it cannot establish.

Test the procedure on synthetic documents, including interrupted jobs and failed cleanup. Record outstanding copies or retention constraints rather than promising immediate universal erasure.

What does this buy you?

The aim is a workflow whose processing locations, access rules, review responsibilities and running costs can be explained and checked. It is not an automatic compliance certification or a guarantee of perfect extraction.

Compare the effort of checking the result with the current manual task. Include extraction, embedding, hosting, human review, correction, maintenance and support.

What are the limits?

Speed, memory needs and extraction quality depend on the inputs, tool versions and operating setup. I do not quote a universal local-versus-cloud speed ratio or a fixed delivery time before scoping the work.

A pilot should include difficult pages, missing information and cases that need a person. Agree the document types and acceptance criteria before building. Describe your document work without sending confidential files in the first message.

Questions people ask about keeping documents in house

Is ChatGPT on a business plan a third party? Yes. The documents are processed in OpenAI's systems, under its contract and controls. That can be acceptable for some work, but it is a third party processing them, not your own systems.

Why does OCR matter if the model is local? It is a separate processing step with its own data destination and retention behaviour.

Must everything run locally? That depends on the agreed requirements. Every external route needs permission and appropriate provider review.

Can a small business do this without an AI team? Yes, for one recurring task: one interface, an open-weight model for the documents that must stay in house, and written checks for each step. Someone still has to run the updates and the backups.

At CPLT, I build internal knowledge assistants for teams that need document answers with sources and agreed data boundaries.

Revised 14 October 2026 to answer the question in the title directly and to separate where embeddings are computed from where the index is stored. The previous title was "OCR and Scanned PDFs: Map the Whole Document Path". This article describes a general design and test method. No private operating result, delivery time or performance guarantee is claimed.