Sovereign AI: your documents never leave
Where do your documents go when you use a cloud AI? A lawyer, an accountant or a clinician needs them to stay on their own machines. Here is why and how.
The essentials in 5 points
- The real question for a regulated professional is not “which model is best” but “where does my data go?”.
- With cloud providers, that question has no satisfying answer: the moment data leaves the machine, it escapes your control.
- Two independent safeguards: keep inference local, and redact personal data before any possible transmission.
- You no longer have to sacrifice quality for confidentiality: a consumer computer with 128 GB of memory runs frontier-level models.
- The right architecture is local-first: most queries are handled on site, and only complex cases go to the cloud, once the data is masked.
- Local AI means no longer paying per request: the hardware is bought once, each request costs almost nothing, and you stop paying twice, in money and in data.
The short answer
For a lawyer, an accountant, or a clinician, the question that matters is not which AI model performs best. It is where the documents entrusted to them go.
A client file, an accounting record, a medical report: the moment this data leaves the machine to be processed, it enters a provider’s chain. The professional loses direct control over what protects it.
Sovereign AI approaches the problem the other way round: instead of promising that “it is secure”, it guarantees that the data goes nowhere, which the architecture itself makes verifiable. Two mechanisms make this possible: running the model locally, and redacting personal data before any transmission when part of the processing goes through the cloud anyway.
The rest of this article explains how these two safeguards fit together and what they change for professional secrecy and GDPR.
The question that embarrasses cloud providers
“Where does my data go?” Put to a cloud AI provider, that question never gets a fully reassuring answer for a professional bound by confidentiality. The provider will document encryption in transit, the location of its servers, its contractual commitments. All of this is useful, but none of it changes the central fact: the data has left the machine.
Once gone, it depends on a chain the professional does not control. Recent incidents and legal proceedings involving the major AI services remind us that the content of exchanges can be retained, analyzed, or shared far more widely than the user imagined.
For a law firm or a healthcare professional, this risk is anything but theoretical: professional secrecy is directly at stake. Confidentiality promised by contract remains a promise. Confidentiality that rests on the data never leaving is of a different nature.
Two safeguards to take back control
Two mechanisms, independent and combinable, guarantee that data stays under control.
The first is to keep inference local. The model runs on a machine or an appliance the organization owns, within its walls. The data never passes through a third-party server. It is the strongest confidentiality guarantee, because it rests not on a promise but on a physical impossibility: data that never leaves cannot leak.
The second is to redact personal data before transmission. In cases where part of the processing goes through the cloud, names, addresses, national ID numbers, and other identifiers are masked upstream, before anything leaves the machine.
The remote model then sees only anonymized text. This filtering makes an occasional use of the cloud acceptable without exposing the people concerned.
The local machine or appliance
The idea of an AI running within the company’s walls long ran into one objection: local hardware would be too weak to run a serious model. That objection no longer holds.
A market study confirms it: consumer hardware with around 128 GB of memory runs frontier-level models locally, the ones that rival the best cloud services. The principle is a dedicated machine or appliance, installed on site, on which the model runs without the data ever going out over the network.
For the organization, this changes the nature of the offer. Local AI is no longer a fallback accepted out of caution: it offers comparable processing power, with confidentiality added. The professional no longer has to choose between working with a good tool and protecting their files.
Local-first, cloud as backup
All-local or all-cloud is a false dilemma. The right architecture is local-first, with a fallback to the cloud for the minority of queries that justify it.
The share of queries that can run locally varies with the business use case, hardware, and required quality level. Compatible queries never leave the machine; more complex cases can be routed to a more powerful cloud model, but only after personal data has been masked. This split must be validated on a real sample, not assumed.
This combines the confidentiality of local processing on the vast majority of cases with the power of the cloud on the minority that genuinely needs it, without ever exposing identifying information.
This architecture has a double benefit. It protects data by default, and it reduces dependence on a single cloud provider, whose service, pricing, or terms can change.
No longer paying per request
Cloud AI is pay-per-use: every request consumes billed tokens, and cost tracks adoption. The more your teams use it, the higher the bill. This model makes you pay twice: once in money, once in data, since your content goes off to feed a third party’s chain.
Local inference reverses that equation. The hardware is bought once, the marginal cost of a request comes down to electricity, and usage can grow without the bill following. As for your data, it no longer serves as currency. AI becomes equipment you own, not a subscription.
What it changes for professional secrecy and GDPR
For a regulated profession, local inference shifts the compliance burden. If the data never leaves the premises, the question of transfer to a subprocessor, let alone outside the European Union, no longer arises in the same terms. Professional secrecy is no longer protected by a contractual clause but by the technical architecture itself.
On the GDPR side, the logic is identical: the less data circulates, the less exposed it is. Redacting personal data ahead of any cloud call reinforces this principle of minimization.
One caveat is in order, however. Local does not mean automatic compliance: you still have to secure the machine, document the processing, and keep in mind that a local model is not audited by default.
Security rests on the infrastructure and the procedures, not only on the choice of model.
For the hands-on technical side (processing pipeline, building blocks), see also Confidential AI: Data Stays On-Premises.
Where to start
The right entry point is a scoping exercise: which data is genuinely sensitive, which AI uses touch it, and what share of the processing can stay local. This review leads to a target architecture sized for your needs and budget, with a progressive migration plan.
Colombani.ai designs this kind of sovereign architecture (local inference, personal data redaction, controlled hosting), specifically supports regulated professions bound by confidentiality, and trains teams in these practices through the Local AI course. Colombani.ai is a Qualiopi-certified training provider, and OPCO funding can be requested, subject to the funder’s decision. Its founder holds the Claude Certified Architect Professional certification, issued by Anthropic.
Sources
Frequently asked questions
What is sovereign AI?
Sovereign AI is an architecture in which your data stays under your control and does not go to a third party. In practice it rests on two safeguards: running the model locally, on a machine or an appliance you own, so the data never leaves your premises; and redacting personal data before any transmission, in cases where part of the processing still goes through the cloud. The goal is not to promise that 'it is secure' but to guarantee, by design, that the documents never leave.
Why don't cloud providers answer the question 'where does my data go'?
Because their model relies on sending data to their servers for processing. They can document encryption, server location, and their contractual commitments, but the moment data leaves your machine, it escapes your direct control and depends on the provider's chain. For a professional bound by confidentiality, that dependency is precisely the point that causes the problem.
Do you have to choose between confidentiality and model quality?
No. Hardware with enough memory can now run local models capable of many professional tasks. The right architecture is not necessarily all-local or all-cloud, but local-first: compatible queries stay on premises and more complex cases fall back to the cloud after personal data has been masked. The actual share must be measured on real usage.
Is local AI compatible with GDPR and professional secrecy?
It makes compliance easier. If the data never leaves your premises, the question of transfer to a subprocessor or outside the European Union no longer arises in the same way, and professional secrecy is preserved by the architecture itself. One caveat: local is not automatic compliance. You still have to document the processing, secure the machine, and keep in mind that a local model is not audited by default.
Who is local AI architecture for?
First and foremost, regulated professions and organizations that handle sensitive data: law firms, accountants, healthcare professionals, but also any function processing personal data at a volume that makes the risk of a leak unacceptable. As soon as the question 'where does my data go' has an answer that engages your liability, local inference is worth studying.