> Bron: https://neuralex.nl/en/blog/ai-in-de-zorg > Health data is a special category of personal data under GDPR. That's why on-prem or a tightly controlled private environment is usually the most defensible architecture for AI working with patient records — not an absolute ban on cloud AI, but a much higher bar. [Back to Insights](/en/blog) Healthcare ·29 August 2026 ·8 min read # AI in healthcare: why on-prem is almost always the answer Health data is a special category of personal data under GDPR. That's why on-prem or a tightly controlled private environment is usually the most defensible architecture for AI working with patient records — not an absolute ban on cloud AI, but a much higher bar. For AI systems that process real patient information or medical records, on-premises deployment or a tightly controlled private environment is often the most defensible architecture. The reason is not that cloud AI is prohibited in healthcare. It is that health information is a special category of personal data under the GDPR, so sending it to an external AI provider introduces another processor, infrastructure layer, potential subprocessors, retention policies and sometimes international data transfers into an already sensitive processing activity. On-premises AI reduces that exposure. The language model, retrieval system and supporting databases can run inside infrastructure controlled by the healthcare organisation, without routinely transmitting patient records to a general-purpose external AI service. That does not automatically make a system GDPR-compliant or secure, but it gives the organisation substantially greater control over where data resides, who can access it and how it is processed. ## Health data is not ordinary personal data Medical records almost inevitably contain information about a person's health. Under the [GDPR](/en/blog/ai-en-de-avg), health data belongs to the special categories of personal data and receives additional protection. Processing generally requires both an appropriate legal basis under Article 6 and a valid condition under Article 9, with national healthcare legislation potentially adding further requirements. A healthcare provider being permitted to process patient information for care purposes does not automatically mean that it can send the same information to any new AI service. Purpose, necessity, data minimisation, processor relationships, security and possible secondary uses must still be assessed. This is why casually pasting medical records into a general consumer AI account is difficult to justify as a default working practice — see also [are you allowed to run customer data through ChatGPT](/en/blog/klantgegevens-door-chatgpt). For individual ChatGPT services, OpenAI states that content may be used for model improvement unless the user opts out. Its business offerings and API are treated differently: OpenAI says business inputs and outputs are not used for training by default and offers a Data Processing Addendum. Those contractual and technical controls may be relevant, but they do not themselves create a lawful basis for a healthcare organisation to process health data. ## What on-prem AI actually means In its strictest form, on-prem means that the model runs on infrastructure operated by the healthcare organisation itself. A local stack might contain a language model, vector database, retrieval layer and application interface. Private hosting can be slightly broader. An organisation might use dedicated infrastructure operated by a carefully selected hosting provider while retaining strict contractual and technical controls over data location and access. That is not identical to running servers inside the hospital or clinic, but it can provide much tighter data governance than a general public AI service. A local RAG architecture is particularly useful here. Medical records do not have to be used to train the language model. Documents can remain in an internal repository and be indexed locally. When a user asks a question, the retrieval layer finds the relevant passages and provides only those passages to the model. On-prem deployment is not a substitute for security engineering. Access control, encryption, network segmentation, audit logging, patching, backups, retention controls and export restrictions are still required. A poorly secured local model with unrestricted access to every medical record is not safe simply because no external API is involved. ## Where AI can add practical value Record summarisation is an obvious use case. AI can turn a long patient history into a draft overview of relevant history, recent events and unresolved issues. A healthcare professional should still compare that summary with the underlying source material. Documentation is another candidate. AI can structure notes, remove repetition and prepare draft reports or handovers. The useful part is reducing administrative work, not allowing the model to invent clinical facts that were never present in the record. Retrieval systems can also make protocols, internal procedures and clinical guidelines easier to search. Rather than asking staff to browse large document repositories manually, a RAG system can retrieve relevant passages and link the answer back to the original source. In healthcare, verifiability is usually more important than how fluent the generated answer sounds. Triage support moves closer to clinical decision-making. A system may help organise symptoms, highlight information or present possible issues for a clinician to investigate. That is fundamentally different from allowing the model to independently determine a diagnosis or treatment. For clinical decisions, meaningful human oversight should be part of the system architecture rather than an afterthought. ## On-prem comes with operational costs The strongest argument against on-prem is straightforward: it is harder to operate. A cloud API can often be integrated rapidly. Running an internal AI platform requires hardware or dedicated compute capacity as well as expertise in model deployment, security, monitoring, upgrades, capacity planning and incident response. New models also appear frequently, while an organisation running its own stack has to evaluate and deploy upgrades itself. Local models may also be less capable than the strongest cloud-hosted models for particular workloads. Large context requirements or complex reasoning tasks can make that difference relevant. The economic comparison should therefore not be reduced to API charges versus GPU costs. It also has to include the value of controlling sensitive information and the potential consequences of inadequate processor oversight, misconfigured retention, unexpected subprocessors or accidental disclosure. The Dutch Data Protection Authority explicitly recognises that health data can be processed in the cloud, but stresses that the organisation remains responsible. Its 2026 practical guide "Health data in the cloud" highlights processor and subprocessor relationships, risk assessment, data sovereignty and international transfers among the issues that organisations need to understand. Moving infrastructure to a cloud provider does not outsource accountability. ## When AI may become a medical device Not every AI application used in healthcare is automatically a medical device. Under the EU Medical Device Regulation, the software's intended purpose is crucial. General-purpose software does not become a medical device merely because somebody uses it in a healthcare environment. The situation changes when software is specifically intended for purposes such as diagnosis, prediction, monitoring or treatment. MDR Rule 11 contains specific classification provisions for software that provides information used to make diagnostic or therapeutic decisions, with the classification depending partly on the potential impact of those decisions. An AI tool that helps draft a discharge summary therefore raises different regulatory questions from a system whose output is intended to guide treatment selection. Once an application moves towards diagnosis or treatment decisions, medical-device qualification should be considered during system design rather than shortly before deployment. ## Human oversight must be real Putting "AI may make mistakes" underneath an output is not meaningful human oversight. Users need enough context to verify what the system produced. For summarisation, the underlying record should remain accessible. A protocol assistant should point to the passages supporting its answer. A clinical support system should make clear which information it relied on and where uncertainty remains. Human responsibility also becomes largely theoretical when an interface presents AI output so confidently that staff routinely accept it without review. Human-in-the-loop therefore needs to be implemented technically and organisationally, not merely written into a policy document. ## Why "almost always" is deliberately qualified On-prem is not necessary for every healthcare AI workload. An application searching only public medical guidelines may never process patient information. Cloud AI may also be appropriate where data has been genuinely anonymised so that individuals are no longer identifiable. Pseudonymisation is different. Replacing a patient's name with an identifier reduces exposure, but pseudonymised information remains personal data under the GDPR where re-identification remains possible. It is an important safeguard, not a route around data-protection law. Even identifiable or pseudonymised health information may in some circumstances be processed using a carefully configured business cloud service. That could involve an appropriate processing agreement, strict security controls, clear subprocessor and retention arrangements, suitable data-location measures and a demonstrable legal basis and Article 9 condition. Whether that is appropriate depends on the specific processing operation. ## Conclusion For AI systems working directly with medical records and health data, on-premises or tightly controlled private deployment is often a sensible architectural default because it gives healthcare organisations greater control over their most sensitive information. The trade-off is higher infrastructure, management and expertise requirements. Cloud AI is not categorically prohibited, but convenience alone is not a sufficient reason to send health data to an external AI provider. This area intersects with the GDPR, national healthcare law, the MDR and potentially other regulatory requirements. Neuralex does not provide legal advice. The proposed architecture, lawful basis, processor arrangements and any potential medical-device qualification should therefore be reviewed by qualified legal counsel for the specific use case. On-prem AI for healthcare ## Considering an AI application that works with patient data? We design on-prem and private AI architectures that keep health data inside your own environment, including RAG, logging and human oversight. Curious what that looks like for your situation? [Ask your question](/en/contact) [Read on-prem AI: when is it worth it](/en/blog/on-prem-ai-wanneer-de-moeite) --- Volledige (opgemaakte) versie: https://neuralex.nl/en/blog/ai-in-de-zorg