Air-gapped AI: can a language model run without the internet?

Yes: a language model can run fully air-gapped — physically and logically disconnected from any network, including the internal company network. The model weights live locally on disk, inference runs on your own hardware, and not a single packet of data goes out. No internet connection means no cloud API, so every part of the pipeline — the model, the knowledge base, retrieval — has to be present locally before a question is even asked.

So it can be done. The question that actually matters is different: do you need it? Air-gapped is the extreme end of a scale that starts with plain cloud AI and, via on-prem, ends at full isolation. For most businesses, the right answer sits somewhere in the middle — not at that far end.

Air-gapped is more than on-prem

The two terms are often used interchangeably, but they're not the same. On-prem means the model runs on hardware you manage yourself — a server you control, possibly still with an internet connection for updates, monitoring or the odd cloud service. Air-gapped goes a step further: no network connection, full stop. No outbound traffic, no inbound traffic, no telemetry that "happens" to phone home anyway. Every on-prem setup is therefore not automatically air-gapped, but every air-gapped setup is, by definition, on-prem.

Who actually needs this

Air-gapped AI isn't a default choice — it's an answer to a specific threat: an environment where even a controlled, encrypted connection to the outside world is an unacceptable risk. Think defence and intelligence agencies, critical infrastructure (energy, water management), and work involving state secrets or extremely sensitive research data. In that context the question isn't "how likely is a leak", but "what's the impact if it happens anyway" — and at that level of impact, even a small probability doesn't outweigh the convenience of a connection.

For most SMEs, even ones handling sensitive customer or patient data, this level is overkill. There, on-prem or a GDPR-proof EU environment is usually more than enough — that trade-off is worked out in detail in that article.

What it takes technically

An air-gapped setup shifts work the cloud normally provides for free onto you. Four things need to be arranged locally.

Bringing in the model weights physically. An open-source model gets downloaded outside the closed network, verified (checksum, provenance) and brought in via physical media — no network connection, so a drive. Every step in that chain is a checkpoint: what comes in must be demonstrably what you think it is.

A local knowledge base. RAG works the same here as anywhere else — retrieve, verify, cite — except the entire index (documents, embeddings, search index) lives on your own storage. There's no external search system to fall back on if the local index has a gap.

Updates via a verified channel. A new model version, a bug fix, an expanded knowledge base — everything has to go through the same controlled entry process as the first time. No automatic updates, no background sync. That's not a side detail; it's the bulk of the operational work.

Monitoring without outbound telemetry. Logging, usage statistics and error reports stay within the same boundary. A dashboard that "happens" to call an external service for charts is a leak — no matter how harmless it looks.

The price you pay

Air-gapped isn't free security. The model knows the world up to the moment it came in and no later — no live news, no recent legal changes, unless you explicitly and manually add them to the knowledge base. Scaling costs your own hardware instead of a button in the cloud. And maintenance — patches, new model versions, capacity expansion — is on you, with the same physical-media discipline as the initial install. Choosing air-gapped means consciously choosing a slower, more labour-intensive lifecycle in exchange for the guarantee that nothing can get out.

Conclusion

A language model without internet access is technically entirely possible — it runs, it answers, it can even do RAG. The real question isn't whether it can, but whether your threat model justifies that level of isolation. For state secrets and critical infrastructure, air-gapped is the only responsible choice. For almost everything else, plain on-prem or a GDPR-proof EU environment is where security and convenience balance out — as we show in the GDPR trade-off.