Backups for AI systems: what should you keep besides the data?

For an AI system, you should back up not only the original data but also everything required to turn that data back into the same functioning system. This includes chunking and indexing configuration, embedding models, prompts and system instructions, model versions, tool configuration, agent permissions, evaluation sets and relevant audit information. A backup containing only documents and databases protects the content, but not necessarily the behaviour of the AI system.

The main question is therefore not: what is stored on the server? A better question is: which components do you need to rebuild the system reliably after data loss, corruption or an incorrect change? Some components should be restored directly. Others, such as a vector index, may be better regenerated from controlled source data and recorded configuration.

Back up the components that determine behaviour

Traditional application backups tend to focus on files, databases and application configuration. AI systems add components that directly affect what the system retrieves, generates or is allowed to do.

In a RAG system, for example, the source documents are only part of the picture. You also need to know which embedding model created the vector representations, which model version was used, how documents were split into chunks and which metadata or filters were applied during indexing. Without that information, you may not be able to reproduce the same knowledge system reliably.

At a minimum, preserve the embedding model and version information, chunking configuration, indexing rules and relevant retrieval settings. If you use an external embedding model that you do not store yourself, record the exact model reference and configuration. With local models, model weights may form part of the recovery strategy, particularly when you cannot guarantee that the same version will remain available later. The hardware required to run local models can also affect recovery planning; see hardware for local AI.

A vector index may be better rebuilt than restored

A vector index can be large and usually contains derived information created from the source data. It therefore does not automatically follow that every index needs to be stored as a complete backup. If you retain the original documents, embedding model, chunking rules and indexing configuration, you can often rebuild the index.

Rebuilding may even be preferable to restoring an old index. An index can be corrupted, only partially updated or still contain information that has since been removed from the source. Re-indexing from a controlled dataset reduces the risk of automatically restoring such inconsistencies.

The trade-off is that re-indexing requires time, compute capacity and sometimes API expenditure. For larger systems it can therefore make sense to support both restoration from an existing index and a complete rebuild. These recovery costs belong to the broader operational cost of AI; see what AI really costs.

Version prompts, instructions and configuration

Prompts are part of an AI system's software configuration. A small change to a system prompt, tool description or retrieval instruction can noticeably change behaviour. Prompts should therefore not exist only as separate text files: keep them under version control together with the configuration in which they are used.

The same applies to model settings, routing rules, fallback logic, output schemas and parameters that determine which tools a model may call. For agents, you should also record which permissions the agent had, which systems it could access and when human approval was required.

A recovery in which all documents are restored correctly but an agent suddenly receives different permissions is not a reliable recovery. The same principle applies to chains in which several models or services can replace one another. See AI fallback chains for more on that architecture.

Preserve evaluation sets, logs and approvals deliberately

A restored AI system should do more than start successfully; you also need a way to determine whether it still behaves as it did before the incident. Evaluation sets are therefore valuable in a backup and recovery strategy. A fixed collection of questions, documents and expected properties lets you test whether retrieval, answers and agent behaviour still remain within the intended boundaries after recovery.

Logs are also important, although not every log needs to be retained indefinitely. Operational logs, errors, model and tool calls and approval records may be needed for analysis, auditing and incident investigation. Which logs are necessary and how long they should be retained depends on the system and organisation. See AI monitoring for the role of logging and observability in production systems.

Secrets such as API keys, passwords and access tokens should not be stored as ordinary configuration inside a backup. Keep them in a dedicated secrets vault or key-management environment and make sure its recovery process is covered as well. Your backup documentation should identify which secrets are required without unnecessarily copying those secrets to multiple storage locations.

A backup is only useful if it can be restored

The familiar 3-2-1 principle remains useful for AI systems: maintain multiple copies, use more than one storage medium or environment and keep at least one copy sufficiently isolated from the primary system. The exact implementation depends on your infrastructure and risk profile. The general point is that a single backup file stored on the same platform does not constitute a robust recovery strategy.

More important is the distinction between having a backup and having a recoverable backup. You only know whether a backup works when you actually test restoration. This can mean creating a clean environment, restoring data or rebuilding the index, loading configuration, reconnecting secrets and then running functional evaluations.

External dependencies should be included in this test. A model name in a configuration file is of limited value if that model version is no longer available when recovery is needed. Periodically verify that the references and installation sources on which your recovery process depends still exist.

Retention periods also apply to backups

Backups are not a justification for indefinite retention. If documents, databases, logs or evaluation sets contain personal data, privacy obligations remain relevant. Under the GDPR, organisations must be able to justify why personal data is retained and for how long. See AI and the GDPR for more context.

This does not necessarily mean that every deleted personal data item must immediately be technically removable from every historical backup. The appropriate approach depends on the backup architecture, retention periods and access controls. You should, however, know what your backups contain, how long copies are retained and what happens when an older backup is restored. Privacy-sensitive recovery procedures may require legal review.

Short AI backup checklist

• Source data: documents, databases, metadata and other primary information.
• Vector layer: the vector index or a demonstrably reproducible method for rebuilding it.
• Embeddings: embedding model, version and relevant settings.
• Indexing: chunking rules, parsers, metadata rules, filters and indexing configuration.
• Models: local model weights or exact references to the model versions in use.
• Prompts: system prompts, templates, tool instructions and other behaviour-defining configuration under version control.
• Agents: tool configuration, permissions, approval rules and access policies.
• Evaluation: test sets and evaluation configuration used to verify correct operation after recovery.
• Audit information: necessary logs, error information and records of human approvals.
• Secrets: API keys and passwords stored through a separate secrets vault, with its own tested recovery procedure.
• Recovery testing: regularly perform a complete recovery rather than merely checking that backup files exist.