> ## Documentation Index
> Fetch the complete documentation index at: https://docs.tinfoil.sh/llms.txt
> Use this file to discover all available pages before exploring further.

# Encrypted persistent volumes

> Create and attach persistent disks, manage their lifecycle, and keep their contents encrypted under a key you hold.

> [!WARNING]
>
> Encrypted persistent volumes are in beta because they use [V3 attestation](https://docs.tinfoil.sh/resources/feature-status.md), which is still being deployed.

> [!NOTE]
>
> Tinfoil offers two ways to persist data. **Encrypted volumes** (this page) are block storage:
> a filesystem attached to one enclave at a time, suited to large working sets such as build
> trees, datasets, and fine-tuning checkpoints, encrypted under a secret you hold. The
> [buckets sidecar](https://docs.tinfoil.sh/containers/persistent-storage.md) is object storage: encrypted
> objects in S3, reachable from any number of enclaves and clients, encrypted under keys your
> end users hold.

## Overview

A secure enclave encrypts its memory with a key that only the enclave's own hardware can
use. That is what guarantees no operator, including Tinfoil, can read the data being processed
inside it. Memory, however, is not persistent, and even if it were, the key protecting it lives
in hardware registers that cannot be read out or saved. Memory and the enclave's ephemeral
filesystem are lost when it stops; data written to an unlocked persistent volume is retained.

Encrypted persistent volumes extend the enclave model to storage. They compose two primitives:
[private secret delivery](https://docs.tinfoil.sh/containers/private-secrets.md) and authenticated disk
encryption. A volume's contents are unreadable to anyone who does not hold its volume secret, and that
secret never exists outside your infrastructure and the enclave.

```mermaid theme={"dark"}
flowchart LR
    client["Your infrastructure<br/>(holds the original volume secret)"]
    subgraph enclave["Secure enclave"]
        boot["Boot"] -->|derives cipher and MAC keys| dm["dm-crypt + dm-integrity"]
        dm --> app["Your container<br/>/workspace"]
    end
    disk[("Persistent disk")]
    client -->|volume secret, released after attestation| boot
    boot ~~~ disk
    dm <-->|ciphertext, tags, IVs, and public header| disk
```

## Using managed volumes

A managed volume is a disk owned by your organization. It lives on one host, attaches to one
container instance at a time, and survives container deletion. Tinfoil manages the disk's
allocation and attachment; you manage its encryption secret.

The volume (disk), its named volume mount, and its mount path are separate:

| Example | Meaning |
| - | - |
| `dev-disk` | Managed volume created with the CLI or in the dashboard |
| `workspace` | Mount name declared by `volumes[].name` in `tinfoil-config.yml` |
| `/workspace` | Path inside the application container, set by `workspace:/workspace` |

Declaring a mount does not create a disk. A mount requires an attached volume before deploy
only when it declares `key-secret` (JSON `key_secret`). Mounts without it are optional and may
remain empty; without an attached disk, they provide no persistent storage.
Volumes count toward your organization's [storage limit](https://docs.tinfoil.sh/containers/limits.md#infrastructure-tiers)
at their full size until they are deleted, even when detached; the **Volumes** tab in the
dashboard shows allocated and available storage.

The API's `volume_slots` field lists declared mounts, and a volume's `slot` field contains
its assigned mount name. Neither field is a filesystem path.

### Before deploying with storage

Choose an unlock method **before** creating the instance. Allocating or attaching a disk does not supply its encryption secret, unlock its filesystem, or make the mount writable.

* **Automatic unlock:** generate and durably store the volume secret in infrastructure you control, run a reachable [keyserver](https://docs.tinfoil.sh/containers/private-secrets.md), set `keyserver-url`, and declare the secret with `key-secret`. Authorize the exact config repository, release tag, and instance domain in the keyserver policy before boot. The keyserver must release that secret after attestation on every boot; declaring its name is not enough.
* **Manual unlock:** omit `key-secret` and use a workload that implements the [runtime unlock flow](#unlock-at-runtime). Verify the enclave before delivering the original volume secret through that flow. There is no generic `tinfoil volume unlock` command. If your application has no unlock flow, use automatic unlock instead.

For CLI-assisted setup with customer-owned AWS Secrets Manager, follow
[Automatic volume unlock](https://docs.tinfoil.sh/containers/volume-auto-unlock.md). It separates one-time project
configuration, new-key creation or original-key selection, release publication, and customer
policy approval. The CLI prepares artifacts; it does not deploy or approve your keyserver.

For an attached disk, inspect the workload's unlock result and confirm the filesystem is usable at the configured mount path before writing durable data. Attachment status alone is not proof that unlock succeeded.

For non-debug `keyserver-url` delivery, `volumes[].key-secret` declares an
external secret; its name and value do not need to exist in Tinfoil's managed
secret store. Never upload the volume key there to satisfy validation. An
existing instance with managed secrets must [explicitly clear its saved selection](https://docs.tinfoil.sh/containers/private-secrets.md#switching-an-existing-instance-to-private-delivery)
when switching to private delivery. Required disk attachments are still checked,
but a successful update plan does not verify keyserver authorization, key release,
or disk unlock. Debug mode ignores the keyserver and is not a confidential way
to access a production volume.

### Create and attach a volume

1. [Prepare an unlock method](#before-deploying-with-storage), [declare a named volume mount](#configuration) in your repository's `tinfoil-config.yml`,
   and [publish a release](https://docs.tinfoil.sh/containers/updates.md#starting-an-update). Authorize that release before deploying if you use a keyserver.
2. Create a disk on the host you will use, then create the instance with `--volume` so it
   attaches and deploys:

```bash theme={"dark"}
tinfoil container hosts
tinfoil volume create dev-disk --size 16GB --host HOST_NAME
tinfoil container create my-app \
  --repo myorg/my-app-config \
  --tag v1.0.0 \
  --volume dev-disk:workspace
```

`dev-disk` is the managed volume. `workspace` is the config's `volumes[].name`. Omit
`:workspace` when the config declares exactly one mount. Create uses the disk's host; an
explicit `--host` must match. The disk is never created implicitly. Repeat `--volume` to
attach several disks, and specify each mount name. Pass a disk for every required mount;
optional mounts may remain empty. If required mounts exist and you omit `--volume`, the CLI
refuses before creating the instance and prints commands to create and attach the disks.

In the dashboard's create form, select an existing disk or create one for each required
mount. Optional mounts do not require a disk. All selected disks must be on the instance's
host. Creation attaches the selected disks and then deploys the instance.

To swap disks later, stop the instance and wait until it is **Stopped**, then:

```bash theme={"dark"}
tinfoil volume detach dev-disk
tinfoil volume attach other-disk my-app --as workspace
tinfoil container deploy my-app
```

`--as MOUNT_NAME` selects the mount and defaults to the only declared mount when omitted. `tinfoil container deploy my-app --volume other-disk:workspace`
also attaches a detached disk before deploying. The container must already be on that disk's host.
Once attached, later deploys and updates reuse the disk; omit `--volume`.

You can also create, attach, and deploy from the dashboard: **Containers → Volumes**, then
**Create volume** on the instance's host, **Attach** it to the `workspace` mount, and deploy
the instance. Reuse an unattached volume on that host instead of creating one.

Files written outside the unlocked volume's mount path stay in the enclave's ephemeral
filesystem and do not survive Stop followed by Deploy. Attaching a disk does not move existing
files onto it.

Reusing a volume requires its original encryption secret. Back up that secret in infrastructure
you control before relying on the disk. Stop followed by Deploy keeps the saved disk attachment;
use the same disk and original key, not a newly created replacement. Unattended unlock also
requires your keyserver to be reachable and to authorize the deployed release on every boot.

### Volume lifecycle

| Action | Effect on storage |
| - | - |
| Stop or deploy without `--volume` | Keeps the volume and its saved attachment. Unlock it on each boot. |
| Update the container | Reuses the volume after stopping the old enclave. Updates have downtime and cannot be held for review; see [updates with persistent volumes](https://docs.tinfoil.sh/containers/updates.md#updates-with-persistent-volumes). |
| Detach the volume | Keeps its data and removes the saved attachment. It can be attached to another stopped container on the same host. |
| Delete the container | Retains its volumes as unattached disks. |
| Delete the volume | Permanently deletes the disk and its data. The volume must be detached first. It does not delete the original key from your private secret store or remove keyserver policy approvals. |

Stop the container before attaching or detaching a volume. If its previous enclave is still
shutting down, wait for it to release the disk and retry. Volume assignments cannot change
while an update is in progress. Detach a volume before removing its mount from the config or
moving the container to another host; the disk remains on its original host.

Volume sizes are fixed at creation. **Rename** changes only the disk's display name, not its
mount name or contents. Stopped and unattached volumes still count toward your storage allocation;
capacity is released only after volume deletion completes.

## Key management

Persistent storage needs a long-term secret to encrypt and authenticate data under. That secret cannot be managed
by Tinfoil, since anyone holding it can read or modify the data. It is created and kept on your side.

A volume secret must contain at least 16 bytes. It can be random bytes or a passphrase;
the length requirement applies to the raw bytes. Store it durably in infrastructure you
control. For a randomly generated key, this command produces 32 bytes encoded as base64:

```bash theme={"dark"}
openssl rand -base64 32
```

Your keyserver supplies the key as standard base64. Runtime unlock requests carry raw
bytes. Plain `tinfoil volume create` allocates an empty disk; it does not generate, store,
or deliver an encryption key. The enclave never generates or persists this secret.

With explicit [new-volume auto-unlock mode](https://docs.tinfoil.sh/containers/volume-auto-unlock.md#new-disk-generate-the-key-once),
the CLI generates the key once and stores it in your AWS Secrets Manager account. Existing
volumes must use their original key; generation is not the default behavior of volume creation.

On every boot, the volume secret is delivered to the enclave after attestation, over a connection that
terminates inside the enclave. There are two ways to do this:

* **Automatically**, by declaring the volume secret as a private secret and letting your
  [keyserver](https://docs.tinfoil.sh/containers/private-secrets.md) release it once the enclave's attestation matches your
  policy.
* **Manually**, by sending the secret over the attested connection to the running enclave, which
  unlocks the volume at runtime.

Inside the enclave, Argon2id derives an AES-XTS encryption key and an HMAC integrity key
from your volume secret and the salt in the [volume header](#volume-header). You provision
one secret; the derived keys exist only in enclave memory.

> [!WARNING]
>
> The volume secret is the confidentiality boundary for everything on the volume. If it is lost,
> the data cannot be recovered by you, by Tinfoil, or by anyone else.

### Volume keys are not ordinary rotating secrets

An existing encrypted volume must keep its original key. Do not generate a replacement key
when enabling automatic unlock, recovering a failed setup, or moving the same disk to another
instance. Import the original key into your private secret store instead.

Replacing the secret value, enabling automatic secret rotation, or changing `key-secret` does
not re-encrypt the disk. A different key cannot unlock the existing data. Key rotation requires
an explicit data migration or supported re-encryption procedure, with the original key and a
recoverable backup retained until the migrated data has been verified. Do not use ordinary
private-secret rotation as a volume-key rotation procedure.

## Disk encryption

Once the enclave holds the derived keys, it opens the volume with two Linux kernel primitives,
`dm-crypt` and `dm-integrity`, configured for authenticated encryption:

* **Confidentiality**: AES-XTS with a fresh random IV for every write, so identical data
  produces different ciphertext each time it is written, regardless of where it lands on disk.
* **Integrity**: an HMAC-SHA256 tag over every sector, stored with its IV in the
  `dm-integrity` layer. Any sector whose tag does not verify is rejected on read.

After successful unlock, your container sees an ordinary writable filesystem. The disk, and anyone with access to it,
sees only ciphertext, tags, IVs, and the public volume header.

## Volume header

The first 4 KiB of an Argon2id volume is reserved for a plaintext header:

| Field | Value |
| - | - |
| Format marker | `tinfoil-volume-2` |
| Salt | 32 random bytes generated when the volume is initialized |

The marker defines Argon2id with 3 iterations, 256 MiB of memory, 4 threads,
and 96 bytes of derived key material. These settings are fixed in the enclave
image. The header contains no keys. Twelve reserved bytes between the marker
and salt are ignored. The enclave also accepts `tinfoil-volume-1` headers with
this profile. Both markers use runtime format byte `0x02`.

The host can read and edit the header. An unrecognized marker prevents unlock;
a changed salt causes authentication of the existing data to fail on the next
unlock. The format does not detect replay of an older valid disk snapshot.

## Threat model

The goal is to give the volume the same guarantees as the enclave's encrypted memory, and to be
explicit about what it does not cover.

**Protected.** An attacker with full access to the disk cannot read its contents, and cannot
forge or alter a sector's contents without the enclave detecting it. Because every write uses a random IV, the
attacker also cannot tell whether two sectors hold the same data or whether a sector was
rewritten with its previous contents, which rules out ciphertext-comparison attacks.

**Not protected: rollback.** Integrity protection guarantees the ciphertext has not been
tampered with, but an attacker who replays an earlier, valid version of a sector, or restores
the whole disk to an earlier state, is not detected. This mirrors the memory model, where the
hardware protects against modification but not replay.

**Not protected: access patterns.** A privileged attacker can observe which sectors the enclave
reads and writes, and when. This side channel can leak information about enclave activity even
though the contents are encrypted. It can be closed at the application level by running an
Oblivious RAM (ORAM) scheme on top of the volume, at a performance cost that depends on your
workload. For an example of such implementation and a detailed study of this tradeoff, see
[Opal](https://arxiv.org/abs/2604.02522), a private memory system for personal AI
built on Tinfoil.

## Configuration

Declare named volume mounts in the top-level `volumes` list of `tinfoil-config.yml`.
Reference them by name in each application container's `persistent_volumes` list.
Create and attach their backing disks [separately](#create-and-attach-a-volume).
`key-secret` names the private secret that carries the volume secret; omit it to unlock the
volume manually at runtime instead. A mount with `key-secret` requires a disk before deploy;
one without it may remain empty. The secret must be a private secret released by your keyserver. A
Tinfoil-managed secret would give Tinfoil the ability to read the volume.

```yaml tinfoil-config.yml theme={"dark"}
cvm-version: "<CVM_VERSION>"

keyserver-url: https://keys.example.com

volumes:
  - name: workspace
    key-secret: WORKSPACE_SECRET

containers:
  - name: app
    image: "ghcr.io/myorg/app@sha256:<digest>"
    persistent_volumes: [workspace:/workspace]
```

| Field | Type | Required | Description |
| - | - | - | - |
| `name` | string | Yes | Mount name (lowercase letters, digits, hyphens; must start with a letter), referenced from `containers[].persistent_volumes` as `<mount-name>:<mount-path>`. It does not need to match the managed volume's name. |
| `size` | string | No | Accepted by the config schema, but does not allocate or resize a managed volume. Choose capacity when creating the disk. |
| `key-secret` | string | No | Name of the private secret holding the base64 volume key. It must decode to at least 16 bytes. Omit to unlock at runtime. |
| `owner` | string | No | Owner of the control socket and initial filesystem root, as numeric `uid:gid` or one ID for both. Defaults to `0:0`. |
| `exec` | bool | No | Allow executables on the volume |

> [!NOTE]
>
> Docker's writable layers, pulled images, and ordinary Docker volumes live in memory and
> are lost when the enclave stops. Only data written through the unlocked persistent
> filesystem at its configured mount path is durable on the volume.

## Unlock at runtime

A volume without `key-secret` stays locked until a container supplies its key.
Until then, the mount is read-only and writes fail with `EROFS`.

A container that mounts the volume through `persistent_volumes` also gets
access to `/run/tinfoil/volumes/<name>/control.sock`. Run the unlocking process
as the UID in the volume's `owner` field. The socket belongs to that user and
has mode `0600`.

Connect to the socket using Unix `SOCK_SEQPACKET`. Send one packet with a
format byte, an ASCII operation byte, and the raw key bytes:

| Format | First byte | Operation | Key |
| - | - | - | - |
| Argon2id | `0x02` | `u` to unlock, `i` to initialize a blank disk | At least 16 bytes |
| Legacy HKDF | `0x01` | `u` to unlock | Original 64-byte key |

The request can contain up to 64 KiB, including the two prefix bytes. The
response is `ok`, `rejected`, or `failed`. The format byte selects how the key
is derived. Initialization requires a blank disk.

Wait for `ok` before starting the application. The unlocked filesystem appears
at the container's existing mount, ready for reads and writes. If unlocking
fails, the mount stays read-only.

## Unlock a legacy volume

Legacy volumes use HKDF-SHA256 and require their original 64-byte key. To open
one, send a legacy unlock request to its runtime control socket. These volumes
have no Tinfoil volume header and cannot be unlocked at boot with `key-secret`.

Stop application writes before preparing the instance for migration:

1. Remove the volume's `key-secret`. Add that secret's name to the migration
   container's `secrets` list, and keep `keyserver-url` configured.
2. Mount the volume through `persistent_volumes`. Set the migration container's
   `user` to match the volume's `owner`.
3. Publish and authorize the updated release, then deploy the instance with
   the volume attached to the same mount. Run the unlock code below inside
   the migration container.

This config fragment gives a migration container access to the `data` volume
and its key:

```yaml tinfoil-config.yml theme={"dark"}
keyserver-url: https://keys.example.com

volumes:
  - name: data
    owner: "1000:1000"

containers:
  - name: migration
    image: ghcr.io/myorg/migration@sha256:...
    user: "1000:1000"
    secrets: [DATA_VOLUME_KEY]
    persistent_volumes: [data:/var/lib/app]
```

Use an image with Python installed, and run this code inside the container
before accessing the data:

```python theme={"dark"}
import base64
import os
import socket

encoded_key = os.environ.pop("DATA_VOLUME_KEY")
key = base64.b64decode("".join(encoded_key.split()), validate=True)
if len(key) != 64:
    raise SystemExit("legacy volume key must contain 64 bytes")

with socket.socket(socket.AF_UNIX, socket.SOCK_SEQPACKET) as connection:
    connection.connect("/run/tinfoil/volumes/data/control.sock")
    connection.sendall(b"\x01u" + key)
    if connection.recv(64) != b"ok":
        raise SystemExit("legacy volume unlock failed")
```

The volume remains in the legacy format after unlocking. Legacy support is
available for migration.

To migrate, keep application writes stopped and copy the files to a newly
initialized Argon2id volume, preserving filesystem metadata. Check the copied
data, then confirm that you can reopen the destination after Stop followed by Deploy.
Retire the legacy volume only after these checks succeed.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.