# Encrypted fields

An entity field can be stored encrypted. It is sealed with your app's one
`field_encryption_key:` when it is written and opened when it is loaded,
so your code reads it as it reads any field, and a copy of the database
holds only sealed bytes. A clinic keeps each consultation's notes that way:

```jadpo
application {
    connections: {
        main: {
            driver: Driver.postgres
            url: config.database_url
        }
    }
    field_encryption_key: config.field_encryption_key
}
```

```jadpo
config {
    database_url: PostgresUrl
    field_encryption_key: EncryptionKey
}
```

```jadpo
use clinic/types { ClinicalNote }

/// A consultation, whose notes only the application can read.
entity Consultation {
    id: Uuid
    notes: ClinicalNote {
        encrypted: true
    }

    policy: {
        create: [Access.public]
        read: [Access.public]
    }
}
```

```jadpo
use clinic/entities { Consultation }

test "a consultation's notes read back as they were written" {
    var written_up: Consultation = record_consultation("Allergic to penicillin.")
    assert written_up.notes == "Allergic to penicillin."
    var read_back: Consultation = attempt Consultation.get(written_up.id)
    assert read_back.notes == "Allergic to penicillin."
}
```

A field of any stored type may be `encrypted: true`, whatever columns it
would have had: it is kept as its type's JSON, sealed, in one `bytea`, so
it holds as much as a structured field does. Each value is bound to its
column and its record's `id`, so moved to another row it opens for no one,
and one value sealed twice gives different bytes.

A key type, a `TotpSecret` and a `SecretText` are always kept sealed, in a
field of their own, whether or not the field says so, and `encrypted:
false` on one is refused
([How a secret is kept](/docs/secret-values.md#how-a-secret-is-kept)).
There are no per-person keys (see [Erasure](/docs/erasure.md#what-erasure-doesnt-rewrite)).
Keys can also be held as data: a key per device or person in a key-type
field, usable only as a `crypto` call's `key:`
([A key is never a value](/docs/hashing-and-cryptography.md#a-key-is-never-a-value)).

Every field sent to an AI model is listed in the report above
([Following personal data](/docs/following-personal-data.md)).

## What a sealed field can't do

Only your app can read a sealed column, and its bytes change every time it
is written, so nothing that has the database read it can work.
`data.encrypted_field_used` refuses a sealed field in:

- a unique rule or an index;
- a query's filter, ordering or group;
- a search's `fields:`;
- a lifecycle's conditions and the fields it sets, and a retention's or a
  purge's `since:`;
- a value the database works out in an update by `id`;
- a computed backfill, which is SQL or runs without the key;
- a failure's fields, since a failure is written to the log.

What the database itself has to read can't be sealed at all:
`data.encrypted_field_invalid` refuses `encrypted:` on the `id`, a
reference, a generated field and a `File`.

Reading one in an action and answering with it is fine, and policy decides
who may. A field's `retention:`, and erasure, clear a sealed value as they
clear any. The history of an [audited](/docs/audit-trail.md)
record says a sealed field changed and never what it held. Spans and
runtime logs carry no field's value at all.

## The key

`field_encryption_key:` is written at the top of `application { }`, and
reads an `EncryptionKey` setting: 32 random bytes, written as 43 Base64url
characters. A `SigningKey` won't do (`data.field_encryption_key_invalid`).
Your app needs one only when it has an encrypted, key-type, `TotpSecret` or
`SecretText` field, and then `data.field_encryption_key_required` says so at
the first of them. The key your fields are sealed with is derived from the
setting for this one use, so a `crypto.encrypt` call of your own under the
same setting never meets them.

| Where | How it is given |
|---|---|
| Your machine | `jadpo config set field_encryption_key --generate` writes 32 random bytes into `.env.local` and prints nothing. `jadpo config check` then says `set`, and `jadpo dev` refuses to start until it is |
| A native server | `FIELD_ENCRYPTION_KEY` in the environment or from a secret manager. `jadpo config generate field_encryption_key` makes one, only into a pipe; at a terminal it refuses |
| Workers | A Wrangler secret, which `jadpo build` lists in `secrets.required`: `jadpo config generate field_encryption_key \| wrangler secret put FIELD_ENCRYPTION_KEY` |
| Tests | `jadpo test` makes a throwaway key for each run, as it does every secret |

## Rotating the key

To replace the key without taking the app down:

1. Set the new key as `FIELD_ENCRYPTION_KEY`, and move the old one to
   `FIELD_ENCRYPTION_KEY_PREVIOUS`. New values are sealed with the new key,
   and values sealed with either read.
2. An app that keeps anything sealed has a built-in schedule,
   `field_encryption.reseal`, that runs every hour. While `_PREVIOUS` is set
   it rewrites rows still under the old key, at most 20 batches of 500 a run,
   and changes nothing else about them. `jadpo schedules` lists it, and
   `jadpo schedules run field_encryption.reseal` starts a run.
3. `jadpo config rotation` counts each sealed field's rows against both keys.
   When it says `FIELD_ENCRYPTION_KEY_PREVIOUS can go`, remove it.

A key held in a key-type field rotates by being replaced: store a new one,
as `crypto.new_key(as: EncryptionKey)` makes it.

## What can't be undone

Sealing trades one risk for another, and some of what follows has no fix:

- **Losing the key loses every sealed value for good.** Backups don't help,
  because they hold the same sealed bytes. Keep an offline copy of the key,
  somewhere apart from your backups.
- **Removing `_PREVIOUS` too soon, or rotating twice before the re-seal has
  finished, loses the rows still under the older key.** Removal is safe only
  once `jadpo config rotation` says so. Each rotation starts with a run of
  it.
- **A restored backup needs the key it was written with.** Keep each old key
  for as long as you keep a backup written under it.
- **A key leaked together with a copy of the database can't be fixed by
  rotating.** Rotating protects what is written from then on, and the copy
  still opens with the old key. Treat that as a breach of everything sealed.
- **Turning `encrypted:` on or off needs a data migration.** `jadpo migrate
  new` refuses it, since stored values would have to be sealed or opened one
  by one with a key SQL never sees. Add a new field with the setting you
  want, copy each record's value across in an action, then retire the old
  field. Its column keeps what it held, in the clear when you are turning
  encryption on, until you [drop it](/docs/migrations.md#removing-data). The same goes
  for a new required sealed field, and a narrower type on one.