# File fields, `put`, `read` and the disk driver A `File` field, `store.put` and `store.read` each refuse what doesn't fit. This topic has the exact rules, and where `jadpo dev` and tests keep files, for when a write is refused or a read is too large ([Files](/docs/files.md) has how to use them). ## What a `File` holds A `File` holds the connection its bytes are in, the key the runtime made for them, its name (none for one `store.put` made), its size and its `FileType`. It is kept as its JSON in a `jsonb` column. Only the runtime builds one: `File { ... }` is `type.not_record`. Nothing from outside carries a `File`, which would let the sender name any file. It is `type.batch_input` in an `Input` or a route's placeholder, in an operation's `output:` or `wire_output:`, in a webhook's event, and in an event a queue lists. A route sends a `File` as a signed link, which comes with [downloads](/docs/files.md#downloading). Until those are built, a `File` in a route's `output:` or a failure's `client:` is `toolchain.not_built_yet`. A failure's `internal:` fields stay on the server and may hold one. Sending a file to a provider isn't built. A `File` in an operation's `input:` or a webhook's `reply:` is `toolchain.not_built_yet`, since the provider would get the store and key and none of the bytes ([Types](/docs/calling.md#types)). Each of these holds however deep: a list, a map, an `Object`, an entity's field named as a type, a `pick:`, or an `ExportSummary`, which holds the file it wrote. The other ways a value leaves or arrives refuse one too: | Where | Refused as | |---|---| | A prompt's field, however deep | `prompt.placeholder_type` | | A model's `as:` | `semantic.ai_output_type` | | `json.write`, `json.read` and `text.parse` | `type.mismatch` | | `crypto.verify_jwt`'s `as:` | `type.mismatch` | | A row an export writes | `type.batch_row_invalid` | | A row an import reads | `type.batch_input` | | An email's template | `email.template_value_invalid` | | A message's wording | `translation.field_type` | | A setting in `config.jadpo` or a package's `settings:` | `configuration.type_unsupported`, `package.provider_invalid` | | A feature flag | `flag.declaration_invalid` | | A search's fields | `search.settings_invalid` | | A field an audit trail keeps | `toolchain.not_built_yet` | A `File` field on an `EmailMessage` is the one that leaves: it is sent as an attachment, the bytes rather than the store and key ([Attachments](/docs/email.md#attachments)). A program reads none of its parts. `file.key`, and `file.nope`, are `type.field_on_non_record`, in code and in a template. ## A `File` field's settings - `store:` must name a connection that provides `files`. - `max_size:` is required: a whole number of `b`, `kb`, `mb` or `gb`, each a thousand times the last. `5mb` is 5,000,000 bytes, which is why this page writes "100 MB" apart from "16 MiB". - `types:` is optional. A mistake in any of them is `data.file_field_invalid`. The three settings on a field that isn't a `File` are `type.unknown_setting`. Every write of the field holds the file it is given to them ([What a write checks](/docs/files-put-read-and-the-disk-driver-in-development.md#what-a-write-checks)). ## `store.put(bytes, type)` You call `store.put` only in an action, before its first write, as every outside call is made. Its key is made from the invocation's randomness, so a replay keeps it. It goes out as a keyed call, since writing the same key again writes the same object, which never changes. A store with no room answers `storage_full`, the call's one condition. Its `attempt` maps it to a failure of kind `unavailable`; it isn't a fault of a value, so `invalid:` doesn't cover it. A store that can't be reached is the runtime's own `unavailable`. ## What a write checks A `File` is kept in a field only when it fits the field's settings. Every `create`, `update` and patch that sets a `File` field checks, before its statement runs, that the file is: 1. in the store the field's `store:` names; 2. of a type its `types:` lists, when it lists any; 3. no larger than its `max_size:`. A file that breaks one stops the write with the fault `file_refused`, naming the field and what didn't fit, and nothing the action wrote is kept ([Defects](/docs/faults.md#defects)). It is a fault, not a failure, because only the runtime makes a `File`, from the bytes, the type and the store the action gave `put`, so the mismatch is the program's own. Bound the bytes before the put, such as by the `max_length:` of the text they are made from, or change the field's settings. Clearing a field, or a patch that leaves it out, checks nothing. Here the bakery's sheets are at most 32 bytes, so a four-line sheet isn't kept: `domains/bakery/entities/price_sheet.jadpo`: ```jadpo use bakery/types { SupplierName } /// A supplier's price sheet, kept as it was sent. entity PriceSheet { id: Uuid supplier: SupplierName sheet: File { store: sheets max_size: 32b types: [FileType.csv] } policy: { create: [Access.public] read: [Access.public] } } ``` `domains/bakery/actions/keep_prices.test.jadpo`: ```jadpo use bakery/entities { PriceSheet } use bakery/types { SheetText } test "a short sheet is kept" { var price_sheet: PriceSheet = attempt keep_prices("Millers", "flour,1.20\nrye,2.40\n") assert price_sheet.supplier == "Millers" } test "a longer sheet isn't kept" { var sheet: SheetText = "flour,1.20\nrye,2.40\nspelt,3.10\noats,0.90\n" var price_sheet: PriceSheet = attempt keep_prices("Millers", sheet) assert price_sheet.supplier == "Millers" } ``` ```text FAIL a longer sheet isn't kept --> domains/bakery/actions/keep_prices.test.jadpo:11 fault: file_refused in action keep_prices at: domains/bakery/actions/keep_prices.jadpo:11:12 | 11 | return PriceSheet.create { | ^^^^^^^^^^^^^^^^^^^ not kept reason: `PriceSheet.sheet` holds files of at most 32 bytes, and this one is 41 bytes; a field keeps only a file that fits its settings, so the write stopped before it ran and nothing the action wrote was kept. next: Bound the bytes before they are put, such as by the `max_length:` of the text they are made from, or raise `PriceSheet.sheet`'s `max_size:` (agent) stack: domains/bakery/actions/keep_prices.jadpo:11 action keep_prices domains/bakery/actions/keep_prices.test.jadpo:11 test "a longer sheet isn't kept" rule: test.fault rerun: jadpo test domains/bakery/actions/keep_prices.test.jadpo:11 --seed 1 test failed: 1 failed, 1 passed, 2 action tests ``` ## `store.read(file)` `store.read` reads at most 10 MB into memory. Reading a field whose `max_size` is over that is `type.file_read_too_large`, where it is read. A `File` whose field isn't known, such as a parameter of type `File`, is held to the limit as it runs, and going over it is a fault. A file whose bytes are gone from its store is a fault too. `FileContents` is the file's bytes, and a program reads none of its parts. It goes to a provider: an operation's `input:` holding one sends its Base64 in the JSON, as `Bytes` is sent. Two other places can hold one on its way there: - **An event's field**, so a subscriber can pass the bytes on. They are kept in the outbox, as Base64 in the event's JSON, until the event is deleted ([What the tick deletes](/docs/schedules.md#what-the-tick-deletes)): 24 hours after it was emitted once every delivery has succeeded, and for as long as a delivery has failed. An event's JSON is at most 64 KB, so `emit` faults on a larger file. - **A failure's `internal:` field**, which stays on the server, so the caller's answer holds none of the bytes. Nothing else takes it: | Where | Refused as | |---|---| | An entity's field | `toolchain.not_built_yet` | | An `Input` or a route's placeholder | `type.batch_input` | | An operation's `output:` or `wire_output:`, a webhook's event, an event a queue lists | `type.batch_input` | | A route's `output:` or a failure's `client:` | `toolchain.not_built_yet` | | An email or a partial ([Attachments](/docs/email.md#attachments)) | `toolchain.not_built_yet` | | A prompt's field | `prompt.placeholder_type` | Each holds however deep, but for an email. A provider's own bytes come as `Bytes`, which `put` keeps. Like `put`, `read` is called only in an action, before its first read or write. A test reaches files through the application's actions. ## The disk driver in `jadpo dev` `jadpo dev` keeps every store of files under `.jadpo/files//`, whatever its driver, so nothing leaves your machine: - Each file is kept under its key, sharded into two directory levels by its id. A key of any other shape is refused, and so is a symbolic link anywhere on the way. - A file is written to a temporary file readable by the application's user only, synced, and renamed into place, and its folder is synced. - A full disk answers `storage_full`, and so does a put below the store's `free_space_floor:`, as it does in production. Its reads and writes block, a window of at most a megabyte at a time, on your own disk. ## Files in a test In a test, one in-memory store serves every connection ([Provider calls and capabilities in tests](/docs/fakes-for-integrations.md#provider-calls-and-capabilities-in-tests)), and a file helper's run reads two rows a batch. `fixture_file`, and telling the store to fail from a test's source, come with the testing fakes ([What is built](/docs/built-in-services-built.md)).