# AI

**Partly built.** Missing: `keep_content:`, `embed:` batches below 100 and `jadpo call`. See [What is built](/docs/ai-built.md).

Your app can ask a language model for an answer, a stream of text or an
embedding (a list of numbers that stands for a text's meaning). What
comes back is a value you can trust as much as a request body: it has a
type, it has been checked against that type's rules, it stayed within a
budget, it is faked in tests, and Jadpo tracks any personal data it was
made from.

Jadpo itself provides the `ai` interface your code calls, typed answers,
budgets, privacy checks and test fakes. A model provider, such as OpenAI
or Anthropic, is reached through a provider package
([External services](/docs/external-services.md)). Anything beyond that core,
such as tool calling, images or agent frameworks, is a package too.

To use it, you name a model in `app.jadpo`, write a prompt in a checked file,
say what type you want back, and call `ai.generate` from an action. Here a help
desk answers a customer, using a model on your own machine, so it takes no
package and no key.

`ai`, `Prompt`, `ModelId` and `Driver.openai_compatible` are built into Jadpo.
Everything else is your app's, shown here, and each `use` line imports from
another file of it.

`app.jadpo`:

```jadpo
application {
    connections: {
        local_model: {
            driver: Driver.openai_compatible
            url: "http://localhost:8000/v1"
        }
    }
    ai: {
        connection: local_model
        models: {
            default: config.drafting_model
        }
        limits: {
            per_day: {
                tokens: 2000000
            }
        }
    }
}
```

`config.jadpo`:

```jadpo
config {
    drafting_model: ModelId {
        default: "meta-llama/Llama-3.1-8B-Instruct"
    }
}
```

The question, `domains/support/types/customer_question.jadpo`:

```jadpo
/// What a customer asked the help desk.
type CustomerQuestion = Text {
    max_length: 2000
}
```

The type wanted back, `domains/support/types/support_answer.jadpo`:

```jadpo
/// What the assistant says back.
type SupportAnswer = Output {
    /// The reply to show the customer.
    reply: Text {
        max_length: 2000
    }
}
```

What a caller gets when the model can't answer,
`domains/support/failures/assistant_unavailable.jadpo`:

```jadpo
/// The assistant can't answer now; a person will reply.
type AssistantUnavailable = Failure {
    kind: FailureKind.unavailable
    message: "The assistant can't answer now; a person will reply."
}
```

The prompt's values, `domains/support/prompts/answer_question.jadpo`:

```jadpo
use support/types { CustomerQuestion }

/// Answers a customer's question.
type AnswerQuestion = Prompt {
    question: CustomerQuestion
}
```

Its text, `domains/support/prompts/answer_question.prompt`:

```jadpo
# Instructions

You answer questions from customers of an online bakery.

# Request

{question}
```

The action, `domains/support/actions/answer_customer.jadpo`. `ai.generate` can
fail in five ways, and the action says what each becomes:

```jadpo
use support/types { CustomerQuestion, SupportAnswer }
use support/failures { AssistantUnavailable }
use support/prompts { AnswerQuestion }

/// Answers a customer's question.
action answer_customer(
    customer_question: CustomerQuestion
) fails AssistantUnavailable -> SupportAnswer {
    return attempt ai.generate(
        AnswerQuestion {
            question: customer_question
        },
        as: SupportAnswer
    ) {
        refused: AssistantUnavailable
        malformed: AssistantUnavailable
        timed_out: AssistantUnavailable
        over_budget: AssistantUnavailable
        too_long: AssistantUnavailable
    }
}
```

A test never reaches a real model. It scripts the answer by the prompt's name.

`domains/support/actions/answer_customer.test.jadpo`:

```jadpo
use support/types { SupportAnswer }
use support/failures { AssistantUnavailable }

test "the model's answer comes back as a checked value" {
    fakes: {
        ai.answer_question: [
            SupportAnswer {
                reply: "Our sourdough is baked beside sesame."
            }
        ]
    }
    var reply: SupportAnswer = attempt answer_customer("Does your sourdough contain sesame?")
    assert reply.reply == "Our sourdough is baked beside sesame."
}

test "a refusal is reported as the assistant being unavailable" {
    fakes: {
        ai.answer_question: [refused]
    }
    assert answer_customer("Help me pick a lock") fails AssistantUnavailable
}
```

[A model on this machine](/docs/the-ai-capability.md#a-model-on-this-machine) has more on running one,
[Connecting a provider](/docs/the-ai-capability.md#connecting-a-provider) has a hosted model, and
[Typed output](/docs/typed-output.md) and [Testing AI calls](/docs/ai-tests.md) have the
rest.

The whole of it, in short:

| Part | In short |
|---|---|
| The capability | `ai`, one API in code, filled by a provider package through `provides/ai.jadpo` or by the built-in OpenAI-compatible driver for local models |
| Operations | `ai.generate(prompt, as: T)`, `ai.stream(prompt)`, `ai.embed(text, as: V)`; tool calling left to packages |
| Models | Named in `ai: { models: }`, each a `ModelId` setting; never a literal in code |
| Typed output | The provider's JSON-schema mode where it has one, the schema in the instructions where not; always decoded by `json.read`'s decoding; one bounded repair retry |
| Prompts | `type X = Prompt { ... }` with its text in `prompts/<name>.prompt`; placeholders checked as translations' are; free text only ever sent as delimited data |
| Embeddings | A `Vector { dimensions: n  distance: ... }` field in a pgvector column, an HNSW index derived from the queries, `order: [F nearest v]` returning `Nearest<T>` |
| Privacy | Every personal-data category sent to a model listed on the connection; `to_models: false` on a field; prompts and answers never logged by default |
| Limits | Output tokens per call; tokens per request; tokens and an estimated cost per day, for the app and per caller |
| Tests | One fake for every `ai` call; scripted answers and recorded ones; never a real model |

| Page | What it covers |
|---|---|
| [The ai capability](/docs/the-ai-capability.md) | Ask a model for a typed answer does the whole job, start to finish. |
| [Typed output](/docs/typed-output.md) | `ai.generate(prompt, as: T)` never hands back a value that hasn't passed every one of `T`'s rules. |
| [Prompts as files](/docs/prompts-as-files.md) | A prompt is what your app sends a model. |
| [Streaming from a model](/docs/streaming-from-a-model.md) | `ai.stream` sends a model's text to the client as it is written, rather than waiting for the whole answer, as a chat app does. |
| [Embeddings and search](/docs/embeddings-and-search.md) | An embedding is a vector, a list of numbers, that stands for a text's meaning: texts that mean much the same have vectors close together. |
| [Asking for an embedding](/docs/asking-for-an-embedding.md) | To search by meaning, your app needs each text's embedding. |
| [Keeping embeddings current](/docs/keeping-embeddings-current.md) | When a record's text changes, its embedding must be made again. |
| [The vector index](/docs/vector-index.md) | You don't write an index for a vector search. |
| [Without pgvector](/docs/without-pgvector.md) | Vectors need PostgreSQL with the pgvector extension, which the major hosted services offer. |
| [Safety and privacy](/docs/safety-and-privacy.md) | By default, your app may send a model no personal data at all. |
| [Testing AI calls](/docs/ai-tests.md) | A test never reaches a real model. |
| [Recording an answer](/docs/recording-an-answer.md) | `jadpo ai record` asks the model once, in development, and keeps its answer as a recording a test can answer with. |
| [What AI leaves out, and what to use instead](/docs/ai-left-out.md) | Some things you might expect aren't part of Jadpo's AI support. |

[What is built](/docs/ai-built.md) says which of this works today.

## Details

The exact rules, for when you hit an edge case.

| Page | What it covers |
|---|---|
| [Generating, its fake and the local driver](/docs/generating-its-fake-and-the-local-driver.md) | For when a setting is refused or a call surprises you: the full rules for `ai:`, `ai.generate` and the built-in `Driver.openai_compatible`, and for how the fake stands in for them in tests. |
| [How budgets are counted](/docs/how-budgets-are-counted.md) | The full rules for `ai: { limits: }` and how each call is counted against them. |
| [Following personal data](/docs/following-personal-data.md) | The full rules for how the compiler follows personal data to a model. |
| [Embeddings and recordings](/docs/embeddings-and-recordings.md) | The full rules for `ai.embed`, `ai.embed_all` and recorded answers. |
| [A package filling ai](/docs/a-package-filling-ai.md) | The full rules for a provider package's `provides/ai.jadpo`, which maps `ai`'s operations onto the provider's. |
| [Streams, embeddings and the OpenAI and Anthropic packages](/docs/streams-embeddings-and-the-openai-and-anthropic-packages.md) | How a provider package maps streams and embeddings, and what Jadpo's own OpenAI and Anthropic packages send. |
| [The conformance suite](/docs/the-conformance-suite.md) | `jadpo package conform <package> ai` checks that a provider package fills `ai` as the capability says, before the registry lists it. |