# `conform.failed`

A package's answers to `jadpo package conform` don't fit the capability it fills, or the suite couldn't run.

Area 14, [Tools](/docs/tooling.md); not declinable; the next step is usually the agent's.

## Why Jadpo refuses it

`jadpo package conform <package> ai` runs the `ai` capability's
conformance suite through the package's `provide ai` mapping: a typed
answer, a refusal, a cut-off, a stream, an embedding and the usage
reported. Without `--live` it answers from the package's recordings,
served by a local fake, so nothing leaves the machine. A case that fails
means the app would misread what the provider sends, such as taking an
answer the model stopped short as whole. So the suite fails, says which
case and why, and a package isn't published until it passes.

## What it looks like

An app asking its models through its own provider package, Acme
Models, whose mapping says how a refusal looks but not a cut-off:













config.jadpo:

```jadpo
config {
    acme_host: Url {
        default: "https://api.acme-models.example"
    }
    acme_key: Text {
        secret: true
    }
    drafting_model: ModelId {
        default: "acme/acme-small-1"
    }
}
```

app.jadpo:

```jadpo
application {
    connections: {
        acme: {
            package: AcmeModels.v1
            domains: [support]
            credential: config.acme_key
            host: config.acme_host
        }
    }
    ai: {
        connection: acme
        models: {
            default: config.drafting_model
        }
        limits: {
            per_day: {
                tokens: 2000000
            }
        }
    }
}
```

The package's recording of `messages.create` is in its `recordings/`
folder, which the local fake answers from:

```sh
mkdir -p shared/providers/acme_models/v1/recordings
cat > shared/providers/acme_models/v1/recordings/messages.create.json <<'EOF2'
{
  "stop_reason": "{Text}",
  "text": "{Text}",
  "usage": {
    "input_tokens": "{Int}",
    "output_tokens": "{Int}"
  }
}
EOF2
jadpo package conform acme_models ai
```

```text
error: `AcmeModels.v1`'s cut-off doesn't conform to `ai`
  reason: The mapping says nothing of a cut-off, so an answer stopped at `max_output:` would be read as whole.
  next: Correct `provides/ai.jadpo` against the provider's documentation and the package's recordings, then run it again (agent)
  rule: conform.failed
  help: https://jadpo.dev/docs/errors/conform.failed

Conformance suite
  typed answer  passed                        decodes as ConformanceAnswer
  refusal       passed                        refused, for its mark
  cut-off       failed                        no cut_off_when:
  stream        skipped                       AcmeModels.v1 maps no stream:
  embedding     skipped                       AcmeModels.v1 maps no embed:
  usage         passed                        12 in, 7 out, 0 cached

ai doesn't conform: 1 failed, 3 passed, 2 skipped, its recordings and a local fake
```

`(agent)` at the end of the `next:` line means your coding agent can make the fix itself.

A case without a recording to answer from is refused the same way, and
its `next:` line says to record it against the provider's sandbox; a
suite that couldn't start, for lack of memory or of a free loopback
port, says what to free.

## How to fix it

**If the provider marks a cut-off in its answer**, as Acme's
`stop_reason` does, say how in the mapping:

```jadpo
provide ai {
    generate: {
        ...
        refused_when: {
            path: "stop_reason"
            equals: "refusal"
        }
        cut_off_when: {
            path: "stop_reason"
            equals: "max_tokens"
        }
        ...
    }
    ...
}
```















The suite's cut-off case then passes, and an answer the model stopped at
`max_output:` is read as cut off rather than whole.

## Related

- [`jadpo package conform`](/docs/commands-for-packages.md#jadpo-package-conform)
- [The conformance suite](/docs/the-conformance-suite.md)