TokenUsage

The tokens one call to a model used: what it read, what it wrote and what its cache served.

Use it to track what model calls cost. For one number alone, use TokenCount.

Shape

Write every field when you build one, and give a field that isn't required as none.

Its parts keep these rules, and a value that breaks one fails with the rule's name:

• cached_within_input: cached <= input, where it is given

Example

TokenUsage { input: 1200 output: 350 cached: none }

Good to know

• cached is none where a provider doesn't say. It can't be above input.
• == tells you whether two are the same. < and list.sort_by don't take it, so sort by one of its parts instead.

In JSON

An object of its parts, in a request, a response or json.write:

{"input":1200,"output":350,"cached":null}{"input":1200,"output":350,"cached":800}{"input":0,"output":0,"cached":0}

When you read one, a field that isn't required may be left out, or sent as null.

Details