Start here

This set of eleven short documents describes Project Studio, BAML's observability initiative. It is the reader-facing entry point, replacing the earlier data-model-and-query-examples document. The documents build the execution model first and introduce vocabulary one document at a time; no term is used before it is defined.

This document reads like a BEP: prior art, the problem, the design, worked examples, and the artifact an agent loads. Cloud-internal engineering (ingest pipelines, how the hosted query tables are rebuilt, outage behavior of the hosted service) is not covered here. Document 08 states what to trust and where that internal design lives.

The two layers

Every table in Studio belongs to one of two layers.

The complete layer is never sampled, so its counts and rates are exact. When capture itself loses data, the loss is recorded, not hidden (document 06). The retained layer is never complete; it is an explicit selection. Every table belongs to one layer or the other, and the documents state which.

The documents

Reading paths:

• Full review: 01 → 11 in order. Roughly 90 minutes.
• Verify the data model: 02 → 03 → 04 → 06, then 09.
• See the user-facing result first: 01 → 10 → 11.
• Check schemas: 09, with 03/04/05 as background.

How claims are labeled

Every load-bearing claim carries one of three labels. The design corpus forbids presenting an implementation default as product policy; the labels keep the two distinct.

• built: on this branch today. Numbers under this label (the 250 ms summary-update interval, the 16 MiB in-memory event buffer, capture rate limits; documents 03–04) are implementation defaults, reported so behavior can be reasoned about. They are not frozen contracts.
• v1: a committed target with a delivery gate. Designed, not built.
• open: a decision not yet made. The document lists the options. Some open items wait on benchmark data and some are deliberately deferred; the surrounding text says which.

Names in this set

This set uses proposed reader-facing table names throughout. Each is a rename proposal open, mapped here and again where the table is defined in 09:

"CCT" (calling-context tree) remains the engineering term. Document 03 defines it so the internal design documents stay readable.

Where this set sits

• The design authority is CANONICAL/design/; nothing here overrides it. Where this set simplifies, it says so. Where the design is unfrozen, the claim is labeled open.
• query-examples ../query-examples.md remains the schema-level reference behind document 09. This set is the entry point; that file is the appendix.

01: The problem with collecting data

Key points

• Four questions follow an LLM misbehavior: counts across every call, the exact failing call in context, what data is missing, and which compiled code produced it.
• Per-call records answer the first two at a cost that grows with traffic. Studio does not create a row per call by default.
• OTel, Langfuse, Logfire, and Braintrust are bolt-on record-per-operation designs. None provides complete counting, loss accounting, and compiled-version identity together.
• Studio keeps bounded summaries of every call, plus exact evidence for a policy-selected few.

Four questions

A small BAML program processes customer signups: one function takes a batch of customers and, for each, validates the email address and calls an LLM function to classify the customer as Approve, Review, or Reject. Yesterday's batch contained Ada, Bo, and Cy. Bo's classification call failed (HTTP 500 from the model provider) and the code fell back to a safe default. Cy's call succeeded but took over six seconds. Doc 02 shows the program in full; every document in this set uses it.

1. What happened across every call? Did the classifier fail once today or a thousand times? Is six seconds an outlier or typical? These are counting questions, and a count over a sample is an estimate, not an answer.

2. Which exact call went wrong, and what surrounded it? For Bo: the failing call, its actual input, the actual provider error, and what else the program was doing at that moment. Totals cannot answer this.

3. What is missing? Acting on the first two answers requires knowing whether the collection machinery itself dropped anything. A dashboard that silently omits lost data misleads the reader.

4. Which code produced it? Behavior changed; did the code? Each observation should carry the exact compiled version that produced it, so before/after comparisons are a query rather than a reconstruction.

Every telemetry system trades off across these four questions (telemetry: the data a system emits about its own execution). LLM applications make the trade-offs unusually hard, and each existing tool gives up a property Studio requires.

What LLM applications do to telemetry

For ordinary web services, the interesting telemetry is mostly shape (status codes, latencies, counts) and payloads are small. LLM applications differ in four ways.

The interesting data is values. Diagnosing an LLM misbehavior requires the exact prompt, the structured output, and the error body. For Bo, the useful evidence is the provider's error and the triggering input, not a status code.

Values are large. A rendered prompt is commonly kilobytes; contexts and structured outputs can be far larger. A pipeline built for 200-byte log lines refuses, truncates, or charges heavily for a 20 KB prompt on every call.

Failures are bursty. During a model-provider outage, calls fail everywhere at once, exactly when telemetry is under maximum load and most needed. A design that captures little in good times and floods in bad times performs worst when it matters most.

Capture completeness is a first-class question. People act on this data: refund a customer, roll back a model, declare an incident. That requires knowing whether the evidence is complete, so question 3 is not optional in this domain.

The three-way tension

Three properties are needed at once:

1. Complete counting: every call counted, no sampling; question 1 gets an exact answer.
2. Exact evidence: real inputs, outputs, errors, and surrounding context for the calls that matter; question 2 gets a real answer.
3. Bounded cost: overhead low enough to leave on everywhere, storage that follows program shape rather than call volume, no cost explosion during incidents.

The most direct design, one record per call with its data attached, delivers the first two and fails the third: its cost is proportional to traffic by construction (doc 03).

The industry's standard alternatives each give up one property:

Question 3 is absent from all three rows. A sampled backend returns rows without reporting what fraction of reality they represent, or whether the pipeline dropped data on the way in.

Prior art

Three tool families dominate this space. Each is good at what it was built for; none was built for the combination above. The design corpus records almost no analysis of these products; most of this section is fresh analysis from general product knowledge and needs review against current vendor documentation before publication.

OpenTelemetry, Langfuse and Logfire, and Braintrust

How each behaves on the properties Studio needs:

The short cells compress real nuance. OTel sampling turns "how many failures today?" into an estimate and loses rare events first. A Langfuse helper left undecorated does not exist in the data, so no counting question covers it. Braintrust answers "how good are my outputs?", not "what happened across every call?". Logfire inherits the OTel span model, so its attribute-size and sampling constraints apply directly.

A few claims are recorded in the corpus rather than fresh. The one recorded observation about OTel-native vendors notes the same preconditions: flat span attributes, server-only operation, acceptance of silently-empty query results. OTel standardized no query language (every backend picks its own), and Studio's design supports OTLP export only as an explicitly lossy translation, never the source of truth. One of Studio's stated goals is to collect everything Braintrust does, and more.

Instrumentation is chosen in advance

Which trace is needed becomes clear only after an incident. Every instrument-first tool requires choosing upfront what to wrap, log, and sample, and incidents regularly occur where nothing was instrumented. A system that counts every call and preserves interesting moments automatically does not require predicting what will matter.

What all three share

One architecture underlies all three: a library added to the application emits one record per interesting operation to a backend. Three consequences cannot be fixed from outside: cost proportional to traffic, so completeness is traded for affordability; values stored as blobs attached to records, truncated or dropped under pressure; and a recording layer that neither sees what it did not wrap nor accounts for what it lost.

These are reasonable designs for tools that must work with any program in any language. Studio does not build on one of them because the properties it needs are properties a bolt-on cannot provide.

What the BAML runtime changes

BAML programs run inside a runtime the language owns. The runtime sees every function call without wrapping, all spawned concurrent work and its suspensions, and every value crossing a function boundary as a typed BAML value rather than a string blob. It knows the exact compiled program, down to a content hash, because it compiled it. This substrate is on the current branch today and always on built.

Studio's position is that telemetry should be a language feature, not a bolt-on. An observer inside the runtime answers all four questions. In-process summaries count every call without shipping records anywhere (question 1). Evidence is selected by policy, not by a 1% chance (question 2). The observer knows what it dropped, so "what is missing?" is a query (question 3). The runtime knows its compiled identity built, so no hand-updated version tag is needed (question 4). Querying all of it in plain SQL through one baml query command is the committed v1 target v1.

The design has two layers, used throughout this set:

Why Studio does not build on OTel

Each property above lives where OTel is unopinionated or absent. Complete counting requires not emitting a span per call: a rejection of the data model, not an extension of it. Typed, deduplicated values need more than attribute bags. Loss accounting has no standard channel. Interoperability survives: Studio's design includes bounded, policy-aware OTLP export, explicitly lossy and never the source of truth v1. Studio's data can be exported to an OTel world; it could not have been collected through one.

Runs, not traces

Studio does not use the industry word "trace". A trace is a correlation: spans from scattered instrumentation, stitched together afterward by propagated IDs, usually sampled. Studio's unit is a run: the runtime's own record of one top-level execution, recorded whole by the runtime that executed it, with any loss declared by the recorder itself. Nothing needs stitching. Doc 02 defines runs precisely (with processes, calls, and threads) using the batch program above.

02: What is actually running

Key points

• A run is one top-level BAML execution; the runtime records every call and thread it causes as one causal graph it owns.
• A run's outcome is independent of its calls' outcomes; a run fails only when an error escapes to its root.
• total_calls counts the entrypoint call plus every descendant on every thread; a batch of any size is one run.
• The runtime produces the call tree itself rather than stitching emitted spans, so Studio says run, not trace.

Doc 01 described cheap summaries of every call and exact records of the interesting few. This doc defines the units involved: process, run, call, logical thread, and revision.

The example program

Every document in this set uses the same program. It processes a batch of customers (validate each email, classify with an LLM function, fall back if classification fails) and spawns one background audit task.

class Customer {
  name string
  email string
  plan string
}

enum Verdict {
  Approve
  Review
  Reject
}

// An LLM function.
function ClassifyCustomer(c: Customer) -> Verdict {
  client "openai/gpt-5"
  prompt #"
    Decide whether to approve {{ c.name }} on the {{ c.plan }} plan.
  "#
}

// A plain helper. Classification errors are handled with a fallback.
// (validate_email and WriteAuditLog are ordinary BAML functions,
// defined elsewhere in the program.)
function ProcessCustomer(c: Customer) -> Verdict {
  validate_email(c.email)            // throws on malformed email
  ClassifyCustomer(c) catch (e) {
    _ => Verdict.Review,             // degrade gracefully
  }
}

// The entrypoint. Spawns one background audit thread, then maps.
function ProcessCustomers(customers: Customer[], batch_label: string?) -> Verdict[] {
  spawn { WriteAuditLog(batch_label) }
  customers.map(ProcessCustomer)
}

The program has the features that make LLM applications hard to observe: batch fan-out, a helper that can throw, an LLM call that can fail or stall, concurrency, and an optional argument.

One process, two runs

Deployed as a server, one operating-system process has handled one batch and is in the middle of another. The document set uses readable placeholder IDs (run1, call3) throughout:

Later docs represent this same picture at a million times the volume.

Process

A process is the operating-system process the program runs in: a server, a CLI invocation, a test runner. It hosts the BAML runtime, which records what executes inside it built. A process hosts zero or more runs; they start and finish independently. P has finished run1 while run2 is still running. The process is the host, not the unit of work: P being alive says nothing about whether any work succeeded.

Run

A run is one top-level BAML execution: everything caused by one outside invocation of the program (one baml run of main(), one served request, one test case) built. Every call and thread that invocation causes belongs to the run, recorded as one causal graph the runtime owns end to end. (Runtime internals call the root of a run a boundary; the engineering docs use that word.)

A run has its own outcome: run1 succeeded, run2 is still running. A run's outcome is independent of its calls' outcomes. run1 succeeded even though call6 inside it failed: ProcessCustomer caught the provider HTTP 500 and returned the Review fallback. A handled error is a fact about a call, not a verdict on the run.

Run run3, also hosted by P, shows the other direction. (Call IDs are scoped to their run: run3's call3 is a different call from run1's call3.)

Eve's email is malformed, validate_email throws, and nothing catches the error, so it reaches the root and the run fails. A run fails when an error escapes it, not when something inside it goes wrong. Runs, calls, and threads each end with their own status: succeeded, failed, cancelled, and rarer endings such as crashed. The status spellings in the query surface are not frozen yet; current code uses overlapping failed/crashed terms in places open.

Runs are not processes

P hosted a succeeded run, a running run, and a failed run at once. The process is where work happens; a run is one unit of that work with its own beginning, end, and outcome.

Entrypoint, call, and child call

The entrypoint is the function whose invocation roots a run: here ProcessCustomers. Every run has exactly one entrypoint call; in run1 it is call1.

A call is one function invocation inside a run. A child call is a call made by another call: call4 ClassifyCustomer(Ada) is a child of call3 ProcessCustomer(Ada). Every call except the entrypoint call has exactly one parent, and every call ends with its own status. run1 contains eight calls: root call1, spawned call2, and three helper-plus-classify pairs call3–call8. The runtime records each call's start and end as it happens built.

customers.map over three customers is one run with many calls, not three runs: Ada, Bo, and Cy are calls call3–call8 inside the run1 invocation. A batch of a million customers is still one run: one entrypoint call, roughly two million calls under it.

total_calls counts the entrypoint call plus every descendant, on every thread of the run. For run1, total_calls = 8 and total_errors = 1: a handled error still counts, so call6 is included. For run3, total_calls = 3 and total_errors = 2: one ValidationError ended both call3 and root call1 as failed. total_errors counts calls that ended in an error, not distinct errors; one propagating error can end several calls. Later docs use these counters as columns.

Accounting note: validate_email runs inside every ProcessCustomer, but this set folds its work into the caller's frame instead of recording a separate call. That is why run1 counts eight calls, not eleven, and why run3's failing frame is call3 even though validate_email threw. Doc 07 returns to this with the code-identity tables.

The call tree

The call tree (or call structure) is the full set of parent/child relationships among a run's calls: the indented picture above. The runtime does not reconstruct the tree from clues after the fact. It produces the tree, because it creates every call built. The "Runs, not traces" section below rests on this point.

Logical thread

A logical thread is a lane of concurrent execution scheduled by the BAML runtime: concurrency as the program sees it, not an OS thread. spawn creates one. In run1, WriteAuditLog runs on logical thread thread2 while the map proceeds on the main thread. The runtime may move the work across OS threads; the logical thread is the stable identity of the concurrent activity. Thread starts and ends are recorded like call starts and ends built.

spawn is the verb; a logical thread is what it creates. Internal design docs historically used "spawn" for both.

In run3, the run ended while the audit thread was still running, so the runtime cancelled it. Cancellation is a normal ending, not an error.

Revision, the short version

A revision is the exact compiled program a run executed: the content identity of the BAML sources, schema, and compiler inputs. Any source or type-definition edit produces a new revision. Deployment, release, and git labels are dimensions attached to a revision, not the identity itself. Every run records the revision that produced it built: run1, run2, and run3 all ran on rev1, so when their behavior differs, the code is not the variable. Doc 07 covers how functions keep their identity across revisions and what a revision can and cannot prove.

Run boundaries

One run cannot contain another

Calls nest; runs do not. If the program calls another service that itself runs BAML, that service's work is a second run in its own process, linked to the first by an explicit relation rather than merged into one graph v1. Two processes have two clocks, and Studio does not merge two clocks into one timeline. total_calls never includes anything from a related run.

A long-running server hosts many runs

Each hosted run finishes on its own schedule, and a run is visible while it is still running: run2 appears today, with its so-far numbers, not after it ends built. If the process dies mid-run, the run does not get an invented ending: it is classified as crashed when read, from the absence of a recorded completion built.

Runs, not traces

The industry word for the picture of one request's execution is a trace; doc 01 introduced how trace-shaped tools see the world. Studio does not use the term, for the following reasons.

A trace is a correlation. Instrumented code emits spans, context IDs propagate across function and service hops, and a backend later stitches matching IDs into a tree. The structure is inferred from the stitching, so it is only as complete as the propagation: a missed span orphans its subtree, an unpropagated ID splits one execution in two, and under sampling most requests produce no tree at all.

A run is not a correlation. The runtime creates every call and every logical thread, so it does not discover the structure; it produces it. There is nothing to stitch and nothing to sample away: the tree is recorded as it is created, never inferred after the fact built. Structure can still be lost (a process can die mid-run, and a recorder under extreme load can drop records), but the failure differs in kind: the loss is recorded as a typed, queryable fact, and the tree is marked incomplete rather than silently smaller (doc 06 shows where those facts land) v1.

Calling this a "trace" would mislead in both directions. It would undersell the guarantee: readers would assume sampled, stitched, possibly-incomplete trees and design around silent holes that do not occur here. It would oversell the scope: "trace" implies one merged picture across services, which Studio does not fabricate; cross-process work is a related run, linked and separately clocked, never spliced into one timeline v1.

The runtime sees every call the moment it happens but does not keep every call. Doc 03 describes counting every call without storing every call.

03: Counting every call without storing every call

Key points

• A per-call fact table would answer any question, but its cost grows with traffic; Studio aggregates over the program's call graph instead.
• The complete layer keeps one aggregate row per calling context per run; every call folds in, so counts are population totals, not samples.
• Aggregate rows hold counts and summed times but no thread identity, no ordering, and no per-call data; the table grows with program shape, not with traffic.

Doc 02 described a process hosting runs, each run a tree of calls, some on spawned logical threads. "Which functions fail most" and "where did the time go" are questions about all of those calls. This doc covers how Studio counts every call without storing every call.

Programs repeat themselves

A one-row-per-call table can answer any question, and the decision register rejects it: its cost grows with traffic, so there is no default row per call, anywhere. v1 The current engine summarizes a five-million-call run in about 4.5 KiB on disk at about 74 nanoseconds per call (measured on one development machine, not a guarantee) built. Storage follows program shape, not call volume.

This works because a million calls are a few code paths taken repeatedly: ProcessCustomers calls ProcessCustomer, which calls ClassifyCustomer, whether there are three customers or a million. A calling context is one such path: a function plus the whole chain of parents above it, down from the run's entrypoint. The whole chain defines it (ClassifyCustomer under ProcessCustomer is a different context than the same function elsewhere), and it is a location in the call tree, not a moment in time or a thread. Run run1 has exactly four calling contexts.

Every call belongs to exactly one calling context, and the complete layer keeps one row per calling context per run. Each call folds its numbers into its context's row (one more start, one more success or error, its duration) and is then forgotten. Because every call folds and none are skipped, the rows are the entire population of calls, not a sample. This is doc 01's complete layer; its counts are exact totals for the traffic. Retained-layer counts (doc 04 onward) are a lower bound, never a total.

Run run1, folded: 8 calls, 4 rows

The aggregate rows for run run1 in the proposed reader-facing table, called calling_contexts. (Row labels like context1 are readable placeholders; real identifiers are opaque.)

The context4 row shows three things.

• Aggregation. Ada, Bo, and Cy all folded into it: three classifier calls, one row. With a million customers the table would still be these four rows, with started = 1,000,000 in context3 and context4. The complete layer grows with program shape, not traffic.
• Handled errors stay visible. context4 shows errored = 1 even though every ProcessCustomer above it succeeded: Bo's classification failed with a provider error, the code handled it, and the run succeeded, but the failure stays in the record. A fallback that starts firing on a large share of traffic therefore remains visible.
• Await. The time columns sum across the folded calls (inclusive from start to end, self in the function's own code, await suspended and waiting). context4's 8.20s is almost entirely await: most LLM latency is time spent waiting, not computing. Doc 09 has the full column reference.

For contrast, here is failed run run3, where Eve's malformed email made validate_email throw and nothing caught it:

Two details. There is no ClassifyCustomer row: a context row exists only for paths that ran, and Eve's run never reached the classifier. And both context1 and context3 show errored = 1 from one propagating error: the counters record how calls ended, not how many distinct errors existed, and one error passing through two calls ends both. Whether that error is captured once or twice as evidence is a doc 04 question.

How this stays cheap at runtime

1. A call start or end appends one small fixed-size record to its logical thread's buffer: no formatting, no file write, no network.
2. One background thread drains the buffers and folds each event (starts as well as ends) into its calling-context row, so an open call's row advances while the call runs.
3. Every 250 milliseconds, changed rows are flushed to local disk (250 ms is an implementation default of the current code, not a product promise). built
4. When the run ends, its rows are folded a final time and written as a finished summary that is never modified again.

The observed program pays only for the append.

Why folding happens before storage

Aggregating call rows in a warehouse still pays for the rows themselves: written on the hot path, shipped, stored. Folding happens in-process, before anything is stored, so by default the per-call row never exists: not on disk, not on the wire, not in a warehouse. The exceptions are deliberate and bounded: the retained layer keeps exact records for the interesting few, and an explicitly opt-in debugging mode can write everything; both belong to doc 04, not to this layer's population path. Because there is no traffic-proportional cost, this layer is always on rather than sampled.

What the aggregate deliberately leaves out

Everything an aggregate row omits grows with traffic; everything it keeps grows with program shape.

• Thread identity. Async work moves between execution lanes as it suspends and resumes, so "which thread" is not a stable property of a calling context, and keying rows by thread would turn ten thousand identical spawned workers into ten thousand rows. Per-thread detail is kept for the interesting few (doc 04).
• Display paths. A row links to its parent row; the tree is reconstructible, and the display string is a rendering concern.
• Per-invocation data. No individual timestamps, ordering, durations, or argument/return data: a row can say one call errored, not what Bo's call contained (docs 04 and 05).
• Time buckets. A row is a location in the call tree, not a time bucket: a million calls over two hours are still one row per context, with counts advancing while the run is open and final when it ends.

When paths multiply

The table grows with the number of distinct paths, which is the design's weak point. Recursion past depth 512 (an implementation default) reuses the nearest matching ancestor context, visibly flagged: counts and times stay exact, only path uniqueness coarsens; ten thousand identical spawned workers likewise share one row. built In the corpus measured during design, the 99th-percentile program produced about 3,500 contexts: kilobytes. That is not a guarantee: highly dynamic call graphs grow the table, and keeping path count bounded on real workloads is a release gate. v1 Separately, some folded counters can today saturate at a fixed width without an explicit overflow marker; fixing that is a v1 correctness gate. v1

Internally this structure is the calling-context tree (CCT) and the table is cct_population; calling_contexts is this set's proposed public name (docs 00 and 09).

04: Keeping the interesting ones

Key points

• Aggregate rows count every call but identify none; exact evidence records individual calls and moments, selected by policy, never by traffic volume.
• The rolling tape keeps recent structural events in a fixed block of memory, recording execution shape only, never data bodies.
• A trigger (an unhandled error at the run root, a slow call, or a manual request) copies the tape into a durable, rate-limited dump and can mark individual calls for retention.
• A retained call is a single invocation kept with a retention reason; counting retained calls measures retention policy, never traffic.

How evidence is kept

Studio keeps exact evidence in three steps:

1. The runtime appends a small structural event for every call to a bounded in-memory tape.
2. A trigger fires: an unhandled error reaching the run root, a call crossing the latency threshold, or a manual request.
3. The trigger produces two kinds of durable evidence at once: it copies the current tape window into a dump (a preserved moment, one ledger row in tape_dumps), and qualifying calls get retained_calls rows (preserved individuals, each with a retention reason).

The sections below give the details.

The question aggregates cannot answer

Doc 03 produced four aggregate rows for run run1. The context4 row (ClassifyCustomer) shows three calls started, two succeeded, one errored, and almost all of its 8.20 seconds spent awaiting. The row is exact, complete, and by design anonymous: it cannot say which call errored, which was slow, or what else the run was doing at the time.

The complete layer answers "how often, how slow, how many errors?" for every call. Questions about a specific call need the second layer: exact evidence about individual events, kept only for the interesting few.

Two kinds of exact evidence exist, both selected by policy rather than traffic volume: a preserved moment (a tape dump, one ledger row in tape_dumps) and a preserved individual (a retained call, one row in retained_calls). "Tape dumps and retained calls" below introduces the two together, after two prerequisites: the structural event and the rolling tape.

Shape, not data

Structural events are not function inputs and outputs. The shape/data distinction carries through the rest of the document set.

A structural event is a small, fixed-size fact about the shape of execution: a call started or ended, a logical thread was spawned, paused awaiting an LLM response, or resumed. Each event names what it relates (call, parent call, function, logical thread) plus a timestamp, and never carries the data flowing through the program. built

Structural events, in reader terms:

The common call start/end pair costs 54 and 26 bytes; no record exceeds 292 bytes. built Because records are small and bounded, the runtime produces one for every call (the same stream that feeds doc 03's fold) at a cost independent of the program's data.

Values (argument, return, and error bodies) are arbitrarily large user data, handled by separate machinery with separate rules (doc 05). The tape contains shape, never data: a saved slice can show a ProcessCustomer call failed at 0.19 seconds on thread thread1, not the customer's email address.

The rolling tape

A busy service produces millions of structural events. Storing all of them would reintroduce the per-call storage cost that doc 03 rejects, so the runtime keeps a bounded window of recent events instead.

The rolling tape is a bounded, in-memory sequence of the most recent structural events. New events append at one end; once memory is full, the oldest are overwritten. Nothing on the tape is durable; the cost is a fixed block of memory. built

                             overwritten ──▶ gone forever
        ┌──────────────── rolling tape (fixed memory) ────────────────┐
 ...e408│ e409  e410  e411  e412  e413  e414  e415  e416  e417  e418 │◀── newest
        └─────────────────────────────────────────────────────────────┘
              nothing here is durable until a trigger promotes it

In the implementation this is the flight recorder, a ring of recent raw event bytes, plus a small recent-call ring of the last few thousand completed calls per spawn tree (one per run in the example program). built The rest of this document set says "the tape."

Two implementation defaults on the current branch (not product policy) set the trade-off. The tape is 16 MiB, roughly 200,000 call start/end pairs: about 11 seconds of history for a typical busy agent workload, about 21 milliseconds for a pathological hot loop. built The asymmetry is intentional: a hot loop that floods the tape is still counted exactly by the aggregate rows from doc 03. The tape provides context, not counting.

The same bound covers the case of very deep or very busy runs: the tape never grows, so it cannot be overwhelmed. A burst of events means less wall-clock coverage.

Tape dumps and retained calls

A trigger is the policy decision that an instant deserves exact evidence: an unhandled error reaching the run root, a call crossing the latency threshold, or a manual request ("What fires a trigger", below, gives the policy). A firing trigger leaves durable records in two tables at once.

• It copies the current tape contents into a durable saved tape slice (a tape dump) before the ring recycles them, with one ledger row in tape_dumps recording what fired the dump, which run it belongs to, and what span of activity it covers. A dump preserves a moment: a window of time and every call and thread inside it, as structural events only, with no argument, return, or error bodies. A dump answers "what else was going on at that moment?"
• It can mark individual calls for retention: one row each in retained_calls. A retained call preserves an individual: one invocation with a durable, queryable record of identity, timing, outcome, and links to whatever else is known about it; captured values attach under doc 05's rules and surface as per-role fields on the row (doc 09). Every retained call carries a retention reason, and a trigger is one source of reasons among several, so a row can exist with no dump behind it. A retained row answers "show me that call."

The example program's two triggers fill both tables.

What run1 leaves behind

In run run1, Cy's ClassifyCustomer call call8 completes at 6.2 seconds and crosses the program's latency threshold. The trigger leaves one record in each table:

• dump1, a copy of the tape: ~130 events, the 6.2 seconds ending at the trigger.
• a retained_calls row for call8: ClassifyCustomer, succeeded at 6.2 s, retention reason "slow call".

The two records link rather than overlap. call8's row lists dump1, the dump that covers it; a dump never retains the calls inside it. dump1 holds every call and thread active in its window, but only call8 has a row.

A dump reaches backwards from its trigger, never forward: it is the tape's contents at the moment the trigger fires. It may therefore cover only part of a long run: a dump taken in hour two of a long run holds the tape's window, not the run's history.

run1 also leaves a second retained row, with no trigger and no dump behind it. call6, Bo's ClassifyCustomer call, failed with a provider HTTP 500 that ProcessCustomer handled, so no dump fired; but its error body is kept under the value rules for LLM functions (doc 05), and that earns the call a row. Its dump list is empty.

What run3 leaves behind

In run run3, Eve's malformed email makes validate_email throw; nothing handles the error, the run fails, and the error trigger fires at the root. It leaves one dump and two rows:

• dump2: ~40 events. run3 is short enough to fit inside the tape's window, so the dump holds the entire run.
• two retained_calls rows: call1 (ProcessCustomers, the root that observed the unhandled error) and call3 (ProcessCustomer, the frame that threw), both retained for the same propagating error.

call2, the audit write cancelled at run end, appears inside dump2 as an event and has no row of its own. Ada's healthy calls in run1 have no rows either; absence from retained_calls is the normal case.

The stored rows

For the example program, the two tables hold six rows in total: two in the dump ledger tape_dumps (a proposed reader-facing table; internal name: exact windows), and four in retained_calls, one per kept call.

The context column joins each kept call back to its aggregate row in the complete layer (doc 03).

The dump machinery runs today, and the tape_dumps SQL relation that exposes the ledger to queries is part of the table-schema freeze. v1 retained_calls is likewise a proposed reader-facing table v1; the capture machinery beneath it runs today. built

Retention reasons include: the call errored; it crossed the latency threshold; its events lie inside a saved tape slice; staged material about it was promoted by a trigger; a configured sampling policy selected it; a policy asked for it explicitly. The exact enumeration is part of the table-schema freeze. open

Dumps are rate-limited: currently at least 5 seconds apart and at most 16 per engine (one runtime instance; a process can host several, sharing one tape); both are implementation defaults. This keeps exact evidence from becoming traffic-proportional. A dump the limiter suppresses is counted, not silently skipped; the interval over which the 16-dump budget resets is a knob the branch docs do not pin. built

What the tape adds over a traceback

Every error object already carries a callstack. A traceback is one stack, on one thread, at one instant: the chain of frames waiting on the failing call, nothing else. The tape also shows what the rest of the program was doing. The slice dump2 preserves around Eve's failure (event names simplified; shape only, no bodies):

About ten of the ~40 events are the call and thread records shown; the rest are the pause/resume records (elided above) that an awaiting thread writes each time it parks and wakes. Those records are also why dump1, with 6.2 seconds of awaiting inside it, runs to ~130 events.

The traceback for this error says validate_email ← ProcessCustomer ← map ← ProcessCustomers and nothing more: not that a spawned audit thread existed, was still running when the error hit, or was cancelled by run teardown. If the underlying bug is "audit records go missing whenever a batch fails," the traceback never shows it; the tape shows it directly. dump1 in run run1 works the same way: it holds slow call call8 from first event to trigger, including the await gaps and the LLM metadata a stack cannot carry, and its 6.2 seconds of coverage reach back almost to Bo's provider failure, just over six seconds before the trigger. Sibling threads, just-finished calls, and suspensions are what the tape records and a traceback cannot.

How not to read retained_calls

retained_calls invites three misreadings.

Retained is not sampled. The set is not a uniform sample of traffic; even when a sampling policy contributes rows, it is one selection rule among several. The set is policy-selected and deliberately biased toward failures and outliers. Counting retained calls measures how many calls matched retention policy, never how many happened; retained counts are lower bounds on traffic. Population questions go to calling_contexts, which counts everything.

Retention does not spread. Retaining call6 does not retain its parent call5 or anything else in its ancestry or subtree; each call gets its own row or none. A retained call links to its calling context in the complete layer, so "how common is this path?" stays answerable, but the linkage is a join, not a bulk-retention rule.

retained_calls is not a health table. A retained call is not evidence that the application, or Studio, is unhealthy. call8 succeeded; it was slow. Whether Studio's own capture pipeline had problems is a different question with its own machinery, in doc 06.

What fires a trigger

The trigger is the selector both tables share: it seals the tape into a dump, and it can mark individual calls for retention, directly or by promotion (below). There are three kinds.

Errors: at the root, once. In run run3, Eve's malformed email makes validate_email throw. ProcessCustomer, the map, and the root each observe the error; nothing handles it, the run fails, and the runtime saves one dump, dump2, not three. The design constraint this document set adopts as the target: the error trigger fires where the runtime root observes an unhandled failure, not on every propagation hop, and one propagating error produces at most one dump no matter how many frames rethrow it. On the current branch, root error handling is what fires the dump built; the precise written contract (the exact propagation point, and the dedup rule giving one error one identity across rethrows) is not yet frozen. open

Bo's case in run run1 is the contrast. His ClassifyCustomer call (call6) failed with a provider HTTP 500, but ProcessCustomer handled it and returned the Review fallback. The failure never reached the root, so no error dump fired. That is correct behavior: a handled error is normal control flow, not an incident. The failure is still recorded in the aggregate row (context4, errored = 1), and its error body is kept under the value rules for LLM functions (doc 05).

Slow calls. A call that outlives a configured latency threshold fires a dump when it completes. The threshold is a knob, not policy: the only recorded default is an archive-historical 30 seconds, unconfirmed on the current branch, so this document set leaves the number unfrozen. The example program sets the knob near 5 seconds, so in run run1 Cy's ClassifyCustomer call (call8) at 6.2 seconds crosses it and produces dump dump1. built

Manual. The trigger machinery accepts an explicit "save the tape now" request, exercised today from engine-internal code. built How a user asks for one (a language builtin, an SDK call, or a CLI command) is not yet designed. open

A trigger can also perform promotion. Some material is staged cheaply in memory in case it becomes interesting; a trigger upgrades (promotes) the staged material connected to the failure into durable storage. Staging and promotion machinery exists and root-error promotion is wired built; automatically staging helper data so it is there to promote is committed but unfinished, and the v1 gate is to wire it or explicitly drop the promise v1. What gets staged and promoted is a value-capture question, covered in doc 05.

Why cancellation is not a trigger

In this codebase cancellation is routine: racing two futures cancels the loser every time, by design. If cancellation fired dumps, future.race would saturate the rate limiter with recordings of healthy behavior. In run run3 the audit thread was cancelled at run end; that cancellation preserved nothing. The error fired dump2; the cancellation appears inside it as an event. The design position is that cancellation must not be a default trigger. An older design draft had cancellation inside the error-trigger clause, so the final trigger matrix still needs to be written down and frozen. open

Where capture stops

Structural events exist for BAML function calls, logical threads, and the runtime's own operations: suspensions, resumptions, LLM request metadata. The tape does not record arbitrary host-language stdlib frames or the internals of external packages: the runtime records the execution it orchestrates, by construction, since the nine record types can only describe BAML VM lifecycle events and function ids are assigned by the compiler.

Built-in internal functions additionally have all value capture disabled by default, so the machinery never pulls library internals into captured data. built Depth is bounded the same way as volume: a million-call stack meets a fixed-size tape, which covers less time, while the complete layer counts all million calls exactly. The bounding is deliberate design, not an accepted limitation.

The raw firehose

One escape hatch exists for the rare case where a bounded window is not enough: an opt-in raw event stream that writes every structural event to disk, verbatim, with no window at all. It is off by default, and its cost is traffic-proportional by design. Its current job is to serve as the correctness oracle Studio's own developers use to validate the tape and the aggregates. built Whether a bounded "full trace" mode becomes a supported product feature, and how it would be packaged, is an open decision. open

05: Values: inputs, outputs, errors

Key points

• A value is the user data that flowed through a call, in one of three roles: args, return, or error. A failed call has an error and no return.
• Root and LLM calls capture all roles by default, helpers only errors, built-ins nothing. This is the implemented default, not product policy.
• Promote-on-error stages helper values in a bounded in-memory buffer and makes them durable only when a trigger fires. Helper staging is designed but not wired today.
• Identical value bodies are stored once, locally. Query tables hold facts about values, never the bodies.

Doc 04 covers the shape of a run: structural events on the rolling tape, counters in calling_contexts, rows in retained_calls. None of that contains user data. This doc covers the data itself.

Value roles: args, return, error

A value is the user data that flowed through a call: its arguments, its result, or the error it ended with. Values are the expensive half of observability: a structural event is a few dozen fixed bytes, while a value is arbitrarily large (a customer record, a rendered prompt, a 64-turn conversation transcript). Values therefore get their own capture machinery, policy, and storage.

Every captured value has a value role: the call's args, return, or error. Roles are not interchangeable. A failed call has an error and no return: not an empty or null return; the role does not exist for that call. In run run1, ClassifyCustomer(Bo) (call6) failed with a provider HTTP 500: args (Bo's Customer record), an error, no return. Its sibling ClassifyCustomer(Ada) succeeded: args, a return of Approve, no error.

(A fourth role exists for log bodies; the default host turns log capture off, so this doc sets logs aside. built)

The capture policy matrix

Capturing every value for every call would reintroduce the one-row-per-call cost problem doc 03 avoids, so a capture policy decides, per function kind and per role, whether a value is kept. The policy that ships today (everything below is on the current branch built except the two cells marked open):

Notes on the table:

• These defaults apply when the host (today, the CLI) enables value capture; it does by default. built
• Per-call capture flags in the language (capture(inputs=…, output=…, error=…)) override the policy. built
• This is the current implemented default, not product policy. Which roles are captured in which environments is an open policy decision (X1 in the register). open

In run run1, the root call1 captures its args (the customer list [Ada, Bo, Cy], batch_label = "aug-batch") and its return, [Approve, Review, Approve]. Each ClassifyCustomer captures args plus a return or, for Bo, the provider error. The three ProcessCustomer helpers and the spawned WriteAuditLog capture nothing: they succeeded, and helper inputs and outputs are off by policy. Eight calls ran; values exist for four, the root and the three LLM calls. The asymmetry is deliberate and recorded: doc 06 shows how a query distinguishes "off by policy" from "went wrong."

In run run3, ProcessCustomer(Eve) (call3) failed. Helper error capture is on, so the ValidationError is kept at call3 and, by the root's own error capture, at call1. Eve's malformed Customer argument was not captured at the helper: helper args are off. The bad email is visible in this run only because Eve's record sits inside the root's captured args, which is a property of this particular call tree, not a guarantee. Promote-on-error is designed to make it a guarantee.

Promote-on-error

Experimental. Promote-on-error is adjacent to the core system, and this section is not asking for schema-level review. Before the v1 gate closes, the plan is to benchmark what promoted values recover in real incidents against the staging cost, and drop the feature if the customer value does not cover the cost. Read what follows as a candidate design, not a commitment.

Studio does not retroactively capture every input and output up the call stack when something fails; that promise would hide traffic-proportional cost in the error path. Promote-on-error bounds the cost instead.

While a helper runs, its values may be staged: copied into a fixed-size in-memory buffer and tagged speculative, with no encoding, hashing, or disk writes. If the call finishes normally, the staged copy is dropped at no further cost. If a trigger fires (doc 04's triggers: an error the run's root observes, a slow call, a manual request), staged drafts in the failing subtree are promoted to durable storage, each tagged with the justifying trigger. Promoted is provenance (why the value was kept), not a fourth role: a promoted draft is still the args, return, or error it always was. Full capture cost is paid only for failures that mattered. The buffer is byte-bounded (32 MiB native, 8 MiB wasm; implementation defaults built) and evictions are counted (staging_evicted), so an undersized buffer is a visible, tunable fact, not a silent hole. built

Every staged draft ends in exactly one of three states:

The staging buffer, promotion machinery, and eviction counter exist on the branch, and a failure at the run's root calls promotion. built No production code path stages helper drafts, so "a failing helper's arguments are retroactively promoted" is not shipped behavior today; the delivery plan requires wiring helper staging or removing the promise. open Also unfrozen: which ancestors and descendants of a failing call are eligible for promotion, and how a partial promotion (some drafts promoted, some already evicted) is reported. open

Until that closes, the guaranteed places an error's data lives are those in the matrix: the error value itself (helpers included), everything the root captured, and everything LLM functions captured.

What a million failures store

Suppose a provider outage makes a million ClassifyCustomer calls fail.

Aggregates: everything, cheaply. All million failures increment counters on the same few calling_contexts rows. Counting is never sampled or turned off; a million failures cost the same rows as three. built

Tape dumps: a bounded few. Error triggers are rate-limited (at least five seconds between dumps, at most sixteen per engine, and a process normally hosts one engine; implementation defaults built), and dumps suppressed by the limit are counted.

Values: by policy, deduplicated. Each failing LLM call's error is captured per the matrix, and values are stored by content: a million copies of the same provider error are one stored body (see "Where values live"). Staged helper values, once wired, are bounded by the staging buffer, with evictions counted.

A rethrown error is stored once

In run run3, Eve's ValidationError was captured at two frames, call3 and call1: two small rows sharing one stored body. Capturing a propagating error at N frames costs N rows and one body; the cost is bounded by how many frames capture the error, never by how many rethrow it. built The stricter rule (that the rows are also linkable as the same error, one identity across rethrows) is the contract doc 04 marked open. open

Capture cost under an incident is therefore not bimodal: there is no mode where a bad day silently multiplies storage by traffic. Every bound is explicit, every bound gets a counter by design rule, and a hit limit is queryable: doc 06 introduces the table where Studio reports its own shortfalls (evidence issues). Two caveats. First, one bound has no counter yet: under extreme sustained record rates, the structural plane's 1 GiB memory cap currently aborts the process; the graceful degradation ladder meant to replace the abort is not wired. open Second, the full promise ("every value the policy selected is either captured or has an explicit loss marker, and every shortfall is queryable") is the committed target; today a few CLI failure paths are best-effort, and some degradations are not yet persisted into queryable diagnostics. v1

Distinctions that must never collapse

A value can be absent for four different reasons. Studio's rule is that the reasons must stay distinguishable.

Policy-omitted vs promised-but-lost. "Helper args are off by policy" is a healthy system following its configuration. "The policy selected this value but it could not be kept" is a capture failure and must leave an explicit loss record, never a quiet absence. The first is not_captured; the second is lost. Collapsing them would make no absence trustworthy. (The loss-marker contract is the v1 caveat above.)

Omitted optional argument vs explicit null. Run run2 called ProcessCustomers([Dee], null): the caller passed null for batch_label, so the captured args show a real null, ordinary data. Omitting the optional argument entirely would be a different fact, and the two must stay distinguishable in captured args and in queries. The exact representation (args-object shape, spelling of omission) is table-schema freeze work, not settled yet. open

Redacted vs missing. A redaction policy deliberately removes content (a password field, a customer email) while recording that it did so. A redacted value is not missing: the call happened, the value existed, the content was withheld on purpose. Which fields are redacted, and the allow/deny-list mechanism, are open policy work (X1) open; that the state "redacted" exists and is distinct is a committed schema requirement. v1

Privacy of default-on capture

The recorded reasoning for default-on capture of prompts and outputs: captured data never leaves the machine by default; uploading is a separate, explicit opt-in (doc 08); and seeing the exact prompt and output is the point of the product during debugging. The opt-outs are cheap and layered: turn history off, per-class capture flags, retention built, and redaction once the X1 policy work lands. open

These distinctions give each value role a vocabulary of availability states (preview: available, not_captured, omitted, redacted, lost, truncated, corrupt). Every retained_calls row will report one state per role, so a query never guesses why a value is absent. The full state list and exact column spelling are part of the table-schema freeze open; docs 06 and 09 give the full treatment.

Where values live

At capture time the value is deep-copied out of the program's memory into a profiler-owned buffer: a bounded copy and nothing else, so the running program never waits on encoding or disk. built At run end the drafts are encoded into canonical form and written to a local value store holding one copy per distinct content: identical values (the same args twice, the same error a thousand times) occupy storage once. Doc 08 names the mechanism and how data leaves the laptop. The dedup is measured: a 64-turn agent conversation, where each turn's transcript contains all previous turns, saw a ≥20× reduction from storing shared structure once. built (Some current artifacts still carry an inline legacy copy next to the canonical reference; "bodies live only in the value store" is the v1 target. v1)

"At run end" is literal today. The shipping CLI encodes and writes drafts once, synchronously, when the run's root call finishes; a promoted draft joins the same durable queue and lands on that write. built A run that crashes mid-flight loses the drafts still in memory: the structural tape survives with its intact prefix (doc 08's crash behavior); the values plane has no equivalent yet. An off-thread drain service that can encode values continuously while a run is open (needed by a days-long agent loop) exists, but the CLI does not use it, and when incremental value flush becomes the behavior is an open question. open

Two placement rules follow, and both affect queries:

Local first. Values live on the machine that ran the program; doc 08 shows exactly where. Nothing uploads by default; upload is explicit configuration, also in doc 08. built

Never rows in the query tables. retained_calls and the other tables hold small facts about values (which roles exist, their availability state), never the bodies. A query that touches args, return, or error fetches the body on demand from the value store (mechanism in doc 09). Value size never bloats the tables a query scans, and a table scan never accidentally reads customer data. v1 (settled semantics; the query tier itself is target work, doc 09)

What an LLM call records

The bar for LLM calls: capture everything an LLM-evals product records, plus what only the runtime can know, the calling context the call came from.

Each LLM call produces two kinds of record. Structurally (doc 04's tape and counters), the call carries a compact LLM marker: model, token counts, and provider/parse/retry flags (small fixed facts that fold into aggregates). built As values, LLM functions capture all three roles by default: input, output, and on failure the error. built Whether the exact rendered provider request and raw response are captured in addition to the BAML-level args and return is not specified anywhere in the design today; it falls inside the LLM-instrumentation rework noted below. open

The aggregate side has its own table, llm_usage: one row per run, per calling context, per provider and model. Run run1's row:

The row shows two things. The error columns are typed: Bo's HTTP 500 is a provider_error, distinct from a parse_error (the model answered, but the answer did not fit the declared return type); a "is the provider down or is my prompt bad?" query depends on the distinction. And token_state = partial: Bo's failed call reported no usage, and absent tokens are recorded as absent, not as zero, the same distinct-states rule as the rest of this doc.

The LLM instrumentation is being reworked; llm_usage is explicitly provisional pending that work. open

06: When the evidence itself is incomplete

Key points

• Missing evidence is a first-class fact, recorded, counted, and queryable in evidence_issues.
• Run status and evidence completeness are independent axes; neither implies the other.
• Every query ends with a typed outcome grading its own completeness; no outcome means no answer.
• Completeness is measured against capture policy's promise, not against everything the program did.

The failure mode observability tools hide

Every observability tool drops data sometimes: a buffer fills under load, or a process dies mid-write. Most tools drop silently. A dashboard shows 90,000 calls instead of 100,000 with no indication the other 10,000 existed; "it never happened" and "the record was lost" look identical. For an LLM application the ambiguity appears during incidents, when failure counts matter most.

Studio's rule is the opposite: missing evidence is a first-class fact. Everything Studio records about a program (aggregate rows, retained calls, tape dumps, captured values) is its evidence. When promised evidence is missing, the absence is itself recorded, counted, and queryable, like the data it stands in for.

Two different kinds of failure

An application error is the program failing while Studio watched. In run run1, Bo's ClassifyCustomer call errored with a provider HTTP 500. The error is counted in the calling_contexts aggregate (errored = 1 on the ClassifyCustomer context), the error value was captured under LLM capture policy, and the run's evidence is complete.

An evidence issue is Studio failing to watch: somewhere between the call happening and the query, the recording machinery hit a limit, lost data, or produced something it cannot vouch for, even if the program ran correctly. These failures go in evidence_issues, not an errors table: "error" is the program's word, and the separate table keeps the two apart.

Runs run1–run3 are healthy (zero issue rows), so this example row comes from a hypothetical run runX under heavy load:

run=runX  source=profiler  kind=structure  reason=records_dropped
count=10,000  first_seen=12:00:01  last_seen=12:00:04

The runtime's recorder (internally, the profiler) dropped 10,000 structural-event records over three seconds: one row, one typed reason, one count, not 10,000 rows. The aggregates for runX may therefore be missing calls; anything computed over the run is a lower bound, and this row says so.

What counts as an issue, and what does not

Four things belong in evidence_issues:

• Dropped records. Structural events discarded under pressure, as in the runX row.
• Lost values. A promised value capture (for example, an LLM call's prompt) that could not be kept. Losing a promise is an issue.
• Corrupt records. Stored evidence that fails integrity checks on read. Studio reports rather than repairing or guessing.
• Conflicting evidence. Two records that cannot both be true. Rare; surfaced, not silently resolved.

Two kinds of apparent loss are deliberately not issues:

• Normal tape rotation. The rolling tape overwriting its oldest events is the design working as intended (doc 04); no issue row.
• Policy non-capture. A helper's arguments not captured because capture policy says not to (doc 05) is a decision, not a loss; it appears as the per-value state not_captured, never as an issue.

A healthy run has zero rows in evidence_issues, so any row that appears signals a real problem.

The runtime already counts most of these events (every bounded buffer that discards keeps a counter, and loss markers are written on several paths built), but not every loss path is consistently persisted into per-run, queryable diagnostics yet. Closing that gap, so every material loss becomes a queryable issue row, is a v1 correctness gate v1.

Status and evidence are independent axes

A run's status answers how the program did. Its evidence answers how much of what Studio promised is present. Neither implies the other:

run3 failed (Eve's malformed email threw an unhandled error), but its evidence is complete: tape dump dump2, the retained calls, and the captured error value are present, so the failure can be debugged. runX succeeded, yet its numbers are suspect. Program success does not imply data trustworthiness.

The still-running run2 is a third case: its evidence is pending, not incomplete. Counters for a running run are explicitly "so far"; running data is never presented as final v1.

One simple surface, many precise reasons

Evidence health has several dimensions: call-structure completeness, presence of each promised value, integrity of stored records, processing completion, and intactness of everything policy retained. The table schemas carry each as its own run-level state column (doc 09 calls these the run's evidence axes); each axis is a rollup of finer-grained per-call or per-value states. Doc 05 listed the per-value states (available, not_captured, omitted, redacted, lost, truncated, corrupt, unsupported (the machinery could not represent the value), plus pending while a run is open), defined so that no two different situations collapse into one label v1.

A dashboard needs one label, not nine. The run's evidence state summarizes whether the promised evidence is ready and trustworthy: a small enum, on the order of ready / partial / pending / unavailable / untrusted, with the typed reasons one drill-down away. The exact enum, and the mapping from the fine-grained states onto it, is an open design decision open. The settled direction is a simple summary on top, typed reasons preserved underneath.

The query outcome: every answer grades itself

Run-level evidence state covers one run. Most questions span many runs, so each answer must grade itself. Every query ends with exactly one typed query outcome, delivered alongside the rows rather than as a fake final row: whether the query completed, whether the answer was complete, and, when values were involved, how many were attempted, available, and unavailable, by typed reason [v1, settled decision D13].

Example: of the retained ClassifyCustomer calls that succeeded this morning, how many returned Approve?

SELECT count(*) AS approvals
FROM retained_calls
WHERE definition_key = 'ClassifyCustomer'
  AND status = 'succeeded'
  AND "return" = 'Approve';
-- syntax illustrative; column spellings freeze with the v1 table schemas <span class="chip c-open" title="Decision not yet made">open</span>

The status filter is required: call6 is also a retained ClassifyCustomer call, but it failed, and a failed call has an error value and no return (doc 05), so a question about return values must scope itself to calls that have one. Over the healthy runs the query touches one call, call8, its return is available, and the outcome confirms completeness. Widening the window to include runX adds two more succeeded retained ClassifyCustomer calls, so the query touches three, and the overload that dropped 10,000 records also lost one of their promised return values.

The lost value does not become SQL NULL, which would conflate "the value was null" with "the evidence is missing" and return a count one short with no warning. It evaluates to a typed unknown: a marked non-answer carrying its reason, never NULL, never a silent non-match [v1, settled decision D12]. A captured BAML null is ordinary data; the two are never confused. The outcome reconciles the totals:

The count is a labeled lower bound instead of a silent undercount.

No outcome means no answer. If a result stream ends without its outcome (a dropped connection, a truncated response), the caller may not claim the query succeeded. Even a failure of the reporting channel is detectable, because its absence is the signal v1.

Doc 05 showed the same principle: llm_usage marks run1's token totals partial because Bo's failed call reported no usage, and absent tokens are not zero tokens. The query outcome applies that rule to every answer.

What "all promised evidence" means

A run's evidence being complete means everything capture policy promised for that run is present, not that every value of every call was recorded. Helpers' values are mostly not captured, by design (doc 05). A run whose helper arguments were never captured is complete: nothing promised is missing. A run whose promised LLM prompt was lost is incomplete, however well the program ran. Completeness is measured against capture policy's promise.

Error storms do not flood the table

Application failures are not evidence issues: a million provider errors are counted in the aggregates like any other calls. When recording pressure does cause loss, evidence_issues groups by scope and typed reason with a count: one row for 10,000 dropped records, never 10,000 rows v1.

Why self-reported failures are credible

Losses are counted at the bound that causes them: a buffer that overwrites knows how many records it overwrote at the moment it does so. The design rule is that bounded never means silent: every limit has a counter, a marker, or an explicit error v1. Most limits meet the rule today built, but a few internal counters can still saturate silently at extreme scale; marking those, like making every loss queryable, is part of the same v1 gate above. And the mandatory query outcome means a failure of the reporting channel surfaces as a missing outcome, not a plausible answer.

07: Which code was this? Relating runs to code

Key points

• A revision is a content hash of the whole compiled program; any behavior-affecting edit makes a new one. Every run records its revision.
• A function_id is meaningful only inside one revision. Cross-revision grouping uses the definition_key, which a rename changes on purpose.
• An unchanged local definition hash never proves unchanged behavior; the hash is deliberately not dependency-aware.
• Studio's code identity names the code a run came from. It is not version control: no diffs, no deploy tracking.

The question

In August, run run1 classified Ada, Bo, and Cy; Cy's classification took 6.2 seconds. In September, run run4 processed the same three customers and every classification finished in about a second. The cause could be the prompt edit, provider variance, or different code entirely.

Deciding requires knowing which program produced each run, and "the code" is ambiguous: the branch moved twice that week, a teammate deployed Tuesday, one server restarted with a stale build. Doc 02 defined the revision as "the exact compiled program". This doc describes how Studio names programs, functions, and source locations so that runs point at code without guesswork.

Revision: the exact program

A revision is a content hash over the whole compiled program: every source file, class, and prompt, plus the configuration and compiler inputs that affect behavior. The identity commits to every behavior-affecting input, so any edit produces a new revision: a rename, one changed prompt word, a new field on a class that some function consumes. built

Every run records at start the revision that produced it. In the example program, one server process on rev1 hosted run1, run2, and run3. Editing ClassifyCustomer's prompt text, and nothing else, produced rev2, which hosted run4. The revisions table holds one row per compiled program Studio has observed:

(These are teaching rows; doc 09 gives the real column list.)

A git commit, release tag, build number, or service name is a label attached to a revision, not the identity. Two deploys of the same bytes are one revision; one commit built with two compiler versions is two. built

Why the strict rule is affordable

A revision costs one hash and one small identity dictionary per compile, recorded once, not per call. The strictness is deliberate: a class field can silently change what a prompt renders, so the whole program is the only safe unit of "exactly this behavior". The identities below make the strictness practical.

The identity dictionary

With each revision the compiler emits, once, a small identity dictionary: every function's name, source location, capture flags, and the three identity fields defined below. Per-call records hold only small integers (doc 03: recording must cost nanoseconds); the dictionary turns integers back into names at read time. If the dictionary is missing, Studio shows fn#18 rather than a fabricated name. built

function_id: cheap at runtime, meaningless alone

A function_id is a dense integer the compiler assigns per function within one revision. Real functions start at 16; 0 is the "unknown" sentinel that degraded evidence folds into. built The runtime records this integer; integers are cheap enough to write millions of times.

The cost: a function_id means nothing without its revision_id. The functions rows for the example program (one row per function per revision):

ClassifyCustomer is id 18 in rev1 and 16 in rev2. Nothing is wrong; dense ids are assigned fresh at each compile. Joining across revisions on function_id silently compares different functions: cross-revision grouping uses definition_key, never function_id.

validate_email is compiled into the revision and has identity rows, but in the example accounting its invocations fold into the caller's work rather than being recorded as separate calls; that is why run1 counts 8 calls, not 11. The dictionary says what code exists; call counts say what the runtime records.

definition_key: the same function across many revisions

Most questions span revisions: a month of ClassifyCustomer behavior can cover fourteen revisions, one per prompt tweak. The definition_key is the stable identity of the logical function: ClassifyCustomer keeps dk4 through rev1 and rev2 though its id and hash both changed. built

One deliberate exception: a rename changes the definition_key on purpose. Renaming ClassifyCustomer to ScoreCustomer makes a new logical function; Studio does not guess intent. A hash match across a rename is a hint, not identity.

dk4 is a teaching placeholder. The real key is a readable string derived from the function's kind and fully qualified name, closer to function:user.ClassifyCustomer, which is why a rename changes it. built Doc 09's example keys look like function names because that is the key's real form, not shorthand. Its exact spelling, like every physical identifier form, is not frozen for v1 open.

The three identities in one picture:

local definition hash: whether one function changed

Between two revisions most functions did not change. The local definition hash identifies the ones that did: a hash (BLAKE3-256 today built) over one function's own compiled signature and bytecode. In the rows above, the prompt edit changed only ClassifyCustomer's hash (lh_9a → lh_f3); the rest carried identical hashes into rev2. That is the precise reading of "a new revision shipped, but only one function changed".

function ClassifyCustomer(c: Customer) -> Verdict {
  client "openai/gpt-5"
  prompt #"
    … {{ c.name }} …  ← renders customer data
  "#
}

The hash covers the function's own definition; referenced definitions contribute by name only, not their contents. It is deliberately not dependency-aware. Example: a hypothetical third revision adds a field to class Customer. That is a new revision under the strict rule, but ClassifyCustomer's local hash may be unchanged: its own text still says only "Customer". Its behavior may still differ, because the prompt renders customer data, which now has another field. An unchanged local hash never proves unchanged behavior. The revision remains the only exact statement of "this program"; the local hash is a lean change signal. V1 ships tests pinning exactly this: sensitivity to direct edits, and documented non-transitive behavior never presented as transitive proof. v1

Why the hash is not dependency-aware

A dependency-aware hash, where an equal hash means equal behavior, requires hashing each function's full dependency closure: every referenced type, template, and callee, cycle-safe, per definition. That is a real future design, explicitly not in v1 open. Until then Studio keeps the weak signal and does not overclaim.

Which identifiers travel, and which don't

These docs use friendly placeholders (run1, call8, context4, dk4); real identifiers are opaque strings whose exact physical form is a decision still to be frozen for v1 open.

Not every id is a UUID, by design: global names are hashes or opaque ids; identifiers written millions of times per second are small scoped integers. Today's run identifiers are random and do not sort by time; chronology comes from the recorded start timestamp. built

Call sites: where the call was made from

A call site is the static source expression that makes a call: in ProcessCustomer, ClassifyCustomer(c) and validate_email(c.email) are two call sites; in ProcessCustomers, spawn WriteAuditLog(batch_label) is a third. Call sites are static facts about the source, so validate_email(c.email) qualifies even though its invocations fold into the caller's accounting. Recording them enables navigation from a retained call in Studio to the source line that made it.

Status: the per-call record reserves a slot for a call-site reference and the identity dictionary has a call-site section, but the compiler writes no entries, so the call_sites table has no rows today. Populating it is committed target work, tied to retained-call source navigation; until the producer lands, nothing may rely on it. v1 (producer not built today)

Comparing revisions

The opening question in query terms: per revision, how often did ClassifyCustomer run, how often did it fail, and how long did it wait? Group the complete layer by revision, holding the logical function fixed with its definition_key:

-- Illustrative target SQL <span class="chip c-v1" title="Committed target with a delivery gate; designed, not built">v1</span>; exact column spellings are not frozen yet.
SELECT
  revision_id,
  sum(calls_started) AS calls,
  sum(calls_errored) AS failures,
  sum(await_ns)      AS time_waiting
FROM calling_contexts
WHERE definition_key = 'dk4'      -- ClassifyCustomer, in every revision
GROUP BY revision_id;

The rev1 count is 4, not 3, because run2 is still running on rev1 and running work is not excluded. Dee's classification is in flight at this snapshot, about three seconds into its wait, so the rev1 row includes its so-far counters: a fourth started call and its waiting time so far, on top of run1's three sealed calls and 8.18s. Whether a total includes still-open work is read from run states at the query snapshot, not a per-row mark (docs 03 and 06). run3 never ran ClassifyCustomer, so it adds nothing.

These totals are complete: every call in every run contributed, as doc 03 described. They do not by themselves prove the prompt edit caused the improvement; different runs saw different traffic on different days. Doc 10 builds the cross-revision comparison recipe on this query shape.

What this is not

Studio's code identity is an identity dictionary for observations, not a version-control system. It states which exact program produced a run, which logical function is which across revisions, and whether a function's own definition changed. It does not show diffs (git does), track deploys or releases (labels attached as dimensions), or claim two revisions behave the same. It answers one class of question: which code was this.

08: From your laptop to the cloud

Key points

• All evidence so far lives in one local directory, .baml/; local-only operation is a supported mode.
• Sealed evidence is immutable; safe retries, corrections as new facts, and rebuildable downstream tables follow from that.
• Upload is optional, never on the hot path, and reclaim gates on receipts: no receipt, no deletion.
• Hosted evidence is kept indefinitely; the only deletion path is explicit, verified erasure.

One directory

Every run so far (run1's eight calls, run3's unhandled ValidationError, the tape dumps dump1 and dump2, the captured values) was recorded without touching the network, into a directory called .baml/ next to the project. built

run1's and run3's evidence, simplified:

The whole of run1 (eight calls, four aggregate rows, one tape dump, a handful of values) is a few small files. Two properties of these files matter for the rest of this doc.

A sealed artifact is a file the runtime appends to while its run is alive and closes forever when the run ends. run2's file grows while run2 runs; once finished, it is only read, never modified. built

The content-addressed store (CAS) holds captured values: the args, returns, and errors from doc 05. Values are stored under identifiers computed from their bytes, so identical values are stored once. This is the deduplication doc 05 described. Ada's Customer record, captured in the root call's args and again in ClassifyCustomer's args, is stored once. built

The playground, the local browser UI for inspecting runs (served by baml playground), reads this directory and live in-process state directly. built No account, no upload, no cloud. Without upload configured this is the entire system, a supported way to run: capture, tape, values, queries, all offline.

Append, then seal

Evidence is append-then-seal, and sealed evidence is immutable: never edited, for any reason. Three properties follow.

Retries are harmless. A sealed file sent twice arrives byte-identical, so the receiver detects the duplicate by content and keeps one; no coordination protocol is needed to make retrying safe.

Corrections are new facts. If run1's evidence later proves wrong or incomplete, nothing rewrites run1's files; a new fact (an evidence_issues row, from doc 06) is recorded next to them. Doc 06's guarantee holds because originals cannot be quietly overwritten.

Everything downstream is rebuildable. Any summary, index, or table derived from sealed evidence can be discarded and rebuilt with identical results. The cloud design below depends on this.

A crash mid-run does not corrupt evidence. The partly written file keeps its intact prefix, a torn final record is ignored, and the run is later classified as crashed or partial from surrounding evidence. Nothing invents a success or a failure that did not happen. built

Leaving the laptop v1

Upload is optional, and the running application never waits on it. It has five steps:

1. Spool. Sealed evidence is split into upload units called chunks (each a byte range of exactly one sealed file) and copied into a small on-disk holding area beside the runtime, the spool. A separate small local ledger records what is still owed to the cloud.
2. Upload. Chunks are uploaded to object storage (S3: durable file storage in the cloud).
3. Receipt. The service verifies and records what it accepted, and answers with an upload receipt: durable proof that these exact bytes are now the cloud's responsibility.
4. Reclaim. Only receipted bytes may be reclaimed from local disk. No receipt, no deletion: the evidence stays on your machine.
5. Project. The service reads accepted evidence and builds the query tables from it. This rebuild-from-evidence step is called projection.

None of this is on the hot path: doc 03 established that the runtime does no filesystem or network work at call entry. Upload is a separate background component draining sealed files; a slow or absent network changes how full the spool gets, never how fast calls run.

Why reclaim waits for receipts

A successful upload call is not proof of delivery, because networks produce ambiguous outcomes: a timeout after the bytes arrived looks identical to a timeout before. The receipt is the one unambiguous fact, and local reclaim gates on it in order, so a later successful upload can never hide an earlier missing one. The result is the design's guarantee: no acknowledged evidence is ever silently lost. v1

Status: the spool, uploader, and receipt machinery are target v1 work and do not exist on this branch. v1 An older uploader (tracingv2) exists today built but speaks a different, legacy protocol and is explicitly not this path. The full ingest design, including what the service does between receipt and projection, is internal: CANONICAL/design/05-capture-and-ingest.md and CANONICAL/design/02-system-architecture.md.

Where things live

Once upload is configured, four places hold data, each with one job:

ClickHouse is the analytics warehouse that answers SQL. It is rebuildable in a precise sense: wipe it entirely and replaying the sealed evidence in S3 produces the same tables; an explicit release gate performs exactly this rebuild from empty and requires identical results. It never stores value bodies: args, returns, errors, and prompts stay in the value store and S3 and reach queries on demand, as doc 05 described. Keeping values out of the warehouse is settled (decision D8 in the internal register, CANONICAL/design/08-decisions.md).

PostgreSQL holds the mutable state: who owns what, what is mid-flight. Evidence never changes and lives in files and S3; immutability keeps this division clean.

Freshness of each copy

Locally, the aggregate rows fold on a 250 ms cadence while a run is live and are force-flushed and sealed when it ends. That number is an implementation default of the current build built, not a product promise. The playground also reads live in-process state, so local freshness is effectively immediate.

Hosted, 250 ms is not a cloud write cadence. Chunks close by age and size; the exact thresholds are deliberately unfrozen, to be chosen by benchmarks. As an order of magnitude, the queryable-in-seconds target below implies chunks leaving the machine every few seconds under load, but that is an expectation to validate, not a promise. open How quickly accepted evidence becomes queryable is likewise not frozen. The design carries a qualification target (accepted-to-queryable in seconds, p95 under 5), a gate to measure before release, not a measured claim. open

The above covers finished runs. Whether the hosted view must also show still-running runs in v1 (and if so, when an active run first becomes visible in hosted queries) is an unresolved decision. open If required, the sketch is short-lived incremental rows, discarded soon after the sealed final aggregate arrives, so history is always read from immutable evidence. open Batching trades freshness latency against ingest cost. Retained detail is a separate dial: a slower cadence delays evidence, it does not thin it. v1

Error storms

An LLM provider outage that fails a million calls in an hour does not turn capture into a firehose while the system is already under load.

Locally, every capture mechanism is bounded by construction; no throttle is bolted on top. Counting is folding: a million failures are increments to the same few calling_contexts rows, not a million new rows. The rolling tape is fixed memory, and dumps are triggered and rate-limited (an implementation default). built On the current branch one root-observed error fires one dump; the exact dedup contract across rethrows is the question doc 04 left open. open Value capture follows the explicit policy matrix from doc 05: under overload it sheds lower-priority bodies while counting the losses. The counters and loss markers exist today built, and making every such loss a queryable evidence_issues row is the v1 correctness gate doc 06 named. v1

The designed policy has no bimodal "cheap normally, expensive under incidents" mode, with one exception today: a single hard cap, the fixed memory that buffers structural events, still aborts the process when exhausted instead of shedding. Replacing that abort with the typed shed policy is committed v1 work. v1

If upload is configured, an outage grows the spool toward its budget. At that hard boundary the design prescribes a typed, predeclared choice: stop admitting new runs, reserve room to close the runs in flight, then apply one of three named behaviors: fail the run, abort the process, or continue with the run marked incomplete. Fail-the-run is the recommended default. Wiring that ladder is committed v1 work v1; which behavior is the default in each environment is an open policy decision. open

How the hosted service handles an entire fleet storming at once (admission control, backpressure, deduplication) belongs to the internal cloud doc and is not reviewed here. The shape of the answer: the service protects accepted evidence first, slows its own projection work next, then pauses new upload authorizations, and finally tells clients to retry later, so storm bytes wait in local spools, durable and bounded. v1 Details: CANONICAL/design/05-capture-and-ingest.md.

Data lifetime

Locally, disk is governed by budgets and reachability. Old run directories are pruned oldest-first under a size budget, and the value store is garbage-collected: a value survives while something retained points at it, and releasing a run releases its values. built The specific caps are implementation defaults, not policy. With upload configured, the receipt gate outranks the size budget: unreceipted bytes are never pruning candidates, so a stalled upload grows the spool toward its budget and lands in the error-storm ladder above; the design refuses new evidence before it deletes unshipped evidence. v1

Hosted, accepted evidence is immutable and kept indefinitely by default; routine maintenance is forbidden from evicting it. The only deletion path is explicit, authorized erasure: a verified workflow that denies access first, removes the data from every store, copy, and derived table, and reaches its terminal state only after per-store verification. Ordinary retention is not erasure, and erasure is not a best-effort delete. v1 (Settled as D11 in the decision register; optional customer-configured retention windows are a deferred policy decision. open)

With no upload configured there are no receipts to wait for; local budgets alone govern the directory. Local-only operation is a first-class mode, not a degraded one. built

09: Table schemas

Key points

• Eleven relations follow two data patterns: complete-layer tables count every call as aggregates, and retained-layer tables hold exact evidence selected by policy, not by traffic.
• The table schemas are logical and versioned. Physical storage can change freely because every hosted projection is rebuildable from sealed evidence.
• args, return, and error on retained_calls are virtual fields: ordinary SQL predicates apply to them, but value bodies load on demand and are never stored in the warehouse.
• One proposed table, cct_windows, was cut from v1: it grew with active locations × time buckets and had a mutable open bucket.

The user-facing table schemas

The user-facing table schemas are the documented, versioned set of relations that users and agents query. The internal design documents call this set the catalog ("catalog v1", "the catalog freeze"); that term survives as the engineering name, so those documents stay readable. "Public" means documented and stable, not open to the internet. A relation here is logical: a name, a grain, columns, and semantics that Studio promises to keep. How the data is physically stored (which ClickHouse table, file, or index) is private and free to change behind the table schemas, because every hosted projection is rebuildable from sealed evidence v1. Internal relation names carry version suffixes (runs_v1, cct_population_v1) so meanings cannot drift silently v1; this doc drops the suffix for readability.

A relation's grain is what one row stands for. A query that assumes the wrong grain counts the wrong thing.

The IDs below (run1, context3, call7, dump1) are readable placeholders for teaching. Real identifiers are opaque, and their physical form is a freeze decision open.

The table schemas in one picture

Summarized, the table schemas are one pattern applied five times:

The tables at a glance

Every table this doc defines, in the order it defines them. The reader-facing names are rename proposals for the table-schema freeze open; internal design docs use the engineering names.

Each section below states the internal name once, then uses the proposed name.

runs: the anchor

Purpose. The starting point of every investigation: find a run here.

Grain. One row per run, never per call.

Example rows (example program; process P, revision rev1):

run1 succeeded with total_errors = 1: Bo's classification failed, the fallback handled it, and the handled error stays visible. run2 shows so-far counters that are explicitly not final v1.

Answers: which recent runs had problems; whether a run is still open; whether its evidence can be trusted at a glance. Cannot answer: which function failed (that is calling_contexts); why evidence is missing (that is evidence_issues).

Growth. One small row per run; volume tracks run count only.

Lifecycle. The terminal fact is sealed and immutable. While a run is open, status and so-far counters come from live state, merged over the sealed facts at read time, never by rewriting a row v1. How that hosted live state is stored is an open storage decision open.

Physically. Sealed run facts live in local .baml/ artifacts and S3; hosted rows are a ClickHouse projection rebuilt from them (doc 08). Two columns differ: projection_state and retention_state describe workflow and come from PostgreSQL control state, and an open run's so-far numbers come from live state, not ClickHouse.

Schema (adapted from the internal proposal):

Status. Local run evidence exists today built; this public relation freezes at table-schema v1 v1. The single rolled-up evidence-state enum from doc 06 is still open.

calling_contexts: the complete layer's core

Purpose. Complete counts and timing for every call ever made, without a row per call.

Grain. One row per distinct calling context within one run (internal: cct_population; the structure is the calling-context tree, "CCT").

Example rows (run run1: 8 calls fold into 4 rows):

Ada, Bo, and Cy fold into one row per context; a million customers would too. context4 is almost all await time: most LLM latency is time spent waiting, not computing.

Answers: which functions fail most, over all calls; where run time went, split into self and await; complete error rates. Cannot answer: which exact call failed; what the arguments were; how threads interleaved (thread identity is deliberately absent from this grain; doc 03).

Growth. Unique call paths, not invocations. Highly dynamic or recursive paths can still grow it; path count is a release gate v1.

Lifecycle. While a run is active, folded counters arrive as small immutable increments: deltas. At run end one final immutable row per context is written (the delta-then-final pattern). The invariants are settled: sealed rows are never mutated, and a query uses the final row or the deltas, never both v1. Whether the hosted active side stores deltas append-only or uses another overlay is an open storage decision open.

Physically. Folded locally into sealed .baml/ artifacts, uploaded to S3, projected into ClickHouse facts (doc 08); hosted active-view storage is the open decision above.

Schema:

Status. The folding engine is on the branch today built; the public relation is table-schema-freeze work v1. Extremely large runs can lose exact folded counts to a counter-width defect; fixing or explicitly marking overflow is a v1 gate v1.

retained_calls: the retained layer's core

Purpose. The individual calls Studio kept, for exact inspection.

Grain. One row per individually retained call.

Example rows:

Cy's slow classify (call8) was kept because it crossed the slow-call threshold (an implementation default, not policy; doc 04). Bo's handled failure call6 is here because its error body was kept under the value rules for LLM functions (docs 04/05); a handled error fires no dump, so its tape list is empty. Eve's failure kept two calls: the frame that threw and the root that observed it. One propagating error produces one dump and no row per rethrow.

Answers: which exact calls can be opened; what a call received, returned, or raised (via the virtual fields below); when it ran and on which logical thread. Cannot answer: total traffic. Counting these rows does not measure failure rates: retained counts are lower bounds selected by policy, and complete counts live in calling_contexts v1.

Growth. Bounded by retention policy, not by traffic.

Lifecycle. Terminal rows are immutable; a still-running retained call's row is served from live state and merged at read time. List columns are assembled from separate append-only call-to-dump records, so discovering another containing dump never rewrites the row v1.

Physically. Resident columns are ClickHouse facts rebuilt from sealed .baml//S3 evidence; the value bodies behind the virtual fields below stay in the local value store and S3, never in the warehouse (doc 08).

Schema:

Status. Retention mechanisms exist locally built; the relation freezes with the table schemas v1. Whether process_id/engine_id columns are needed at all is unresolved open.

Virtual value fields: args, return, error

A resident field is data physically present in the analytical store: small, typed, filterable (everything in the schema tables above). A virtual field exists only in the SQL surface: when a statement needs it, the query engine follows private evidence handles and loads the value from local evidence or object storage on demand. args, return, and error on retained_calls are virtual; they are never warehouse columns, because value bodies never live there (doc 08).

Ordinary SQL still works against them v1:

-- Exact whole-value equality against a caller-supplied BAML value.
WHERE args = :expected_args

-- A predicate over a nested field of the argument object.
WHERE args['c']['plan'] = 'pro'

The argument object is name-keyed by declared parameter names (args['c'] is ProcessCustomer's c). Positional syntax, if supported at all, normalizes to those declared names rather than becoming a second stored shape open.

Three rules apply. First, = means whole-value semantic equality: never partial-object matching, byte equality, or storage-ID equality v1. Second, resident filters run before any value is loaded; values load in bounded, deduplicated batches; and a LIMIT never applies until value predicates have actually been evaluated v1. Third, an unavailable value (redacted, lost, not captured) evaluates to a typed unknown that is reconciled in the query outcome; it is never a silent SQL NULL or a quiet non-match v1. A captured null is ordinary data. The exact args root shape, subscript spelling and index base, and the behavior of an available value with an absent path are freeze items open.

tape_dumps: the ledger of saved tape

Purpose. Records what exact event evidence exists, why it was kept, and whether it is complete.

Grain. One row per preserved slice of the rolling tape (internal: exact_windows). The events themselves stay in the sealed dump; this table is the small searchable ledger over them.

Example rows:

dump2 is the tape-beats-a-traceback example from doc 04: its slice contains the audit thread's start and cancellation events next to Eve's failure.

Answers: whether exact evidence exists around an incident; what triggered it; whether the slice is complete or truncated. Cannot answer: value bodies (a dump is structural events only); anything outside the preserved slices; and it may cover only part of a long run.

Growth. One row per preserved slice; not per call or clock tick.

Lifecycle. The row is inserted after the dump seals and is immutable; later corruption or loss appears as issue facts composed into evidence_state, never as an edit v1.

Physically. Ledger rows are ClickHouse facts; the event bytes they describe stay in sealed dump artifacts in .baml/ and S3, reachable only through evidence_id.

Schema:

Status. Dumps and triggers run locally today built; the ledger relation is table-schema work v1. Trigger policy details remain open where doc 04 marked them open.

evidence_issues: the health table

Purpose. Makes missing evidence a queryable fact instead of a silent gap (doc 06).

Grain. One immutable grouped summary: one source scope × kind × reason, with a count.

Example row. The three example runs are healthy: zero rows, which is itself the answer. The teaching row is doc 06's hypothetical overloaded run runX:

Answers: whether a run's evidence is complete enough to trust; which pipeline stage lost what, when, and how much. Cannot answer: application errors. Eve's ValidationError never appears here, because your program failing is not Studio failing to watch it.

Growth. Only scopes that had an issue; the count column exists so a storm of identical losses stays one row.

Lifecycle. A row is emitted only when its source range seals; counts never increment in place. A run binding discovered later is attached by a separate append-only linking record, not a rewrite v1.

Physically. ClickHouse facts projected from sealed diagnostic ranges in .baml/ and S3, like everything else in these table schemas.

Schema:

Status. The loss counters and markers exist locally but are not yet consistently persisted into run diagnostics; closing that is required v1 work v1. The grouped-row contract freezes with the table schemas v1.

functions, call_sites, revisions: code identity

Purpose. The identity dictionary that connects observations to code (doc 07). Deliberately not a version-control system.

Grain. One compiled revision; one function within a revision; one static call expression within a revision.

Example rows (functions, revision rev1; real function IDs start at 16: lower values are reserved for the runtime built):

The dk placeholders are doc 07's: a definition_key is deliberately not the name, because renames change it on purpose. revisions has one row for rev1; call_sites has none yet (see status). The cross-revision worked example (the edited prompt) is in doc 07.

Answers: which exact program produced a run; whether a function's own compiled definition changed; where the function is in source. Cannot answer: whether behavior is truly equal across revisions (an unchanged local_definition_hash says nothing about callees; doc 07's caveat), and nothing about source history beyond what runs observed.

Growth. Compile-time program structure, never invocation volume.

Lifecycle. Insert once per revision, then immutable. Reprocessing the same revision must reproduce identical rows; a differing row for the same identity is an integrity conflict, not an update v1.

Physically. The dictionaries seal with the revision's artifacts in .baml/, upload to S3, and project into ClickHouse like run evidence.

Schema (key columns):

Status. Revision dictionaries and function rows exist today built. The call_sites producer is not built: the dictionary section exists but is empty, so retained_calls.call_site_id is not navigable until producer and dictionary land together open.

llm_usage: provisional

Purpose. Token and LLM-error accounting without opening a single prompt (internal: llm_population).

Grain. One row per run × calling context × provider × model.

Example row (run run1: three classify calls, one provider failure):

token_state = partial because Bo's failed call reported no usage, and absence of tokens must never read as zero tokens. run3 has no row at all: ClassifyCustomer never ran.

Answers: which models spent tokens, where in the call tree, and how much; provider failures versus parse failures. Cannot answer: which exact prompt was expensive (follow node_id into retained_calls and its virtual fields); dollar cost. Prices change, so cost is a query-time join to a price relation, not a stored fact.

Growth. Unique combinations, not LLM invocations.

Lifecycle. Same delta-then-final pattern as calling_contexts v1. Physically: same path too: local fold, sealed artifacts, S3, ClickHouse projection.

Schema: run_id, node_id (joins calling_contexts), provider (string; kept only if provider/model stays the public grouping open), model (string), llm_calls (count), token_state (enum), input_tokens / output_tokens (count?), provider_errors / parse_errors (count).

Status. Provisional open: the LLM instrumentation is being reworked, and this relation is expected to change with it. Aggregate-only growth is the part that is settled.

thread_edges: conditional

Purpose. Fan-out accounting: what each context spawned, and how that work ended (internal: spawn_edges).

Grain. One row per unique spawning-context × spawned-function relationship in one run.

Example rows:

Ten thousand identical workers would still be one row with spawned = 10,000.

Answers: total fan-out; spawned work that failed or was cancelled; how many exact instances are inspectable (retained_instances vs instances_dropped). Cannot answer: the timing or identity of a specific spawned thread; that is retained_threads.

Growth. Unique edges. Lifecycle: delta-then-final, like every complete-layer table v1. Physically: same path as calling_contexts: sealed local evidence, S3, ClickHouse projection.

Schema: run_id, edge_id, parent_node_id, child_function_id, spawned / completed / errored / cancelled (count), running_ns (duration; kept only if its accounting can be made exact open), awaiting_ns (duration), retained_instances, instances_dropped (count; so a selective instance table is never mistaken for complete history).

Status. Spawn aggregation runs locally today built; both thread relations enter the table schemas only if concurrency diagnosis is a first-shipped priority open.

retained_threads: conditional

Purpose. The retained layer's mirror for spawns: specific spawned-thread instances you can inspect (internal: spawn_instances).

Grain. One row per individually retained spawned thread.

Example rows:

run3's cancelled audit thread is here because cancellation made it exceptional. Cancellation preserved no tape (doc 04); the thread appears in dump2 only because Eve's error dump happened to cover it.

Answers: inspect a particular spawned thread; link to its exact parent and child calls when those were retained. Cannot answer: total spawn counts; that is thread_edges, same rule as everywhere.

Growth. Policy-retained instances. Lifecycle: terminal rows immutable; an open instance's row is served from live state at read time v1. Physically: same path as retained_calls: resident ClickHouse facts, evidence in .baml/ and S3.

Schema: run_id, spawn_id, edge_id (joins thread_edges), thread_id, parent_call_id / child_call_id (id?; set when those calls were retained), status (enum), started_at / ended_at, tape_dump_ids, evidence_ids, evidence_state (enum; a row being present does not make its evidence readable).

Status. Bounded instance retention exists locally built (first-N plus exceptional instances: implementation defaults, not policy); inclusion in the table schemas is conditional with thread_edges open.

Why there are eleven tables

One anchor, three complete-layer tables, three retained-layer tables, three code-identity tables, one health table: eleven relations, two data patterns. Every data table is either the complete layer (cheap totals over everything) or the retained layer (exact evidence for the selected few); the rest is identity and health. thread_edges/retained_threads repeats the calling_contexts/retained_calls pattern for spawns.

Each table was justified against real queries; one was not. The internal proposal cct_windows would have stored time-bucketed aggregate deltas for "when did this spike?" charts. Its growth was active call-tree locations × elapsed time buckets; at the current local fold cadence one active location would mint four rows per second (an implementation-default figure, not policy); and its open bucket was mutable. Complete totals already live in calling_contexts, live charts are served from the playground's direct view of running processes (doc 08), and incident detail lives in tape_dumps. It was cut from v1, revivable only if a measured historical-chart workflow justifies a coarse, retention-limited derived view v1. The same discipline applies elsewhere: the thread tables are conditional on concurrency diagnosis being a first-shipped priority, llm_usage is provisional pending the LLM rework, and the duration histogram stays only if percentile questions are one too.

Tree queries in plain SQL

Most common questions need no tree walk: failure rates, time by function, and cross-revision comparisons group by definition_key. A tree query anchors to one run_id; parent_node_id plus depth reconstruct the tree within that run's contexts. Measured project corpora put the 99th percentile under a few thousand contexts per run, so per-run reconstruction is bounded and small. Whether a physical provider also keeps flattened ancestor indexes is physical-design freedom behind the table schemas, decided by benchmarks, not by this document open.

Two limits remain. A path-anchored question across many runs ("ClassifyCustomer specifically under ProcessCustomer, fleet-wide, last month") has no resident cross-run path column: either ask the function-anchored version with definition_key, or reconstruct per run; a resident path hash or ancestor list is one of the physical options open. And the delta-then-final rule is enforced by the provider, never by the query author: no query in this set needs a latest-row dedup idiom (the LIMIT 1 BY pattern that is the classically slow part of mutable-aggregate designs).

Where the table schemas end

Everything below the table schemas (which ClickHouse tables exist, how PostgreSQL coordinates uploads, how projections are batched) is internal design. The analytical side (ClickHouse tables, provider caches, projection batching) is rebuildable from sealed evidence, so it can change without breaking a saved query. The workflow truth PostgreSQL owns (acceptance, ownership, retention and deletion state) is not rebuilt from evidence; it reaches the table schemas only through the workflow columns on runs. Studio's stability promise is exactly what this doc described: versioned logical relations with fixed grains, availability semantics, and outcomes v1. Physical DDL, ordering keys, partitions, codecs, and provider overlays are all unfrozen open, owned by the internal design, and unconstrained by this set. The internal cloud document owns the rest.

Terms defined here: user-facing table schemas; logical vs physical; grain; resident field; virtual field.

10: How do I build …?

Key points

• Panels (rates, totals, trends) query the complete layer; drill-downs read values from the retained layer behind id-narrowed filters.
• Answers have three shapes: complete (every call counted), selected (policy-retained calls only), bounded (an exact tape slice).
• Traffic and error totals come from the complete layer only; row counts in retained_calls are a policy-selected lower bound.
• A result stream with no query outcome is not a successful answer.

Docs 00–09 defined the concepts; this doc gives recipes: the question in plain English, the SQL, the queried table's grain, why the query is fast, and the answer's shape. A complete answer comes from the complete layer, where every call is counted; a selected answer from the retained layer, which keeps only interesting calls; a bounded answer from a tape dump, an exact slice around one moment.

All recipes walk the same ladder:

Panels query the top of the ladder; investigations descend it.

1. Which recent runs had problems?

English: list recent runs that failed outright, contained errors, or have incomplete evidence.

SELECT run_id, started_at, status, entrypoint,
       total_calls, total_errors, structure_state, value_state
FROM runs
WHERE started_at >= :from_time
  AND (
    total_errors > 0
    OR status IN ('failed', 'panicked', 'abandoned')
    OR structure_state IN ('incomplete', 'lost')
    OR value_state IN ('partial', 'lost')
  )
ORDER BY started_at DESC
LIMIT 100;

One row is one run; the table grows with runs, not calls, so a run list is always cheap. Toy results: run3 failed (Eve's malformed email reached the root unhandled); run1 succeeded but shows total_errors = 1 from Bo's handled failure; run2 is still running, its evidence states pending, which is normal for an open run (doc 06). The evidence arms name the bad states (incomplete, lost, partial) rather than everything-but-complete, which would flag every open run; drop the problem filter to see run2's so-far counters.

The answer is complete: every run has a row. The filter mixes two independent axes by design: execution status and evidence state (doc 06). The one-word evidence summary for dashboards is still open open; until it freezes, filter on the typed state columns.

2. Which functions fail most, and where did the time go?

English: across every finished call in a time range, rank functions by failures.

SELECT definition_key,
       SUM(calls_started) AS calls_started,
       SUM(calls_errored) AS failures,
       1.0 * SUM(calls_errored) / NULLIF(
           SUM(calls_succeeded + calls_errored + calls_cancelled + calls_exited),
           0) AS failure_rate
FROM calling_contexts
WHERE run_id IN (
    SELECT run_id FROM runs
    WHERE started_at >= :from_time AND started_at < :to_time)
  AND definition_key IS NOT NULL
GROUP BY definition_key
HAVING SUM(calls_errored) > 0
ORDER BY failures DESC
LIMIT 50;

One row of calling_contexts is one calling context in one run; a million identical calls were already folded into it by the runtime (doc 03). definition_key is null for synthetic internal functions; the filter excludes them rather than pooling them into one meaningless group. The denominator sums only the four finished counters, so unfinished calls, including run2's open ones, never dilute a failure rate. ClassifyCustomer shows Bo's failure from run1 even though every ProcessCustomer there succeeded: handled errors stay visible in the complete layer. The answer is complete.

Time spent in one run uses the same table:

SELECT definition_key,
       SUM(calls_started) AS calls,
       SUM(inclusive_ns) AS inclusive_ns,
       SUM(self_ns) AS self_ns,
       SUM(await_ns) AS await_ns
FROM calling_contexts
WHERE run_id = :run_id
GROUP BY definition_key
ORDER BY inclusive_ns DESC;

For run1, ClassifyCustomer shows 8.20 s inclusive, 0.02 s self, 8.18 s await: almost all model wait, not compute, the dominant pattern for LLM programs. Both queries group by resident columns only: no tree recursion, no value reads, no per-call rows (doc 09 explains why tree questions stay cheap).

Do not count rows in retained_calls for the failure panel: that counts retained failures, a policy-selected subset and a lower bound. Traffic and error totals always come from the complete layer. This is the most common mistake when using these tables.

3. Drill into one failure

English: find the retained calls that explain a failed run, then read their captured values.

SELECT call_id, definition_key, status, duration_ns, retention_reasons
FROM retained_calls
WHERE run_id = :run_id
  AND status IN ('failed', 'panicked')
ORDER BY started_at;

For run3 this returns call3 (ProcessCustomer) and the root call1, both with retention reason error: a propagating error is retained where observed, not once per rethrow (the exact retention point is not yet frozen open, doc 04). Load values on demand:

SELECT args, error
FROM retained_calls
WHERE run_id = :run_id AND call_id = :call_id;

args and error are virtual fields (doc 09): the first query touched only resident columns; this id-narrowed one reads value bodies from the content-addressed store. For call3, error holds the ValidationError but args does not hold Eve's Customer: call3 is a plain helper, and helper args are off by policy (doc 05), so args evaluates to a typed unknown with reason not_captured (never a silent SQL NULL), reconciled by the query outcome (doc 06). Eve's record is readable in the root call1's captured args. Neither call here has a return role; a failed call has none (doc 05).

The answer is selected: the calls policy kept, for an unhandled failure the ones that explain it. Nested value predicates like WHERE args['c']['plan'] = 'pro' use the same surface; the argument object is keyed by declared parameter names (doc 09). Root shape and subscript spelling freeze at table-schema v1 open.

4. What exact evidence exists around this incident?

English: list the preserved tape slices for a run, then the retained calls inside one of them.

SELECT dump_id, trigger, started_at, ended_at, event_count, evidence_state
FROM tape_dumps
WHERE run_id = :run_id
ORDER BY started_at;

For run3: one row, dump2, about 40 structural events preserved because the root observed an unhandled error. For run1: dump1, preserved around the slow call8, about 130 events over 6.2 s of surrounding activity. One row is one preserved slice, not one call and not one time bucket; the events stay in the dump's sealed artifact (doc 08). To list the inspectable calls inside a dump:

SELECT call_id, definition_key, status, started_at
FROM retained_calls
WHERE run_id = :run_id
  AND list_contains(tape_dump_ids, :dump_id)
ORDER BY started_at;

The list-membership spelling is freeze work open. dump2 also holds the audit thread's start and cancellation adjacent to Eve's failure: concurrent context a traceback cannot show (doc 04). The answer is bounded: a dump is an exact record of a slice of the run; evidence_state and event_count say how much of one before you open it.

5. Did the new revision make it worse?

English: compare the same logical function's behavior across revisions.

SELECT revision_id, definition_key,
       SUM(calls_started) AS calls,
       SUM(calls_errored) AS failures,
       SUM(inclusive_ns) / NULLIF(SUM(calls_started), 0) AS mean_inclusive_ns
FROM calling_contexts
WHERE definition_key = :definition_key
GROUP BY revision_id, definition_key
ORDER BY revision_id;

Group across revisions by definition_key, never function_id: the dense per-revision id means nothing outside its revision (doc 07). Keep revision_id in the result; collapsing revisions averages two different programs into one meaningless number. For ClassifyCustomer, rev1 shows three started calls and Bo's failure; rev2 (run run4, after the prompt edit) shows three successes and a smaller mean. Counters are started, not finished: open runs contribute so-far numbers, so once run2 reaches Dee's classification it joins the rev1 row and dilutes the mean with so-far time (doc 07); at this snapshot it has not. To check whether the function itself changed or only its surroundings:

SELECT revision_id, definition_key, local_definition_hash
FROM functions
WHERE definition_key = :definition_key
ORDER BY revision_id;

Between rev1 and rev2 only ClassifyCustomer's local definition hash differs: the prompt edit. This is a local-change signal only: equal hashes do not prove equal behavior; a callee or shared type may have changed (doc 07). Both queries are complete and cheap: identity columns are duplicated into the hot aggregate table, so no joins. The comparison is an investigation signal, not statistical proof of a regression.

6. Which model spend is growing, and what are the expensive inputs?

English: total token usage and provider errors by model, over a time range.

SELECT provider, model,
       SUM(llm_calls) AS calls,
       SUM(input_tokens) AS input_tokens,
       SUM(output_tokens) AS output_tokens,
       SUM(provider_errors) AS provider_errors
FROM llm_usage
WHERE token_state = 'available'
  AND run_id IN (SELECT run_id FROM runs WHERE started_at >= :from_time)
GROUP BY provider, model
ORDER BY input_tokens + output_tokens DESC;

One row of llm_usage is one run × calling context × provider × model: aggregate arithmetic, no prompt bodies touched. Cost is your token price times these sums. The token_state filter excludes the example run1 row, whose state is partial because Bo's failed call reported no usage: absence of tokens is not zero tokens. A cost dashboard therefore needs a coverage panel:

SELECT token_state, SUM(llm_calls) AS calls
FROM llm_usage
WHERE run_id IN (SELECT run_id FROM runs WHERE started_at >= :from_time)
GROUP BY token_state;

Spend is complete over what was measured; the coverage query says how much was measured. For the exact expensive inputs, find the worst (run, context) keys (the by-model rows deliberately carry neither), dropping the token_state filter because this hunt wants the partial rows too:

SELECT run_id, node_id,
       SUM(input_tokens + output_tokens) AS tokens
FROM llm_usage
WHERE run_id IN (SELECT run_id FROM runs WHERE started_at >= :from_time)
GROUP BY run_id, node_id
ORDER BY tokens DESC
LIMIT 20;

The top (and only) example row is the one the spend panel excluded: run1's ClassifyCustomer context (context4), ranked by the tokens it did report. Take its (run, context) key and descend:

SELECT call_id, duration_ns, args
FROM retained_calls
WHERE run_id = :run_id AND node_id = :node_id
ORDER BY duration_ns DESC
LIMIT 20;

This surfaces call8, the slow classification retained by the latency trigger; its args holds Cy's Customer. The drill-down is selected: retained LLM calls are examples, not a census. Two open edges: llm_usage is provisional pending the in-flight LLM instrumentation changes open, and per-call token counts are not resident columns, so "sort exact calls by tokens" is not yet expressible open.

7. Is my spawned work healthy?

English: which spawned functions produced failed or cancelled work?

SELECT f.definition_key,
       f.fqn AS child_function,
       SUM(te.spawned) AS spawned,
       SUM(te.errored) AS failed,
       SUM(te.cancelled) AS cancelled
FROM thread_edges AS te
JOIN runs AS r ON r.run_id = te.run_id
JOIN functions AS f
  ON f.revision_id = r.revision_id AND f.function_id = te.child_function_id
WHERE r.started_at >= :from_time
GROUP BY f.definition_key, f.fqn
HAVING SUM(te.errored + te.cancelled) > 0
ORDER BY failed DESC, cancelled DESC;

One row of thread_edges is one parent-context-to-spawned-function relationship per run: the complete layer's view of fan-out. The time range spans revisions, so grouping follows recipe 5's rule: definition_key through the functions join, never the per-revision child_function_id, which would split one logical function across revisions. On the example data, WriteAuditLog shows one cancellation: run3's audit thread, cancelled when the run ended early. Cancellation is deliberately not a default tape trigger (doc 04; the trigger matrix is still freeze work open), so nothing exact was preserved because of it; the complete layer still counted it, which is all this panel needs. retained_threads is the retained-layer counterpart for inspecting individual spawned work: the same two-layer pattern as calls. The answer is complete. Both thread tables are conditional on concurrency diagnosis being a first-shipped priority open.

8. Can I trust this dashboard?

English: before believing any panel, check what the observability system itself failed to capture.

SELECT kind, reason, SUM(count) AS affected_records
FROM evidence_issues
WHERE run_id = :run_id
GROUP BY kind, reason
ORDER BY affected_records DESC;

A healthy run returns zero rows; run1, run2, and run3 all do. The hypothetical overloaded run from doc 06 returns one grouped row, structure / records_dropped, affected_records = 10,000: upstream counts may be undercounts, which changes how to read every panel for that run.

Trust is also per-query. Every query on this page ends with one query outcome: a typed record stating whether the answer is complete, which fixed view of the data it ran against (later arrivals are invisible), and how many value reads were attempted, available, and unavailable, by reason. Illustratively:

resultState: complete
snapshot:    tableSchemaVersion v1, projectedThrough …
valueEvaluations: attempted 2, available 2, unavailable 0

Field names and wire shape freeze with table-schema v1 open; the rule does not: a result stream with no outcome may not be treated as a successful answer, even for a simple count. A budget can expire mid-stream, a value read can fail, evidence can still be pending; rows alone cannot report this.

9. The dashboard checklist

A dashboard built from these table schemas follows four rules:

• Panels (rates, totals, trends) query the complete layer (runs, calling_contexts, llm_usage, thread_edges): cheap, never sampled.
• Drill-downs descend to the retained layer (retained_calls, tape_dumps) and touch values only after id-narrowing filters.
• Every page carries an evidence panel: the evidence_issues summary for whatever is in view.
• Every number surfaces its query outcome; a truncated answer is never presented as complete. Check resultState on every panel, valueEvaluations when a drill-down touched virtual value fields, and projectedThrough on any time-ranged panel (illustrative names; they freeze with table-schema v1 open).

Table coverage: recipes 1–8 exercise runs, calling_contexts, retained_calls, tape_dumps, functions, llm_usage, thread_edges, and evidence_issues. Three tables appear in no ranking recipe and serve navigation instead: revisions resolves a revision_id to one compiled program; call_sites will jump a retained call to the source expression that made it (its producer is not built; the dictionary is empty today open); retained_threads is recipe 7's drill-down counterpart, conditional with the other thread tables open.

The agent loop

The recipes are written for people, but the surface is designed to be used by agents as well v1. An agent's loop: discover the table schemas with baml query --schema (relations, grains, column semantics); query the complete layer first; descend to retained evidence only for examples; check the query outcome before asserting anything; cite claims by run_id and call_id so a human can load the same values and verify. The vocabulary an agent needs is the ten-or-so terms this series defines; doc 11 fits the entire skill into one short file.

11: The agent skill

Key points

• The terminology's acceptance test: an agent must be able to query Studio from one file far smaller than the design corpus behind it.
• The draft skill below is that file: ten terms, the table schemas in one table, seven query rules, the outcome contract, three worked queries, and the claims an agent must never make.
• The skill adds no new claims; every line traces to one earlier doc.

Everything between the rulers below is a draft skill: self-contained instructions an agent loads before querying Studio. The skill doubles as an acceptance test for the vocabulary: if it is usable on its own, the terminology is sufficient; if it requires the other ten docs, the names need revision before the table-schema freeze.

The skill targets the v1 table schemas. The SQL semantics are settled, but the baml query command that executes them is not on the branch today v1; what exists now is the capture machinery beneath the tables built. Table names are the reader-facing proposals from doc 09; the shipped table schemas add a version suffix (runs_v1), and the rename set is a table-schema-freeze decision open.

SKILL (draft): querying BAML Studio

Purpose. You are answering questions about a BAML program's behavior through Studio's SQL surface. This file is everything you need. Discover exact column lists with baml query --schema; never guess a column name.

The ten terms.

1. Run: one top-level BAML execution. A server process hosts many runs; runs never nest.
2. Call: one function invocation inside a run. Calls form a tree under the run's entry call.
3. Calling context: one path of parent functions down to one function. Every invocation along the same path folds into one aggregate row, so a million identical calls cost one row.
4. Complete layer: the tables where every call is counted: totals, never samples. Use it for rates, traffic, and time.
5. Retained call: an individual call kept by policy (error, latency, membership in a dump, promotion, explicit capture). The retained layer holds selected evidence, never totals.
6. Tape dump: one preserved slice of the rolling tape of structural events (call and thread starts and ends, suspensions), kept around an incident or on manual request; the row's trigger column says why. Events only, no value bodies; may cover only part of a run.
7. Virtual value fields: args, return, error on retained_calls. Not stored columns: values load on demand when a statement needs them. Ordinary SQL applies: WHERE args['c']['plan'] = 'pro'; the argument root shape and subscript spelling freeze at table-schema v1 open.
8. Typed unknown: what a predicate over an unavailable value (redacted, lost, never captured) evaluates to. Never a silent NULL or quiet non-match; the query outcome reconciles it. A captured null is data.
9. Evidence issue: a grouped record of data Studio itself failed to keep. Zero rows means healthy. Independent of whether the program failed.
10. definition_key vs revision: a revision_id names one exact compiled program; a definition_key names the same logical function across revisions.

The table schemas. A relation's grain is what one row stands for.

The seven rules.

1. Use runs to find a run.
2. Use calling_contexts for complete all-call totals.
3. Use retained_calls only for selected exact evidence.
4. Filter on small resident columns before touching args, return, or error: values hydrate on demand and are the expensive part.
5. Group across revisions by definition_key, never by function_id: the dense per-revision id is meaningless outside its revision.
6. Treat local_definition_hash as a local-change signal only; equal hashes do not prove equal behavior, because a callee may have changed.
7. Check per-role value states, evidence_issues, and the query outcome before claiming completeness.

The outcome contract. Every SQL stream ends with exactly one typed out-of-band query outcome: whether the result is complete, the fixed snapshot it ran against (later arrivals are invisible to it), and how many value reads were attempted, available, and unavailable, by reason. Rows streamed before a late failure are incomplete. A stream with no outcome may never be reported as a successful answer. Field names freeze with table-schema v1 open; the rule does not.

Three worked examples. English first, always.

Which functions fail most, across every call ever made? (complete)

SELECT definition_key, SUM(calls_errored) AS failures
FROM calling_contexts
GROUP BY definition_key
HAVING SUM(calls_errored) > 0
ORDER BY failures DESC
LIMIT 20;

Handled errors stay visible here: a call can fail inside a run that succeeded.

What exactly went wrong in this failed run? (selected)

SELECT call_id, definition_key, retention_reasons, error
FROM retained_calls
WHERE run_id = :run_id AND status IN ('failed', 'panicked')
ORDER BY started_at;

error is a virtual value field, hydrated for these few rows only, after the resident filters ran. One propagating error is retained where it was observed, not once per rethrow; the exact retention point for a propagating error is not yet frozen open.

Can I trust the data for this run? (health)

SELECT kind, reason, SUM(count) AS affected_records
FROM evidence_issues
WHERE run_id = :run_id
GROUP BY kind, reason;

Zero rows means Studio kept everything capture policy promised. Then read the query outcome before reporting anything.

Never claim:

• completeness without the query outcome: no outcome, no answer;
• traffic or failure totals from retained_calls, retained_threads, or tape_dumps: retained counts are policy-selected lower bounds;
• that a typed unknown is NULL, zero, or a non-match: report its reason and mark the answer incomplete;
• that missing token counts are zero tokens: respect token_state;
• that equal local_definition_hash values mean equal behavior;
• that a tape dump covers a whole run: it is a bounded slice; check event_count and its evidence state first;
• program health from evidence_issues, or evidence health from run status: the two are independent axes.

The skill is about a hundred lines. Run against this set's example program: the first example returns three rows, ClassifyCustomer with Bo's handled failure from run1 plus the two frames of Eve's one unhandled error from run3. The second, pointed at run3, returns call3 and the root call1, both retained for reason error, and hydrates Eve's ValidationError once. The third returns zero rows for all three runs; all three are healthy.

Every line traces to one doc: the terms to 02 through 07, the table schemas to 09, the rules and examples to 10, the outcome contract to 06. The skill adds nothing new, by design. The test is whether a newcomer, human or agent, can query Studio from this file alone; on this draft, they can.

Terms defined here: none, deliberately. This doc compresses what the set already taught.