BAML Studio architecture package

Status: Proposal, 2026-09-09. This package describes a hosted architecture for BAML Studio, its control-plane and data-plane contract, the storage decisions still to pick, query execution, and the HTTP API. It supersedes the 2026-09-01 scale-out proposal and the 2026-09-04 acceptance decision. It incorporates two experiments: a query proof of concept over 15 million synthetic calls, and an S3 storage experiment over 2,328 execution roots. The hosted system is not built.

The design in one paragraph

The profiler records every BAML call inside the customer's application and writes closed segment files and values into a local store. The publisher, a background task inside that application, ships those closed files and values as compressed objects straight to the customer's S3 bucket, with a presigned URL per object from the customer-side gateway. S3 notifies a queue when a meta segment or a value pack arrives. A projector projects an execution when its RootEnded record arrives: it reads the execution's data segments, function table and values by key, checks them, sends allowlisted public facts to Boundary's ClickHouse and appends private values and their indexes to Lance datasets that stay in the customer's bucket. Studio, the CLI and agents run plain SQL against one logical catalog. A query coordinator in the customer's cloud runs the public part in ClickHouse and the private part over Lance. It streams rows with one typed outcome. Long analyses run as background jobs. Boundary keeps no per-upload record: S3 is the evidence, the queue carries the work, and everything else can be rebuilt from the bucket.

Documents

• Hosted architecture: topology, data placement, components, ingest and query flows, correctness rules, failures, deployment and acceptance tests. Includes a step-by-step ingest walkthrough.
• Publisher guide: everything the runtime team needs to build the publisher: spool, deltas, the delta file, the two gateway endpoints, the S3 PUT, failures, limits and conformance.
• System contract: behavior, ownership, states, failure handling and implementation boundaries between the customer data plane and Boundary.
• Storage decisions: the remaining storage choices, each with a recommendation, alternatives, evidence and a place to record the pick.
• Query execution: interactive examples showing how the coordinator splits work between ClickHouse and Lance, built from the proof-of-concept queries.
• OpenAPI reference: endpoints, authentication, parameters, request and response schemas, and the queue message contract.
• Deferred decisions: receipts, commit acknowledgments and the commit ledger are retired from this proposal. This page records what they did, what replaces them and when to reconsider.

Reading order

Start with the hosted architecture. Runtime engineers then read the publisher guide. Use the system contract for behavior and ownership. Use the storage decisions page to pick the remaining storage details. Use the OpenAPI reference for HTTP details. Read the deferred decisions page only to understand what the earlier design had and why it was removed.