# WebSocket protocol Implement this protocol to connect a provider without the SDK. Version `1` carries prices, binding quotes, and execution and cache lifecycle notifications. It does not carry inference request bodies or raw engine metrics. You still need an approved provider account, active model mappings, a provider token, and a reachable inference endpoint. Follow [Register your model](/providers/getting-started#register-your-model); the SDK installation steps are not required. Runtime schemas are exported by `@canopy/provider/protocol` if you want validation without the SDK client. This page describes the wire contract independently of TypeScript. ## Connect Open a WebSocket upgrade request to the inference API: ```http GET /providers/ws HTTP/1.1 Authorization: Bearer canopy_provider_live_replace_with_your_token Upgrade: websocket Connection: Upgrade ``` Your WebSocket library supplies the remaining handshake headers. Use `wss://your-canopy-host/providers/ws` for a remote connection. Use a server-side client that can set the authorization header, and never expose the token in browser code or logs. The optional query parameter `cleanCache` accepts `true` or `false`. Omission means no reset. Use `?cleanCache=true` only after physical cache loss; it invalidates the provider's cache accounting before activating the connection. | HTTP status | Meaning | | ----------- | --------------------------------------------------- | | `400` | Invalid `cleanCache` value | | `401` | Missing or invalid token, or account not authorized | | `426` | Request did not ask for a WebSocket upgrade | | `500` | Upgrade or internal failure | | `503` | Server shutting down | The server sends `session.welcome`, then `session.state`. Restore the state before processing subsequent routing messages, then publish prices. The server restores its state before making the connection eligible for routing; there is no client restoration acknowledgement message. ## Envelope Each message is a JSON object: ```json { "v": 1, "id": "msg_quote-1", "type": "rfq.quote", "replyTo": "msg_rfq-1", "ts": "2026-08-31T01:00:00.200Z", "payload": { "requestId": "rfq-1" } } ``` | Field | Contract | | --------- | ----------------------------------------------------------------------------------------------------- | | `v` | Required integer, currently `1` | | `id` | Required nonempty message ID, at most 64 characters | | `type` | Required nonempty message type, at most 64 characters | | `replyTo` | Optional reply correlation ID, at most 64 characters; inbound `null` is allowed | | `ts` | Server UTC timestamp; optional on provider messages, at most 40 characters; inbound `null` is allowed | | `payload` | Message-specific object; required by the price and quote handlers | Generate a distinct message ID for each message. Server acknowledgements and errors use `replyTo` to identify the provider message. RFQ matching uses `payload.requestId` and the authenticated provider connection, not `replyTo`. Domain IDs, including RFQ, execution, commitment, and session IDs, are nonempty strings of at most 128 characters. Canonical model IDs in price updates allow letters, digits, `.`, `_`, `/`, and `-`. Domain timestamps such as deadlines and expiries use UTC ISO strings ending in `Z`, with seconds and optional one to three fractional digits. ## Message types | Direction | Type | Purpose | | ------------------ | -------------------- | --------------------------------------------------------------- | | Provider to server | `price.update` | Publish standing rates | | Provider to server | `rfq.quote` | Offer binding terms or decline | | Provider to server | `ping` | Request an application-level pong acknowledgement | | Server to provider | `session.welcome` | Session identity, models, heartbeat interval, protocol versions | | Server to provider | `session.state` | Restored commitments and active executions | | Server to provider | `rfq.request` | Request a binding quote | | Server to provider | `quote.award` | Notify an execution award before HTTP dispatch | | Server to provider | `quote.release` | Release an unawarded offer | | Server to provider | `execution.release` | Finish an execution | | Server to provider | `commitment.update` | Maintain cache terms and expiry | | Server to provider | `commitment.release` | Release a cache commitment | | Server to provider | `ack` | Confirm handling of a provider message | | Server to provider | `error` | Report a rejected message | The following examples show payloads unless explicitly described as full messages. ## `session.welcome` ```json { "userId": "provider-1", "sessionId": "session-1", "heartbeatIntervalMs": 30000, "protocolVersions": [1], "models": [{ "canonicalModelId": "openai/gpt-oss-20b", "status": "active" }] } ``` Read the heartbeat interval from the message rather than assuming the example value. Each model has a `canonicalModelId` and an `active` or `inactive` status. Use this list as the connection's model configuration. ## `session.state` ```json { "commitments": [], "executions": [] } ``` `commitments` contains [commitment objects](#commitmentupdate). `executions` contains [award objects](#quoteaward) for active work. Restore both before handling new messages. Maintained commitment expiries take precedence over provisional commitment values embedded in awards. ## `price.update` Send a full message like: ```json { "v": 1, "id": "msg_price-1", "type": "price.update", "payload": { "updates": [ { "canonicalModelId": "openai/gpt-oss-20b", "inputPerMTok": "0.35", "outputPerMTok": "0.61", "currency": "USD" } ] } } ``` `updates` accepts 1 to 100 entries. Each requires an active, configured canonical model and both rates. `currency` is optional and can only be `"USD"`. Duplicate model entries in a batch are invalid. The server acknowledges with `{ "applied": 1 }`, where `applied` is the number of updates. Standing rates describe the offer; routing selects from valid RFQ quotes. This message does not publish cache terms. ## `rfq.request` ```json { "requestId": "rfq-1", "canonicalModelId": "openai/gpt-oss-20b", "deadline": "2026-08-31T01:00:01.100Z", "estimate": { "inputTokens": 2400, "reusablePrefixTokens": 1800, "expectedOutputTokens": 1024, "maximumOutputTokens": 2048 }, "request": { "streaming": true, "hasTools": false, "hasStructuredOutput": false } } ``` All shown fields are required except `estimate.maximumOutputTokens`. Estimates are finite JSON numbers. The optional top-level `endpoint` is `chatCompletions`, `responses`, or `anthropicMessages`. The server omits it for Chat Completions. Quote only for a protocol and capabilities your configured endpoint supports. The RFQ contains no prompt or request body. Respond before `deadline` on the same authenticated connection that received it. ## `rfq.quote` ```json { "requestId": "rfq-1", "quote": { "inputPerMTok": "0.35", "cachedInputPerMTok": "0.175", "outputPerMTok": "0.61", "currency": "USD", "expiresAt": "2026-08-31T01:00:01.000Z", "cache": { "ttlSeconds": 300, "minCacheableTokens": 1024 } } } ``` Omit `quote` to decline: ```json { "requestId": "rfq-1" } ``` Only one reply is accepted per RFQ and authenticated provider, and only from the connection that received it. The RFQ's `requestId` also identifies the pending quote, scoped to that provider. The server acknowledges a resolved reply with `{ "accepted": true }`, including a decline. This means it handled the reply, not that the quote won. A closed, duplicate, late, or mismatched RFQ receives `UNKNOWN_RFQ`. ### Quote terms | Field | Required | Contract | | -------------------------- | ----------------------- | ------------------------------- | | `inputPerMTok` | Yes | Fresh input price | | `outputPerMTok` | Yes | Output price | | `cachedInputPerMTok` | No | Cached input price | | `currency` | No | Only `"USD"` | | `expiresAt` | No | UTC quote expiry | | `cache.ttlSeconds` | When `cache` is present | Integer from 1 to 86,400 | | `cache.minCacheableTokens` | No | Integer from 1 to 2,147,483,647 | Prices are decimal USD strings per million tokens, matching `^\d{1,6}(\.\d{1,6})?$`. Do not send numeric prices or exponent notation. Schema-valid cache terms are not a guarantee of cache routing eligibility; see [How routing works](/routing). A quote commits the provider to serving the request at those terms if selected before expiry. Track outstanding offers in your admission policy. Expire them locally in case a release is lost. The SDK uses five seconds after the RFQ deadline, bounded by any earlier explicit `expiresAt`, for its pending quote lifetime. ## `quote.release` ```json { "requestId": "rfq-1", "reason": "lost" } ``` `reason` is `lost`, `expired`, or `cancelled`. Remove the unawarded offer even if an award notification was lost. Canopy releases losing or invalid offers, replies that miss the RFQ deadline, and offers in cancelled auctions. A winning award replaces its pending quote with an active execution. ## `quote.award` ```json { "executionId": "execution-1", "requestId": "rfq-1", "quoteId": "rfq-1", "canonicalModelId": "openai/gpt-oss-20b", "terms": { "inputPerMTok": "0.35", "outputPerMTok": "0.61" }, "expiresAt": "2026-08-31T01:15:00.500Z", "cached": false } ``` | Field | Meaning | | ------------------ | --------------------------------------------------------------- | | `executionId` | Unique execution attempt, used for release and HTTP correlation | | `requestId` | Consumer inference request ID | | `quoteId` | Original RFQ ID for a new auction; absent for cached follow-ups | | `canonicalModelId` | Model to execute | | `terms` | Accepted quote terms | | `expiresAt` | Hard execution deadline, fifteen minutes after issue | | `cached` | Whether this request uses an existing cache commitment | | `commitment` | Optional commitment object with ID, model, terms, and expiry | An award is a notification, not a request for acceptance. Record it once per execution ID. Duplicates must not increase the active count or repeat callbacks. Canopy records the execution, sends the award, and immediately dispatches the HTTP request to your registered inference endpoint. It replaces the client's preset model with your upstream model name. It does not wait for an award acknowledgement or make another WebSocket liveness check before dispatch. The HTTP request carries `x-canopy-execution-id` for correlation. For a new auction, `x-canopy-request-id` matches the award's `quoteId`. A failed award send lets the auction try another valid quote. A failed HTTP connection ends the selected execution. Disconnects, expired quotes, or failed dispatch can cause selection of another quote before an upstream request is sent. Requests already sent upstream are not automatically retried. ## `execution.release` ```json { "executionId": "execution-1", "reason": "completed" } ``` `reason` is `completed`, `cancelled`, `failed`, or `expired`. Remove the execution from active accounting. Stream completion, upstream rejection, connection failure, consumer cancellation, and the hard deadline all end executions. Enforce the supplied deadline locally. It covers the complete HTTP request and response stream, not the time to answer an RFQ. Expiry bounds accounting but does not prove that physical engine work has stopped. Releasing an execution does not release a maintained cache commitment. ## `commitment.update` ```json { "commitmentId": "commitment-1", "canonicalModelId": "openai/gpt-oss-20b", "terms": { "inputPerMTok": "0.35", "cachedInputPerMTok": "0.175", "outputPerMTok": "0.61", "cache": { "ttlSeconds": 300, "minCacheableTokens": 1024 } }, "expiresAt": "2026-08-31T01:05:10.000Z" } ``` All four top-level fields are required. This object also appears in awards and restored session state. An initial award reserves a possible cache commitment until execution expiry plus the cache TTL. After successful cache accounting, `commitment.update` supplies the maintained terms and actual expiry. Each successful cached request refreshes that expiry. Canopy stores qualifying prefixes and their accounting. It does not infer physical cache-hit token counts merely because a request used a cached route. The provider must retain the physical cache and capacity needed to honor the promise. A cached follow-up skips the RFQ and receives an award with `cached: true`, the same commitment, and the original terms. Follow-ups can overlap without a per-commitment queue. Account for this potential work when quoting new requests. Count a commitment as idle only while it is unexpired and has no active executions. A cache promise does not reserve a dedicated execution slot. ## `commitment.release` ```json { "commitmentId": "commitment-1", "reason": "expired" } ``` `reason` is `expired` or `invalidated`. Remove the commitment. The server sweeps expiry every second and sends a release when no execution is using it. Also enforce supplied deadlines locally. Failed initial executions release provisional commitments. Provider cache invalidation releases maintained commitments. ## Heartbeats Respond to WebSocket protocol-level ping frames with pong frames. Most server-side WebSocket libraries do this automatically. The server uses `heartbeatIntervalMs` from the welcome message and terminates connections after two unanswered pings when the next heartbeat check runs. An optional application message `{ "v": 1, "id": "msg_ping-1", "type": "ping", "payload": {} }` receives an `ack` with `{ "pong": true }`. This acknowledgement does not reset the server's protocol-level missed-pong counter. Do not use it as a substitute for WebSocket pong frames. ## Errors and limits An error payload has `code`, a human-readable `message`, and optional `details`. Its envelope includes `replyTo` when the server can identify the rejected message. | Code | Meaning | | --------------------- | --------------------------------------------------- | | `MALFORMED_MESSAGE` | Invalid JSON or envelope | | `UNSUPPORTED_VERSION` | `v` is not supported | | `UNSUPPORTED_TYPE` | Message type has no provider handler | | `VALIDATION_FAILED` | Invalid payload or duplicate model in a price batch | | `UNKNOWN_MODEL` | Price update names an unconfigured model | | `MODEL_INACTIVE` | Price update names an inactive model | | `UNKNOWN_RFQ` | No open RFQ matches this reply and connection | | `RATE_LIMITED` | Inbound message queue is full | | `INTERNAL` | Server failed to handle the operation | Messages have a 64 KiB payload limit. The default inbound queue holds 64 messages. Queue overflow returns `RATE_LIMITED` and closes with `1013`. Three malformed JSON or envelope messages in 60 seconds close with `1008`. State restoration failure closes with `1011`. Shutdown uses `1001`. Provider suspension or token rotation closes an existing connection with `1008` when detected. Fix authorization failures instead of retrying indefinitely with an invalid token. ## Reconnect and cache loss Reconnect with backoff after transport failures. Restore the new `session.state` before applying subsequent messages. Ordinary reconnects preserve maintained cache commitments. If the engine lost its physical cache, request `cleanCache=true`; do not reset caches on every reconnect. In-flight work remains bounded by its execution deadline. Server restart restores maintained commitments from the database, but transient awards are not durable and cannot be resumed as dispatch authority. The current inference API keeps live WebSocket and award state in one process. Running multiple routing processes requires shared coordination that this protocol does not provide. The bearer-token protocol still needs stronger transport protections before use in an internet-facing marketplace. Use TLS, rotate credentials, and redact secrets, but do not treat those steps as message signing or replay protection.