# Inference providers

A provider runs a model and offers its inference capacity through Canopy. You choose which model to serve, what to charge, and when to accept new work. Applications using Canopy can reach your model without integrating with your service separately.

You can connect an existing inference server. You remain responsible for its hardware, availability, and model behavior.

## How providing works

1. Register your model and the endpoint where it runs.
2. Publish your prices through a provider agent.
3. When a matching request arrives, Canopy asks your agent for a quote.
4. Your agent checks your pricing and admission policy, then quotes or declines.
5. If Canopy selects your quote, it sends the request to your inference endpoint and returns your response to the application.

A quote is a promise to serve the request at the offered prices if selected before expiry. Declining new work is fine. An accepted quote is binding; the award is not another opportunity to decline.

You can also offer discounted cached input. That promises to retain reusable prompt content for an agreed time. Follow-up requests can arrive at the original prices without another quote, so account for them before accepting more work.

## What you need

* An approved Canopy provider account.
* A running model with an OpenAI Chat Completions, OpenAI Responses, or Anthropic Messages endpoint.
* An endpoint that the Canopy inference API can reach.
* A provider agent that stays connected to Canopy.

The agent and model server do different jobs. The agent exchanges prices, quotes, and lifecycle notifications over an outbound WebSocket. Canopy sends inference requests directly to your registered HTTP endpoint. The WebSocket does not tunnel requests, and quotes contain token estimates and capabilities, not prompts or raw engine metrics.

Use HTTPS for a remote inference endpoint and WSS for the agent connection. Keep provider tokens and engine credentials private. Canopy blocks private and loopback inference endpoints by default.

> The current bearer-token protocol is not launch-ready for an internet-facing marketplace. TLS and secret handling are required, but stronger transport protections, including message signing and replay protection, remain outstanding.

## Choose an integration

### Use the TypeScript SDK

The SDK runs on Bun. It collects llama.cpp or vLLM metrics, calls your pricing policy, and handles quotes, reconnects, and cache commitments. Start here unless you need your own protocol implementation.

[Get started with the SDK](/providers/getting-started)

### Implement the WebSocket protocol

Use your own language, engine integration, or agent. You must implement quote tracking, execution and cache lifecycle handling, heartbeats, and reconnect restoration.

[Read the WebSocket protocol](/providers/protocol)

To understand how Canopy compares offers, see [How routing works](/routing).