AI Gateway now supports service tiering. Service tiers let you optimize for latency, throughput, and cost per request to match your use case. Pick a faster tier for interactive workloads (less queueing, higher token throughput), or a lower cost tier for background jobs that can tolerate more latency.
At launch, service tiering is available for OpenAI and Gemini models.
Service tiers work across every AI Gateway API format: AI SDK, Chat Completions API, Anthropic Messages API, OpenAI Responses API, and OpenResponses API. AI Gateway adjusts billing based on the tier each request used.
If a service tier is not specified, requests run on the default tier.
Set
The applied tier is returned in the provider metadata so you can confirm which tier served the request. This is useful when a priority request falls back to default capacity, or when comparing observed latency across tiers.
Service tier is serviced on a best-effort basis: if a tier can't be applied, the request runs on the default tier at the default rate, and only an invalid service tier value fails the request.
If a model can be served by multiple providers and you only want a service tier on some of them or to configure different service tiers across each, set the tier on the provider's own namespace instead.
AI Gateway applies per-tier rates automatically based on the tier each request actually used. If a
More details about providers and tiering are in the service tiers reference.
Read more
Continue reading...
At launch, service tiering is available for OpenAI and Gemini models.
Service tiers work across every AI Gateway API format: AI SDK, Chat Completions API, Anthropic Messages API, OpenAI Responses API, and OpenResponses API. AI Gateway adjusts billing based on the tier each request used.
Tiers
default: Standard processing
priority: Faster processing at increased cost
flex: Lower cost with potentially higher latency
If a service tier is not specified, requests run on the default tier.
Basic usage
Set
serviceTier under providerOptions.gateway. The same option works across all models and providers that support service tiers, so you can swap the model without restructuring provider-specific options:The applied tier is returned in the provider metadata so you can confirm which tier served the request. This is useful when a priority request falls back to default capacity, or when comparing observed latency across tiers.
Service tier is serviced on a best-effort basis: if a tier can't be applied, the request runs on the default tier at the default rate, and only an invalid service tier value fails the request.
Per-provider control
If a model can be served by multiple providers and you only want a service tier on some of them or to configure different service tiers across each, set the tier on the provider's own namespace instead.
Pricing
AI Gateway applies per-tier rates automatically based on the tier each request actually used. If a
priority request gets downgraded to default capacity, billing reflects the default rate, not the priority rate.Tier | default | priority | flex |
Cost relative to default | Baseline | ~1.8-2x default pricing | ~0.5x default pricing |
More details about providers and tiering are in the service tiers reference.
Read more
Continue reading...