Routing rules now available on AI Gateway

Vercel AI Gateway now supports routing rules.

Routing rules are firewall-style rules that control which models your team can use, applied at the gateway level instead of in your application code.

When a model goes down or gets retired, you usually have to ship a code change to move off it. With routing rules, you push one rule and every request reroutes instantly. There are two types:

Type	What it does	Use it to
Rewrite	Serves a request for one model using another	Keep traffic flowing when a model is unavailable, migrate off a retired model, standardize on one model, or route an expensive model to a cheaper one
Deny	Blocks requests for a model	Keep your team off models you haven’t approved

Rules apply to every request made with your team's AI Gateway credentials. You manage them with the Vercel CLI.

Rewrite​

Create a rewrite with a source and a destination model. The Gateway swaps in the destination transparently, so your application keeps requesting the source model:

Deny​

Create a deny rule to block a model. Requests for it return a 403:


Rules only change which model serves a request. Everything else you’ve configured still applies to the destination model, including:

• 
  Request-level: BYOK, model fallbacks, sorting, the only filter, and provider options.
  
  
• 
  Team-level: Zero Data Retention and the provider allowlist.
  

Routing rules are in beta. For more information, read the routing rules docs.

Read more

Continue reading...