AI Gateway Routing Configurations
Gateway routing configurations are JSON policy objects that control how AI Gateway routes, retries, caches, and load-balances LLM requests.
| Where Can I Use This? | What Do I Need? |
|---|
- Prisma AIRS AI Gateway (Americas region)
|
- AI Gateway activated
- At least one LLM integration configured
|
A routing configuration (config) is a saved JSON object that tells AI Gateway how to
handle a request beyond simple forwarding. Configs are created in Strata Cloud Manager
under Policies & Profiles → Configuration and are referenced in API requests by a
pc- prefixed identifier. Once created, a config is synced to the gateway
data plane every 30 seconds.
Configs support the following capabilities:
AI Gateway configuration capabilities
| Capability | Description |
|---|
| Load balancing | Distributes requests across multiple LLM targets using weighted routing.
Useful for spreading traffic across providers or model versions. |
| Fallback | Tries alternative targets in priority order if the primary target returns
an error. Supported error conditions include HTTP 429, 500, 502, 503, and 504. |
| Retry | Automatically retries failed requests up to 3 times before returning an
error to the caller. |
| Simple cache | Returns cached responses for semantically identical requests, reducing
latency and provider costs for repeated queries. |
| Semantic cache | Returns cached responses for semantically similar requests using embedding
comparison, not exact-match. |
| Conditional routing | Routes requests to different targets based on request properties such as
model name, user identity, or metadata values. |
| Guardrail hooks | Runs guardrails before the request is forwarded to the provider
(before_request_hooks) or after the response is received
(after_request_hooks). |
Configs can be versioned. Each time you save a config, Strata Cloud Manager stores the
previous version so you can restore it if needed. You can also attach a config to a gateway
API key so every request authenticated with that key automatically uses the config, without
requiring the application to reference it in the request body. To prevent applications from
overriding the attached config with inline JSON, enable Block Inline Configurations
in AI Gateway settings.
