Overview

The LLM Gateway is a transparent proxy that sits between your applications and any LLM provider (OpenAI, Anthropic, Azure OpenAI, Bedrock, on-prem). Every request is classified, safety-checked, and routed before it leaves your network. No code changes are required in your application — point it at the gateway endpoint instead of the provider.

Data-Classification Routing

Requests are classified into one of four tiers. The routing policy is evaluated per-request and per-workspace, allowing different policies for different teams.

bash

Configure routing policy for a workspace

1curl -X PUT https://sovereign.yourcompany.com/api/v1/workspaces/WS_ID/llm-routing \
2  -H "Authorization: Bearer $TOKEN" \
3  -H "Content-Type: application/json" \
4  -d '{
5    "policies": [
6      { "classification": "PII",  "route": "on-prem",   "mask_before_send": true },
7      { "classification": "PHI",  "route": "on-prem",   "mask_before_send": true },
8      { "classification": "CONFIDENTIAL", "route": "azure-openai-eu", "mask_before_send": false },
9      { "classification": "INTERNAL",     "route": "openai",          "mask_before_send": false }
10    ]
11  }'

Classification	Default Route	PII/PHI Masking
PUBLIC	Any configured cloud provider	Off
INTERNAL	Preferred cloud provider	Off
CONFIDENTIAL	Approved cloud provider only	Partial (names, emails)
PII / PHI	On-premises endpoint only	Full — all 22 entity types

Prompt Safety & Jailbreak Blocking

Every prompt is evaluated by a local safety classifier before routing. Jailbreak attempts, prompt injection patterns, and policy-violating content are blocked and logged. The gateway returns HTTP 422 with a X-Sovereign-Block-Reason header — your application receives a safe error response, never the raw block reason.

bash

Test a prompt through the gateway

1# Route requests through the gateway — same API shape as OpenAI
2curl -X POST https://sovereign.yourcompany.com/api/v1/gateway/chat \
3  -H "Authorization: Bearer $TOKEN" \
4  -H "Content-Type: application/json" \
5  -d '{
6    "model": "gpt-4o",
7    "messages": [{ "role": "user", "content": "Summarise this contract" }],
8    "workspace_id": "WS_ID"
9  }'
10
11# Jailbreak attempt → blocked
12# HTTP 422 | X-Sovereign-Block-Reason: jailbreak-pattern-detected

Per-Team Cost Chargeback

Token usage is metered per workspace and per model. Monthly budget alerts fire at 80% and 100% of the configured cap. Chargeback reports export as CSV for finance teams.

bash

Set a monthly token budget

1curl -X PUT https://sovereign.yourcompany.com/api/v1/workspaces/WS_ID/budget \
2  -H "Authorization: Bearer $TOKEN" \
3  -H "Content-Type: application/json" \
4  -d '{
5    "monthly_usd_cap": 2000,
6    "alert_at_percent": 80,
7    "hard_cutoff": false
8  }'
9
10# Export chargeback report
11curl "https://sovereign.yourcompany.com/api/v1/llm/usage/export?workspace_id=WS_ID&month=2026-04" \
12  -H "Authorization: Bearer $TOKEN" > chargeback-april-2026.csv

Route Simulator

The Route Simulator (Gateway → Simulate in the UI) lets you paste a prompt and see exactly which classification tier it would receive, which endpoint it would route to, whether PII masking would fire, and which tokens would be redacted — without sending the request to any LLM.