Skip to content
Signpost
Rolling outAI

LLM gateway

One endpoint for every model we use, with per-team quotas, cost attribution, audit logging and failover between providers.

Lifecycle

Next: generally available on 19 October 2026, in 2 weeks. Each audience arrives on its own schedule, below.

  1. Proposed12 months ago
  2. In development9 months ago
  3. Rolling out (current stage)4 months ago
  4. Generally availableplanned, in 2 weeks

By audience

Currently reaching Product teams with a reviewed use case.

  • The AI platform team's own services

    Phase 1 — pilot

    Generally available
    Rolling out
    12 Jan 2026
    Generally available
    25 May 2026
  • Product teams with a reviewed use case

    Phase 2 — reviewed use cases

    Rolling out
    Rolling out
    15 Jun 2026
  • All engineering

    Phase 3 — everyone

    Proposed
    Generally available
    19 Oct 2026

Detail

Why this exists

Model access grew organically: several teams hold their own provider keys, spend is invisible until the invoice arrives, and there is no shared record of what was sent to a third party.

Who this affects

Every team calling a model API. The change is small — a base URL and a key — but it is mandatory, because the old keys are being revoked.

What we need from you

Move before 2 November. If you have a use case the gateway does not cover yet, raise it in #ai-platform now rather than after the cutover.

Updates

Breakingtakes effect

Direct provider API keys stop working on 2 November

From 2 November the provider keys held by individual teams are revoked. Calls made directly to a provider will fail with a 401; calls through the gateway are unaffected.

Switching over is a base URL change and a different key. The quickstart has the exact diff for each SDK we support.

Action required

Route your model traffic through the gateway

Teams using models in production should move now rather than in October — the gateway gives you retry handling and a cost breakdown you do not currently have.

Info

Gateway is serving its first internal traffic