Aurabase Logo
aurabasedocs
docs›Services›AI Gateway

AI Gateway

Unified proxy OpenAI, Anthropic, Google Gemini. Semantic cache, per-tenant budget, token observability. A single SDK for your LLM workloads.

8 min read·Level intermediate·Revised on Apr 15, 2026
This English text was generated automatically from the French original and has not been reviewed yet.
#
Overview

One endpoint, N providers

The AI Gateway is a single HTTP/SDK surface that proxies major LLM providers. The same code runs against OpenAI, Anthropic, Mistral or Vertex AI — you only change provider.

The gateway adds three layers on top: semantic cache (prompt embeddings, hit rate ~30% on average), per-tenant budget (quota $ per user), observability (tokens, cost, latency, provider).

#
Providers

Five GA supported

ProviderChat/completionEmbeddings templates

OpenAIgpt-5.5, gpt-5.4, gpt-5.4-mini/nano``text-embedding-3-large / -small

Anthropicclaude-fable-5, opus-4-8, sonnet-5, haiku-4-5``—

Google Geminigemini-3.8-flash, 3.7-flash, 3.6-flash, 3.1-pro``gemini-embedding-2 / text-embedding-004

#
Semantic cache

~30% of prompts already known

For each incoming request, the gateway embedding the question (internal lightweight model, <10ms) and searches in a similar vector in a TTL cache. If the cosine distance exceeds your threshold (default 0.96), it returns the hidden response without calling the provider.

configtypescript
await aura.ai.chat({
  provider: 'anthropic', model: 'claude-opus-4-5',
  messages: [...],
  cache: {
    enabled: true,
    threshold: 0.96,   // min similarity
    ttl: '24h',
  },
})
Astuce

The cache is scoped per project. Two different users in the same project can benefit from a hidden response — add cache.scope: 'user' if you want to isolate by user.

#
Examples

Chat, embeddings, RAG

functions/chat.tstypescript
const stream = await aura.ai.chat({
  provider: 'anthropic',
  model: 'claude-opus-4-5',
  stream: true,
  messages: [{ role: 'user', content: prompt }],
})

for await (const chunk of stream) {
  process.stdout.write(chunk.delta)
}
#
Budget & observability

Quotas per user, live metrics

Each call is logged with in/out tokens, cost calculated to the hundredth of a cent, provider, model, latency, cache hit. Set a monthly budget per user to avoid surprises.

Attention

Roadmap — not available. The aura commands shown below are planned and not delivered yet: the CLI does not have them today.

clibash
# Budget €10/month per user on Claude Opus
aura ai budget set \
  --user-scope \
  --provider anthropic \
  --model claude-opus-4-5 \
  --limit 10_euros_per_month

# See real-time consumption
aura ai usage --by user --top 10
Attention

Exceeded budgets return HTTP 402 Payment Required with a reset_atfield. Your app can handle this gracefully (downgrade to a cheaper model, prompt the user to upgrade).

Last updated · Apr 15, 2026