polygate logopolygate

One function.
Any LLM provider.

A tiny, dependency-light client that gives 26 providers — and any endpoint you host yourself — one consistent request and response shape.

GitHub
from polygate import chat
 
response = chat(
    provider="anthropic",  # or "openai", "gemini", "moonshot"
    model="claude-sonnet-4-6",
    messages=[{"role": "user", "content": "Say hi in one word."}],
)
 
print(response.content)   # "Hi!"
print(response.usage)     # Usage(prompt_tokens=..., ...)

Swap providers by changing one string

Every adapter translates the same message list into whatever shape the provider actually expects. API keys come from the argument or fall back to the environment.

OpenAI

GPT models
"openai""gpt"

OPENAI_API_KEY

Anthropic

Claude models
"anthropic""claude"

ANTHROPIC_API_KEY

Google Gemini

Gemini models
"gemini""google"

GEMINI_API_KEY

Mistral

Mistral models
"mistral"

MISTRAL_API_KEY

Groq

LPU inference
"groq"

GROQ_API_KEY

Together AI

Open-weight models
"together""togetherai"

TOGETHER_API_KEY

Fireworks

Fast open models
"fireworks"

FIREWORKS_API_KEY

Perplexity

Sonar, web-grounded
"perplexity""sonar""pplx"

PERPLEXITY_API_KEY

xAI

Grok models
"xai""grok"

XAI_API_KEY

Cerebras

Wafer-scale inference
"cerebras"

CEREBRAS_API_KEY

DeepSeek

DeepSeek models
"deepseek"

DEEPSEEK_API_KEY

Moonshot

Kimi models
"moonshot""kimi"

MOONSHOT_API_KEY

Z.AI

GLM models
"zai""z.ai""glm""zhipu"

ZAI_API_KEY

Amazon Bedrock

Every model AWS hosts, via Converse
"bedrock""aws"

AWS_ACCESS_KEY_ID

Azure OpenAI

Your deployments
"azure"

AZURE_OPENAI_API_KEY

Google Vertex AI

Gemini and partner models
"vertex"

GOOGLE_ACCESS_TOKEN

Databricks

Model Serving endpoints
"databricks"

DATABRICKS_TOKEN

Cloudflare

Workers AI, on the edge
"cloudflare""workers-ai"

CLOUDFLARE_API_TOKEN

OpenRouter

Hundreds of models, one key
"openrouter"

OPENROUTER_API_KEY

Vercel AI Gateway

Routing across every vendor
"vercel""ai-gateway"

AI_GATEWAY_API_KEY

Baseten

Hosted open models
"baseten"

BASETEN_API_KEY

DeepInfra

Open models per token
"deepinfra"

DEEPINFRA_API_KEY

SambaNova

Open models on RDU silicon
"sambanova"

SAMBANOVA_API_KEY

Nebius

European AI Studio
"nebius"

NEBIUS_API_KEY

Novita

GPU marketplace pricing
"novita"

NOVITA_API_KEY

Hyperbolic

Spare GPU capacity
"hyperbolic"

HYPERBOLIC_API_KEY

Your own endpoint

Ollama, vLLM, any OpenAI-compatible host
"custom""ollama""local""vllm"

POLYGATE_BASE_URL

Adding a provider is one adapter file and one registry line. Contribute one

Every call returns the same shape

No more learning four response formats. Anything a provider returns that polygate does not normalize is still there, untouched, under raw.

Extra keyword arguments pass straight through to the provider, so provider-specific options like tools, top_p, or stop sequences keep working without polygate needing to know about them.

content
the assistant's reply text
role
always "assistant"
model
the model that was actually used
provider
which provider served the request
usage
prompt, completion, and total token counts
raw
the untouched original provider response

Rotate keys, ride out rate limits

A rate limit is per key, not per host. Sleeping through a 429 while another key sits idle wastes the whole backoff window, so polygate rotates first and only backs off once it runs out of keys.

A key the provider rejects outright is dropped for the rest of the process rather than retried forever. Deterministic failures are never retried at all: a 400, 404, or 422 cannot succeed on a second attempt, so sending it again just bills you twice.

Both features are off unless you ask for them. A single api_key with no retry behaves exactly as it always has.

from polygate import chat, Retry
 
response = chat(
    provider="anthropic",
    model="claude-sonnet-4-6",
    messages=[{"role": "user", "content": "Say hi."}],
 
    # One key, a list, or a reusable KeyPool.
    api_key=["sk-1", "sk-2", "sk-3"],
 
    # Omit retry= for the old behavior: one request, no retries.
    retry=Retry(max_attempts=4),
)

Small on purpose

polygate is not trying to out-feature LiteLLM, Portkey, or Helicone. It is the smallest layer that normalizes requests and responses across providers, with each adapter readable in under 100 lines. Key rotation and backoff are opt-in and off by default; caching and routing are still yours to build on top, with whatever opinions your project actually needs.

What it does

  • One chat function across all providers
  • One message format, one response shape
  • API keys from arguments or environment
  • Passthrough for provider-specific options

What it leaves to you

  • Retries and backoff
  • Caching and cost tracking
  • Routing and fallback logic
  • Framework integrations