Self-hosted

Your own endpoint

Ollama, vLLM, LM Studio, or any OpenAI-compatible host.

Quick start

import polygate
 
response = polygate.chat(
    provider="custom",
    base_url="...",
    model="llama3.1",
    messages=[{"role": "user", "content": "Say hi in one word."}],
)
 
print(response.content)   # "Hi!"
print(response.usage)     # Usage(prompt_tokens=..., completion_tokens=...)

Names

Any of these work as provider. They all reach the same adapter, so pick whichever reads best in your code.

Configuration

Any of these may hold a comma-separated list. polygate rotates across them, parks a rate-limited one for its Retry-After, and drops one the provider rejects — so a key pool is configuration, not code.

Endpoint

There is no fixed host: the URL contains your own account, workspace or project, so base_url is required. Omit it and polygate raises MissingEndpointError naming the variable to set, rather than letting the request fail somewhere less informative.

Models

Examples that work today. Model names change; the provider's own list is the authority.

Things worth knowing

• No API key is required. Local runtimes authenticate nothing, and demanding a key would fail the most common case — a laptop running Ollama — for a reason that does not exist.
• Defaults to http://localhost:11434/v1, which is Ollama's. The ollama, local and vllm aliases exist because someone reaching for those names should not be told the provider is unknown.

Pointing at a local server

import polygate
 
response = polygate.chat(
    provider="ollama",
    base_url="http://localhost:11434/v1",
    model="llama3.1",
    messages=[{"role": "user", "content": "Hello"}],
)

Batch

Your own endpoint has no offline batch API. To run many requests at once, use map_chat — full price, but it works everywhere and returns in seconds. See Batching & concurrency.