Provider

Groq

Open models on LPU silicon, very fast.

Quick start

import polygate
 
response = polygate.chat(
    provider="groq",
    model="llama-3.3-70b-versatile",
    messages=[{"role": "user", "content": "Say hi in one word."}],
)
 
print(response.content)   # "Hi!"
print(response.usage)     # Usage(prompt_tokens=..., completion_tokens=...)

Names

Any of these work as provider. They all reach the same adapter, so pick whichever reads best in your code.

Configuration

Any of these may hold a comma-separated list. polygate rotates across them, parks a rate-limited one for its Retry-After, and drops one the provider rejects — so a key pool is configuration, not code.

Endpoint

https://api.groq.com/openai/v1

Pass base_url to point the same wire format somewhere else — a proxy, a regional endpoint, or a gateway in front of it.

Models

Examples that work today. Model names change; the provider's own list is the authority.

Things worth knowing

• Groq hosts other people's models, so a model id names no host. llama-3.3-70b-versatile is not identifiable as Groq's from the name alone — always pass provider explicitly.
• Groq's batch tier accepts a 7-day completion window as well as 24 hours, and polygate asks for the longer one: under load a 24h job is likelier to expire than to finish, and an expired batch has to be resubmitted whole.

Batch

Groq has an offline batch tier at roughly half price. See Batching & concurrency.