# Together AI

Canonical: https://inetgeek.com/llm-apis/together-ai/

Serverless per-token inference for open-weight models from other labs, plus dedicated GPU endpoints, hourly GPU clusters and fine-tuning, paid from prepaid credits.

## Who it suits

Open-weight models from many labs behind one prepaid API: Kimi, Qwen, DeepSeek, GLM and Gemma, from $0.09 per million input tokens, with dedicated GPU endpoints and H100 clusters at $3.99 an hour when serverless limits are not enough.

Consider something else if You need a free trial, fixed rate limits or a chosen region: there is no free offer, rate limits rise with traffic and are not published per model, serverless requests pick no region, and HIPAA covers dedicated deployments only.

## iScore

56 / 100 (80% interval 47–64), rank 6 of 10 in llm-apis — interval overlaps the next rank, so this is an ordering, not a finding.

inetGeek's own reading, computed from sourced facts against category peers by disclosed rules. Not a provider claim. Intervals are 80%; a provider with few documented facts is pulled toward the category middle rather than scored on what little is known.
Method: https://inetgeek.com/iscore/. Full output with every criterion's basis: https://inetgeek.com/scores.json. Every llm-apis peer ranked: https://inetgeek.com/llm-apis/together-ai/alternatives/

- Price: 32 (18–45), weight 42%, 6 of 7 criteria scored; scored for peers, not here — undocumented or stated without a number or a yes/limited/no: free_card_required
- Capacity and limits: 61 (43–79), weight 30%, 2 of 4 criteria scored; scored for peers, not here — undocumented or stated without a number or a yes/limited/no: max_output_tokens, regions
- Capabilities: 100 (94–100), weight 5%, 1 of 1 criteria scored
- Trust and compliance: 76 (66–86), weight 12%, 7 of 12 criteria scored; scored for peers, not here — undocumented or stated without a number or a yes/limited/no: customer_managed_keys, encryption_at_rest, pci_dss, support_included, uptime_sla
- Evidence: 94 (94–94), weight 11%, 4 of 4 criteria scored

## What the documentation says

### Pricing

- Free tier: No ([source](https://docs.together.ai/docs/billing-credits), read 2026-09-25)
- Kind of free offer: No free offer ([source](https://docs.together.ai/docs/billing-credits), read 2026-09-25)
- Entry paid plan: No plan fee, but the platform is fully prepaid: access requires a minimum $5 credit purchase, and API access is suspended when the balance reaches zero. Purchased credits do not currently expire. ([source](https://docs.together.ai/docs/billing-credits), read 2026-09-25)
- Pricing model: Four products priced separately: serverless inference per 1M input and output tokens for each model, with a lower cached-input rate on some; dedicated inference and GPU clusters per GPU-hour; and fine-tuning per 1M training tokens, with a minimum charge per job. ([source](https://www.together.ai/pricing), read 2026-09-25)

### Infrastructure

- Regions: Serverless requests are routed across Together's own infrastructure with no region selection; DeepSeek models are named as hosted in North America data centers. Region choice, including EU regions, comes through dedicated endpoints or an arrangement with Together. ([source](https://docs.together.ai/docs/privacy-and-security), read 2026-09-25)

### Inference

- Input price, top model: $3.00 per million input tokens for Kimi K3, the most expensive model in the serverless chat table, with cached input at $0.30. Together hosts other labs' open-weight models, so "flagship" here means the top of what it serves. ([source](https://www.together.ai/pricing), read 2026-09-25)
- Output price, top model: $15.00 per million output tokens for Kimi K3. ([source](https://www.together.ai/pricing), read 2026-09-25)
- Input price, cheapest model: $0.09 per million input tokens for Qwen3.8 Flash, with output at $0.28 - the cheapest paid chat model on the pricing page. DeepSeek V4 Flash 0731 is $0.14 in and $0.28 out. Ternary Bonsai 27B is listed at $0.00 for both input and output. ([source](https://www.together.ai/pricing), read 2026-09-25)
- Context window: 1,048,576 tokens on Kimi K3 and on DeepSeek V4 Pro 0813 and V4 Flash 0731; 1,048,575 on GLM-5.3 and GLM-5.2; 1,000,000 on Qwen3.8 Flash and the Qwen3.6/3.7 Plus models; 131,072 on gpt-oss-120b and Llama 3.3 70B. The window is the model's, as listed in the docs catalogue. ([source](https://docs.together.ai/docs/serverless/models), read 2026-09-25)
- Max output tokens: No separate output cap is published: max_tokens is unset by default, and the model generates until it hits a stop condition or the context limit. ([source](https://docs.together.ai/docs/inference/chat/parameters), read 2026-09-25)
- Prompt caching: Supported ([source](https://docs.together.ai/docs/serverless/overview), read 2026-09-25)
- Batch discount: Batch API advertised at up to 50% off serverless rates, but only two models are listed as discounted, at 50%: Llama 3.3 70B Instruct Turbo and Whisper Large v3. Other models run at standard rates, and batch jobs on dedicated inference get no discount. ([source](https://docs.together.ai/docs/inference/batch/overview), read 2026-09-25)
- Rate limits: Dynamic limits set per organization and per model, rising with sustained successful traffic; no fixed per-model figures are published. Requests over the limit get a 429; dedicated endpoints are the route to a fixed limit. ([source](https://docs.together.ai/docs/serverless/rate-limits), read 2026-09-25)

### GPU

- H100 SXM, per GPU-hour: $3.99 per GPU-hour for an on-demand NVIDIA HGX H100 on GPU Clusters; preemptible is $1.99, and reserved falls to $3.19 for 91-180 days. ([source](https://www.together.ai/pricing), read 2026-09-25)
- Largest GPU offered: 280GB per GPU on a B300 or GB300 for dedicated inference, both priced by sales. The largest card with a listed hourly price there is the B200 180GB, at $8.99. ([source](https://docs.together.ai/docs/dedicated-endpoints/pricing), read 2026-09-25)

### Compliance

- SOC 2: SOC 2 Type 2; the latest report was posted to the Together AI Trust Center on June 17, 2026. ([source](https://trust.together.ai/), read 2026-09-25)
- ISO 27001: ISO 27001:2022 certified, announced May 29, 2026; the certificate is listed in the Trust Center. ([source](https://trust.together.ai/), read 2026-09-25)
- HIPAA: Dedicated inference deployments can be restricted to HIPAA-attested clusters with a placement compliance policy. No BAA, and no HIPAA coverage for serverless, is documented. ([source](https://docs.together.ai/docs/dedicated-endpoints/manage), read 2026-09-25)
- GDPR / data residency: No region choice on serverless. For GDPR-driven residency, Together offers EU-region private networking and VPC-based deployments through dedicated endpoints or by arrangement. ([source](https://docs.together.ai/docs/privacy-and-security), read 2026-09-25)
- Role-based access control: Role-based access control at two levels: organization roles admin and developer, project roles admin and editor, each with a documented permissions table. Together says roles are still being rolled out across its products. ([source](https://docs.together.ai/docs/roles-permissions), read 2026-09-25)
- Private networking / VPC: Private networking and VPC-based deployments, including in EU regions, for customers with residency or compliance needs; arranged through dedicated endpoints or with Together, with no mechanism named. ([source](https://docs.together.ai/docs/privacy-and-security), read 2026-09-25)
- Dedicated infrastructure: Dedicated model inference serves a model on reserved GPUs billed per GPU-minute, with no hard rate limits beyond the hardware and the autoscaling bounds. ([source](https://docs.together.ai/docs/dedicated-endpoints/overview), read 2026-09-25)