Together AI

Cloud platform for running open-source AI models cost-effectively.

Developer API

Overview

Together AI is the production-serving platform for open-weight AI in 2026 — a cloud where developers run Llama 4, DeepSeek V3, Qwen 2.5, Mistral, Mixtral, FLUX image models, and roughly 200 other open-weight models at per-token prices that undercut every closed-provider frontier API, with fine-tuning, dedicated endpoints, and a serverless inference tier under one roof. If your production workload is built on open weights and cost matters, Together AI is probably on your shortlist alongside Groq and Fireworks.

Founded in 2022 by Vipul Ved Prakash and a team of ex-Apple and ex-Stanford ML engineers, Together AI has grown into the second-largest open-weight inference host in the market and one of the most active contributors to open-source AI infrastructure. The company publishes benchmark results transparently, invests heavily in inference kernel optimization (the FlashAttention, Sequoia, and Medusa work all traces back to Together-affiliated researchers), and prices its API surface to reward heavy production usage rather than one-off prototyping.

The one-line positioning: Together AI is the pick when open-weight quality is enough and per-token cost is the constraint — the broadest hosted open-weight catalog with the deepest fine-tuning and dedicated-endpoint story in the category. It is not the absolute fastest (Groq wins on LPU throughput) and not the widest catalog (Hugging Face wins on model count), but the balance of catalog, price, latency, and customization is the strongest general-purpose bet for production open-weight workloads.

The product surface splits into four real products: Serverless Inference (pay-per-token, OpenAI-compatible), Dedicated Endpoints (reserved GPU capacity for guaranteed throughput), Fine-tuning (LoRA and full fine-tuning across the open-weight catalog), and Together Code Sandbox / Together Compute (raw GPU clusters for training runs). Most production teams live on Serverless plus a Dedicated Endpoint for their hottest model.

Key Features

Together AI's feature set in 2026 covers the full lifecycle of open-weight inference:

  • Serverless inference across 200+ open-weight models. Llama 4 8B / 70B / 400B, Llama 3.3 70B, DeepSeek V3, Qwen 2.5 32B/72B, Mixtral 8x22B, Mistral Nemo, Yi, Command R+, and specialized code and math models. When a major open-weight release ships, Together is typically live within 48 hours.
  • OpenAI-compatible API surface. Chat completions endpoint that accepts the same request shape as OpenAI. Switch base URL and API key and existing SDK code runs. Function calling, JSON mode, and streaming are all supported on the models that support them.
  • Fine-tuning across the open-weight catalog. LoRA and full fine-tuning available for most models. Pay per-token of training data plus a per-model base cost. Fine-tuned checkpoints deploy as serverless endpoints or dedicated endpoints without re-uploading.
  • Dedicated endpoints with reserved capacity. Pin a model to a specific GPU (A100, H100, H200) with guaranteed throughput and no cold starts. Priced per GPU-hour, cheaper than the equivalent on Hugging Face Inference Endpoints for most SKUs.
  • Image and multimodal models. FLUX.1 dev and FLUX.1 pro image generation, Stable Diffusion 3.5, and multimodal Llama variants. Serverless per-image pricing that undercuts most closed image APIs.
  • Together Embeddings. Hosted embeddings models (BGE, GTE, and specialized code embeddings) at per-token prices competitive with OpenAI's text-embedding-3-small.
  • Together Compute for training runs. Raw GPU cluster access (H100, H200) for teams doing pre-training or heavy fine-tuning outside the managed fine-tuning product. Pricing is competitive with AWS on-demand H100 for the same SKUs.
  • Streaming with 200-300 tok/sec on 70B-class models. Not as fast as Groq's LPU throughput, but genuinely competitive on GPU inference — often 3-4x faster than the same model served on a hyperscaler.
  • Tool use and function calling. Native support on the models that support it (Llama 3.3+, DeepSeek V3, Qwen 2.5). Not as polished as Anthropic's tool use, sufficient for most production agentic patterns.
  • Batch API at 50% off. Standard batch discount for asynchronous workloads.
  • First-class SDK support. Official Python and TypeScript SDKs plus OpenAI SDK compatibility. Most existing OpenAI SDK code migrates in minutes.

Pricing

Together AI's serverless inference is per-token, tiered by model size and family. Approximate per-million-token prices in mid-2026:

Model Input ($/M) Output ($/M) Speed (tok/s)
Llama 4 400B $1.80 $1.80 ~120
Llama 4 70B $0.88 $0.88 ~250
Llama 4 8B $0.18 $0.18 ~350
Llama 3.3 70B $0.88 $0.88 ~275
DeepSeek V3 $1.25 $1.25 ~180
DeepSeek R1 $3.00 $7.00 ~90
Qwen 2.5 72B $1.20 $1.20 ~200
Mixtral 8x22B $1.20 $1.20 ~180
FLUX.1 dev $0.025/image — ~5 sec
FLUX.1 pro $0.05/image — ~7 sec

Dedicated endpoints price per GPU-hour: roughly $1.30/hour for A100 80GB, $2.60/hour for H100, $4.00/hour for H200, with committed-usage discounts for annual reservations. Fine-tuning bills at $0.30-$3.00 per million training tokens depending on model size, plus base fees.

The pricing that actually matters: Llama 3.3 70B at $0.88/$0.88 is roughly one-sixth the price of GPT-5 mini for input/output tokens and 3-5x the throughput on generation. For workloads where Llama-70B-tier quality is sufficient, the total-cost delta versus a closed frontier provider is dramatic. DeepSeek V3 at $1.25/$1.25 offers GPT-5-mini-adjacent quality on many benchmarks at a similar price point.

Pros and Cons

Pros

  • Broadest hosted open-weight catalog with production-grade serving quality
  • Per-token prices that undercut closed frontier providers by 3-5x on comparable-quality workloads
  • OpenAI-compatible API means near-zero migration cost from existing SDK code
  • Fine-tuning is a real first-class product, not a bolted-on afterthought
  • Dedicated endpoints solve the cold-start and capacity-guarantee problem cleanly
  • Image and multimodal models (FLUX, SD3.5) at prices closed APIs don't match
  • Batch API and volume discounts scale sensibly at production scale
  • Transparent benchmark publishing — you can trust the marketing numbers
  • Strong support for the newest open-weight releases (usually live within 48 hours)

Cons

  • Not the absolute fastest — Groq's LPU throughput is 1.5-2x higher on the same models
  • No frontier closed models (no GPT-5, no Claude, no Gemini) — this is open-weight-only
  • Free tier is limited compared to Groq's — production evaluation usually requires topping up credits
  • Rate limits on new accounts are conservative and require billing history to unlock
  • Dedicated endpoints have real minimum monthly commitments once you commit to reserved capacity
  • Documentation is functional but less exhaustive than OpenAI's or Anthropic's
  • Context windows track the underlying open-weight defaults (typically 128K on Llama 4)

Best Use Cases

  • Cost-sensitive production workloads where open-weight quality is enough. Classification, extraction, moderation, RAG, agent loops — anywhere GPT-5 flagship is over-provisioned, Together AI's 3-5x cost reduction is the biggest lever in your stack.
  • Fine-tuning open-weight models on your own data. Together's fine-tuning surface is the best-integrated of any open-weight host — you fine-tune, deploy, and serve from the same platform without re-hosting.
  • Multi-model production stacks. Run Llama 3.3 70B for chat, Qwen 2.5 for classification, DeepSeek V3 for hard reasoning, and FLUX for image generation — all through one API key with one billing surface. This is the workload where Together's catalog breadth compounds.
  • Startups scaling out of OpenAI pricing. Teams that prototyped on OpenAI and want to cut per-token costs at scale without rewriting their SDK code — Together's OpenAI compatibility plus per-token savings is the shortest migration path.
  • Agentic loops with many small calls. Same per-call latency reasoning as Groq — high-frequency LLM calls compound cost, and Together's per-token prices multiply savings across the loop.
  • Image generation at scale. FLUX.1 pro at $0.05/image is dramatically cheaper than DALL-E 3 at equivalent quality for most tasks. Production image generation pipelines benefit substantially.
  • Enterprise pilots and POCs. Together's dedicated endpoint pricing plus transparent billing makes enterprise POCs cleaner than opaque hyperscaler AI pricing.

Alternatives

Together AI competes primarily on the open-weight tier, and different competitors win on different axes:

  • Groq — faster inference on a narrower catalog. Pick Groq if latency dominates and the model you need is in Groq's curated list.
  • Hugging Face — broader model catalog and dedicated endpoints for any model on the Hub. Pick Hugging Face if you need a specific long-tail model Together does not host, or if you want the Enterprise Hub tooling.
  • OpenAI API — GPT-5 family. Pick this if you need frontier closed-model quality.
  • Anthropic API — Claude Sonnet 4.5 and Opus 4.7. Pick this for coding depth, long-form writing, or 1M-token context.
  • Fireworks AI — closest direct competitor on the "fast + cheap open-weight" positioning. Speed and price are within margin of Together's for most models; worth benchmarking on your actual workload.
  • Replicate — community-run open-model APIs. Broader long-tail catalog but pay-per-second billing that gets expensive at production scale.
  • DeepInfra, Anyscale, Perplexity Labs — smaller open-weight hosts, competitive on niche workloads.

For the consumer-side comparisons of models Together does not host, see ChatGPT, Claude, and Gemini. For the broader token-pricing landscape, read The Economics of AI Inference at Scale.

Getting Started

  1. Sign up at api.together.ai and generate an API key. New accounts get a small credit to run evaluation traffic — enough to compare quality on your actual workload before committing.
  2. Point your OpenAI SDK at Together. For the Python openai SDK: client = OpenAI(base_url="https://api.together.xyz/v1", api_key=os.environ["TOGETHER_API_KEY"]). Change the model name to meta-llama/Llama-3.3-70B-Instruct-Turbo or similar and existing code works.
  3. Benchmark cost and quality on your actual traffic. The 3-5x cost delta versus closed providers is easy to measure. The quality delta versus GPT-5 mini varies by workload — some tasks are indistinguishable, others show a gap. Only your traffic tells you which category each of your calls falls into.

For production traffic, consider a dedicated endpoint on your hottest model once serverless throughput becomes a bottleneck — the per-GPU-hour cost is often lower than the equivalent serverless spend at high utilization. For fine-tuning, prepare data in the standard JSONL format and use LoRA before considering full fine-tuning; LoRA is cheaper, faster, and usually sufficient.

FAQ

Does Together AI host GPT-5 or Claude? No. Together AI is open-weight-only by design. If you need GPT-5, use the OpenAI API; for Claude, use the Anthropic API.

How does Together compare to Groq? Different bets on the same open-weight tier. Groq wins on raw throughput (custom LPU hardware, 400+ tok/sec on 70B models) and has a narrower curated catalog. Together wins on catalog breadth, fine-tuning integration, and dedicated-endpoint flexibility. For most production workloads, benchmark both — the answer is workload-specific.

Can I fine-tune models on Together AI? Yes. LoRA and full fine-tuning are supported across most of the open-weight catalog. Fine-tuned models deploy to serverless or dedicated endpoints without re-uploading. This is one of Together's most differentiated features against Groq and closed-provider APIs.

Does Together train on my API traffic? No. API traffic is not used for model training. Fine-tuning data you upload is used only for the fine-tune you request. Data privacy terms are explicit in the enterprise agreement for dedicated deployments.

What's the largest context window on Together? Tracks the underlying open-weight model. Llama 4 supports 128K on Together as of mid-2026; DeepSeek V3 supports 128K; some specialized models support 200K+. Larger windows exist on frontier closed providers (Claude Sonnet at 1M, Gemini at 2M).

Verdict

Together AI is the production-serving platform of choice for open-weight AI in 2026. For teams whose workload is served well enough by Llama 4, DeepSeek V3, Qwen 2.5, or the broader open-weight catalog, Together delivers a 3-5x cost reduction versus closed frontier providers on serverless inference, plus fine-tuning and dedicated endpoints under the same umbrella. The OpenAI compatibility means migration cost is measured in minutes, and the transparent pricing makes total-cost planning easier than most hyperscaler AI SKUs.

Where Together stops being the obvious pick is on the two extremes: absolute fastest latency (Groq wins on custom silicon) and frontier closed-model quality (OpenAI and Anthropic win on flagship models). Teams whose UX requires sub-150ms time-to-first-token will find Groq's LPU advantage worth accepting a narrower catalog for. Teams whose product depends on GPT-5- or Opus-4.7-tier reasoning cannot substitute Llama 70B and expect the same output.

The honest recommendation for most production teams in 2026: treat Together AI as the cost-optimized open-weight tier of a multi-provider stack. Route the majority of production traffic — classification, extraction, RAG, agentic loops, image generation — to Together and reserve frontier providers for the calls where flagship quality is measurably worth the price. Add Groq for the specific latency-sensitive UX calls where 400 tok/sec versus 250 tok/sec is the difference. Most serious AI products in 2026 use two or three of these providers together, and Together AI is typically the biggest single line item on the bill because it is doing the most volume at the lowest per-call cost.

Explore more in the Developer API category, or read The Economics of AI Inference at Scale for the broader open-weight economics.