High-speed inference API with an OpenAI-compatible chat completions surface
Cerebras documents Shared Inference and Dedicated Inference at inference-docs.cerebras.ai. Shared Inference on Free Trial and Pay as You Go tiers lists models such as gpt-oss-120b (about 3000 tokens/s, 65k context on free / 131k on paid) and qwen-3.8-27b (about 1850 tokens/s, 64k / 128k) per the model catalog. Chat completions use the Cerebras Cloud SDK (`cerebras.cloud.sdk` in Python, `cerebras_cloud_sdk` in TypeScript) with `client.chat.completions.create()`, message roles including system, user, assistant, tool, and developer, plus optional `reasoning_effort` and `stream`. Docs describe tool calling (`tool_choice` none/auto/required or a named function, strict mode, parallel calls) for gpt-oss-120b and qwen-3.8-27b on Shared Inference. Additional model families, reserved capacity, and production SLAs are documented under Dedicated Inference. The catalog states Shared Inference models are unpruned originals, with selective weight-only quantization for storage and full-precision activations.
Use cases
- Low-latency chat and agent loops that need high tokens-per-second inference
- OpenAI SDK-style migrations onto Cerebras chat completions
- Tool-using agents on gpt-oss-120b or qwen-3.8-27b
- Compare Shared Inference rate limits with Dedicated Inference SLAs before production
- Pair with observability tools on this site when logging Cerebras request usage
Key features
- OpenAI-style chat completions via the Cerebras Cloud SDK
- Shared Inference models gpt-oss-120b and qwen-3.8-27b with published speed and context windows
- Streaming responses via stream=true
- Tool calling with strict mode and parallel calls on documented Shared Inference models
- Dedicated Inference for additional model families and reserved capacity
Who Is It For?
- Developers who want an OpenAI-compatible inference API with published throughput figures
- Teams evaluating fast open-weight serving versus hosted frontier APIs
- Platform engineers choosing shared versus dedicated Cerebras capacity
Frequently Asked Questions
- Which models does Shared Inference list?
- The model catalog lists gpt-oss-120b and qwen-3.8-27b for Shared Inference. Other families are described under Dedicated Inference.
- Is the API OpenAI-compatible?
- Docs show chat completions through the Cerebras Cloud SDK with message roles and streaming, matching the common chat-completions pattern.
- Does Cerebras host pruned models on Shared Inference?
- The catalog says Shared Inference serves original unpruned models; pruned research models are not offered on Shared Inference.
Related
Related
3 Indexed items
CoreWeave
CoreWeave documents inference products at docs.coreweave.com/products/inference spanning Serverless, Dedicated (BYOW on H100/B200/A100-class GPUs), and CKS options, all exposing OpenAI API-compatible endpoints per the inference introduction. The Inference API at api.coreweave.com (v1alpha1) manages gateways, deployments, and capacity claims over REST/JSON, gRPC, or Connect with Bearer tokens requiring Inference Viewer or Inference Admin roles. Getting-started guides walk through gateway creation with IAM authentication, body-based routing on the model field, and chat completion requests against deployed weights in CoreWeave Object Storage.
NVIDIA NIM
NVIDIA NIM documents performance-optimized inference microservices at docs.api.nvidia.com/nim and docs.nvidia.com/nim that expose industry-standard APIs (OpenAI-compatible `/v1/chat/completions`, `/v1/completions`, `/v1/responses`, Anthropic-compatible `/v1/messages`) from containerized models backed by TensorRT-LLM, vLLM, or SGLang per deployment. Teams can self-host GPU-accelerated models on cloud, data center, or RTX workstations, or prototype via NVIDIA-hosted NIM API endpoints through the Developer Program. Management endpoints such as `/v1/health/ready` and `/v1/metrics` support readiness probes and Prometheus metrics on self-hosted containers per the LLM API reference.
Baseten
Baseten documents a training and inference platform at docs.baseten.co where teams deploy models via the open-source Truss framework or call hosted Model APIs without standing up infrastructure. Config-only Truss deployments point at Hugging Face checkpoints, select GPU resources, and engines such as TensorRT-LLM; `truss push` builds optimized containers and exposes OpenAI-compatible sync endpoints like `https://model-{model_id}.api.baseten.co/environments/production/sync/v1`. Custom architectures use a Truss `Model` class with `load` and `predict` in `model.py`. Model APIs provide immediate OpenAI-SDK-style access to catalog models (DeepSeek, Qwen, GLM, and others listed in docs) using `BASETEN_API_KEY`.