Serverless GPU apps and Model APIs for image, video, audio, and custom inference
fal documents a serverless platform at fal.ai/docs where teams deploy custom models as Python `fal.App` classes with `@fal.endpoint` handlers on auto-scaling H100/A100/B200 runners, or call 1,000+ hosted Model APIs through a unified client. The workflow uses `fal run` for temporary cloud testing and `fal deploy` for persistent endpoints (for example `your-username/my-model` via `fal_client.subscribe` or `https://queue.fal.run/`). Docs describe `setup()` for one-time model loading, machine_type GPU selection, auth modes (private vs public), per-second Serverless billing versus hourly fal Compute for training, and built-in App Analytics with Prometheus-compatible metrics.
Use cases
- Deploy proprietary diffusion or video pipelines without operating Kubernetes
- Prototype with `fal run` then promote the same code to a private authenticated endpoint
- Mix hosted Model APIs for commodity tasks with custom Serverless apps for fine-tunes
- Run fine-tuning on fal Compute while serving inference on Serverless autoscale
- Publish apps to the fal marketplace so external callers use their own API keys
Key features
- Native `fal.App` with `@fal.endpoint`, `@fal.realtime`, and `@fal.function` decorators per Python SDK reference
- Model APIs marketplace for image, video, audio, speech, and 3D workloads via unified fal client
- `fal deploy` remote container builds with revision rollbacks and environment separation
- Configurable machine_type fallbacks (for example GPU-H100 then GPU-A100) and keep_alive warm runners
- Prometheus metrics, error analytics, and log drains documented for production apps
Who Is It For?
- Generative-media engineers shipping GPU endpoints
- ML teams needing per-second billing for bursty inference
- Startups combining marketplace models with custom `fal.App` containers
Frequently Asked Questions
- How is fal Serverless different from Model APIs?
- Model APIs call fal-hosted models; Serverless runs your own `fal.App` code with your weights and container environment per fal.ai documentation.
- Do I need Docker locally to deploy?
- Docs state `fal deploy` builds containers remotely via Depot—you need the fal CLI and Python code, not a local Docker daemon.
- What is the default production auth mode?
- Documentation notes `fal deploy` defaults to private auth requiring an API key unless you configure public access.
Related
Related
3 Indexed items
RunPod
RunPod documents a serverless platform at docs.runpod.io where teams deploy containerized AI handlers without managing servers, paying only for compute time used. Developers write Python handler functions with the Runpod SDK (`runpod.serverless.start`), package Docker images, and expose queue-based endpoints at `https://api.runpod.ai/v2/{ENDPOINT_ID}/runsync` or `/run` with `Authorization: Bearer RUNPOD_API_KEY`. Docs cover streaming handlers, load-balancing endpoints with custom HTTP frameworks, Pods for persistent GPUs, network volumes, and a REST API at rest.runpod.io for programmatic resource management.
Modal
Modal documents a serverless cloud at modal.com where engineers run compute-intensive Python with zero infrastructure configuration: deploy OpenAI-compatible LLM services, batch workflows, job queues, GPU training and fine-tuning, and thousands of isolated Sandboxes for agent-generated code. Official guides show defining apps with `@app.function`, container images via `modal.Image`, and GPU types in code rather than YAML. Modal states pricing is per-second serverless usage with pooled capacity across major clouds, and supports calling functions from JavaScript/Go clients in addition to Python.
Baseten
Baseten documents a training and inference platform at docs.baseten.co where teams deploy models via the open-source Truss framework or call hosted Model APIs without standing up infrastructure. Config-only Truss deployments point at Hugging Face checkpoints, select GPU resources, and engines such as TensorRT-LLM; `truss push` builds optimized containers and exposes OpenAI-compatible sync endpoints like `https://model-{model_id}.api.baseten.co/environments/production/sync/v1`. Custom architectures use a Truss `Model` class with `load` and `predict` in `model.py`. Model APIs provide immediate OpenAI-SDK-style access to catalog models (DeepSeek, Qwen, GLM, and others listed in docs) using `BASETEN_API_KEY`.