For teams that want custom training code alongside scalable inference and batch processing, Modal is the leading choice in this guide. Its training infrastructure combines code-defined workloads, on-demand NVIDIA GPUs, multi-node Clusters with RDMA networking, and per-second compute billing. Modal also provides CPU compute and secure Sandboxes for adjacent AI applications, making it a versatile platform beyond GPU training alone.
Key Takeaways
- Code-first model development: Modal lets developers define training Functions, container Images, GPU resources, and distributed Clusters in Python. Its SDK ecosystem also includes JavaScript and TypeScript and Go clients for working with Modal resources. Sandboxes can execute programs in the languages their workloads require.
- Distributed training with RDMA: Modal Clusters combine coordinated node scheduling with high-bandwidth communication. The October 2026 Clusters announcement describes RDMA fabric bandwidth of up to 6.4 Tbps per node. An earlier product update documented 3.2 Tbps InfiniBand for multi-node Clusters.
- Fast cold starts and efficient iteration: Modal Function containers use an optimized filesystem to come online quickly without letting large Images unnecessarily slow startup. GPU Memory Snapshots can streamline initialization for compatible deployed inference Functions. Training startup also includes data and checkpoint preparation, which is distinct from container cold-start time.
- Usage-based compute: Modal bills compute by the second and supports scale-to-zero Functions, aligning resource usage with experiments and scheduled training jobs. Billing is based on resources used or requested, including intentionally retained warm compute.
- One platform across AI workflows: Modal combines model training, inference, batch processing, and isolated Sandboxes in an integrated infrastructure platform for CPU and GPU workloads.
1. Modal
Modal provides code-first serverless infrastructure for training, inference, batch computation, and isolated execution. Developers define GPU training Functions and their runtime environments through Python instead of provisioning servers or operating a Kubernetes cluster. Modal's JavaScript and TypeScript SDK and Go SDK provide additional ways to interact with deployed resources, while Modal Sandboxes support execution of code in different programming languages.
On October 1, 2026, Modal Clusters became generally available. The capability extends Modal's on-demand compute model to coordinated, multi-node GPU training and inference. Modal has highlighted production use cases including 1X's world-model pretraining and Decagon's post-training workloads.
How Does Modal Work?
Modal packages code and dependencies into reusable container Images and schedules workloads on available compute infrastructure. Developers can select NVIDIA GPU options such as B300, B200, H200, H100, and A100, alongside CPU resources. Training jobs can run on a single GPU, multiple GPUs on one node, or a coordinated multi-node Cluster.
- Code-first infrastructure: Python decorators and custom Images define dependencies, hardware, and execution settings. JavaScript, TypeScript, and Go SDK clients integrate Modal resources into other application backends.
- Fast cold starts: Engineered for fast cold starts and shorter feedback loops, Modal Functions use an optimized filesystem that makes files available as needed. This helps Function containers come online quickly, including when Images contain substantial dependencies.
- Multi-node GPU Clusters: Modal uses the
@modal.clustereddecorator to schedule GPU containers together, with cluster size defined in code and bounded by workspace GPU limits. Each clustered container requests all GPUs on its host, andrdma=Trueenables RDMA networking on supported hardware. - GPU flexibility: Teams can match GPU memory and compute characteristics to fine-tuning, pretraining, evaluation, or inference, while using the same platform for CPU-based preparation and orchestration.
- Per-second compute billing: Modal's billing model meters compute resources used or requested, and autoscaling lets Functions scale with their workload.
Distributed Training Capabilities
The Modal Clusters announcement and current Python SDK reference describe coordinated GPU training with @modal.clustered, including optional RDMA networking for PyTorch and NCCL.
- RDMA networking: The Modal Clusters announcement describes RDMA fabric bandwidth of up to 6.4 Tbps per node. Modal also documented 3.2 Tbps InfiniBand in an earlier multi-node update. Automated configuration for PyTorch and NCCL supports high-bandwidth GPU communication during distributed training.
- Coordinated scheduling: Cluster nodes are scheduled as a group and communicate over private networking. The Python
rdma=Trueoption enables RDMA placement and configuration on supported GPUs. - Framework flexibility: Modal provides PyTorch and NCCL integration, TensorFlow GPU examples, and customizable Images for compatible training frameworks and dependencies.
- Persistent training artifacts: Modal Volumes, Cloud Bucket Mounts, and configurable execution retries support training code that writes checkpoints and resumes work from saved state.
- Proven AI training applications: The Modal Clusters launch describes 1X using multi-node B300 Clusters for world-model pretraining and separate GPU jobs for evaluations.
Security and Compliance
Modal has completed a SOC 2 Type II audit. Its infrastructure uses gVisor-based workload isolation and documents encryption for data in transit and at rest. Modal supports HIPAA-compliant workloads on Enterprise plans via a Business Associate Agreement. These security capabilities complement its managed GPU infrastructure for production AI development.
Best For: ML engineers and AI teams seeking code-defined custom training, flexible NVIDIA GPU compute, RDMA-capable multi-node Clusters, and integrated infrastructure for training, inference, batch processing, and secure execution.
2. Google Gemini Enterprise Agent Platform
Google's Gemini Enterprise Agent Platform, renamed from Vertex AI on April 22, 2026, offers managed custom model training within Google Cloud. The service provisions infrastructure for submitted jobs, while teams define the training code, accelerators, worker pools, and data access. It integrates with BigQuery and Cloud Storage.
Key Features
- Managed multi-worker training jobs with user-defined worker pools, GPU types, and replica counts
- Cloud Storage and BigQuery data sources for ML workflows
- PyTorch, TensorFlow, and custom training container support
- Reduction Server optimization for NVIDIA GPU all-reduce in distributed training
- Managed training infrastructure with Google Cloud security and governance features
- Eligible H200, B200, and GB200 training compute configurations
Enterprise Focus
The platform supports organizations using BigQuery and Cloud Storage as part of custom training pipelines. Developers configure worker topology, accelerators, and regional resources for each job, while the managed infrastructure handles provisioning and release of compute. Data services, training jobs, and Google Cloud governance tools can be combined into a broader model development workflow.
Best For: Enterprises using Google Cloud that want managed custom training jobs and integration with BigQuery, Cloud Storage, and Google Cloud governance tools.
3. Azure Machine Learning Serverless Compute
Azure Machine Learning serverless compute provides managed, on-demand training job compute, including GPUs, within the Microsoft Azure ecosystem. It supports model training without maintaining a user-created Azure Machine Learning compute cluster and integrates with Azure identity, governance, and MLOps services.
Core Capabilities
- On-demand serverless compute for training jobs, including command, AutoML, sweep, parallel, and pipeline workloads
- PyTorch, TensorFlow, and MPI-based distributed training support
- Microsoft Entra ID workspace authentication and user-assigned managed identities for serverless jobs
- Managed network isolation and regional resource placement options
- MLflow and Azure DevOps integrations within Microsoft's broader ML tooling
Compliance and Governance
Azure supports enterprise identity, policy integration, auditing, and networking controls. The cloud security shared-responsibility model and Azure Policy's SOC 2 control mappings describe how cloud platform services and customer configuration participate in enterprise governance. These capabilities fit regulated AI workflows within the scope of an organization's requirements.
Best For: Organizations building Azure-native GPU training jobs with enterprise identity, experiment tracking, governance tools, and Microsoft MLOps integrations.
4. Beam Cloud
Beam Cloud offers a managed, Python-oriented serverless GPU service associated with its open-source Beta9 runtime. Beam is Y Combinator-backed and provides GPU functions, scheduled tasks, and persistent storage for training and batch workflows. Its managed service and the independently deployable runtime are distinct options.
Key Features
- Open-source Beta9 runtime and self-hosting option
- Python SDK for defining GPU functions
- Built-in cron scheduling for periodic GPU jobs, using UTC scheduling
- Persistent volumes for datasets and checkpoints
- Usage-based GPU execution billing, with compute resource selection determined by each workload
Developer Experience
Beam's scheduled GPU jobs support periodic retraining and batch experimentation, while volumes provide storage for datasets and model artifacts. Beam's published cost example describes resource-based billing across serverless GPU workloads. The platform provides a Python interface for teams developing custom training tasks.
Best For: Teams running Python-based GPU fine-tuning, scheduled retraining, and checkpointed batch experiments, including users interested in the Beta9 runtime.
5. Fal.ai
Fal.ai provides managed training APIs for selected generative image and video models. Its serverless inference product and dedicated fal Compute training machines are separate offerings alongside model-specific LoRA trainer APIs.
Specialized Capabilities
- Wan 2.2 text-to-video and image-to-video LoRA trainer endpoints
- Optional dedicated fal Compute instances with 8 H100 SXM GPUs as a separate machine-based training option
- Model-specific training support for generative image and video models
- Wan 2.2 trainer constraints and a minimum charge of 100 training steps for the documented trainer endpoints
- Supported Wan 2.2 workflows include a returned
.safetensorsLoRA artifact for compatible inference - API-driven dataset submission, training status, and artifact retrieval
Media Model Focus
The Wan 2.2 guide describes dataset submission, LoRA training, and subsequent use of the adapter with a compatible inference model. The separately billed fal Compute product provides another execution model for custom training programs. These services are oriented toward different training formats, including structured trainer APIs and dedicated GPU machines.
Best For: Teams fine-tuning supported image and video model families through specialized LoRA trainer APIs, with dedicated compute as another option for custom training.
6. Cerebrium AI
Cerebrium AI, a Y Combinator Winter 2022 company, offers serverless GPU infrastructure for custom ML applications, training experiments, and real-time AI services. Cerebrium has announced ISO 27001 compliance and completion of a SOC 2 Type II audit. Its published memory snapshot information principally concerns workload initialization, which is distinct from restoring a model training checkpoint.
Platform Highlights
- CPU/GPU memory snapshots for compatible workloads
- Custom Python GPU workloads, including documented PyTorch/CUDA examples
- SOC 2 Type II audit and ISO 27001 compliance announcement
- Weights & Biases hyperparameter sweep example for training experiments
- Support for custom ML pipelines and model applications
MLOps Integration
Cerebrium's W&B sweep example illustrates tracked hyperparameter experimentation with serverless GPU workloads. Its custom Python environments can run ML application code with compatible dependencies, and its snapshot feature offers a distinct form of initialized-state reuse. A dated 2025 walkthrough also describes the W&B sweep example.
Best For: Teams using Python-based GPU training experiments, parallel evaluations, and Weights & Biases experiment tracking alongside AI application deployment.
7. Together AI
Together AI's managed fine-tuning service provides model customization for supported language and multimodal models. Teams upload datasets, select models and methods, and run training without managing underlying GPU machines. Together AI has advertised training support for eligible models up to 1 trillion parameters. A completed fine-tuning job makes a fine-tuned model available; deployment requires a separate action.
Managed Fine-Tuning
- Supervised fine-tuning and DPO with LoRA or full-model updates
- Fine-tuning support for 100B+ parameter models in eligible configurations
- Catalog of supported base models and fine-tuning options, including selected language, reasoning, tool-calling, and vision-language models
- Token-based billing for training and validation
- Deployment of a fine-tuned model to a dedicated inference endpoint through an additional serving workflow
Separate from these managed tuning methods, custom reinforcement-learning training was announced in closed beta in July 2026. This is a distinct training offering from Together AI's supervised and preference-tuning options.
Managed Model Customization
The managed fine-tuning workflow lets developers retrieve the trained model name and explicitly provision a serving endpoint. Hosting charges are independent of fine-tuning charges, reflecting the separate training and deployment services.
Best For: Teams seeking managed fine-tuning APIs for supported LLMs and multimodal models, with separately configured model-serving endpoints.
Why Modal Stands Out for Serverless GPU Training
High-Performance Distributed Training
Modal combines RDMA-capable Clusters, coordinated node scheduling, and integrated GPU orchestration. Its supported configurations offer RDMA fabric bandwidth of up to 6.4 Tbps per node, as specified in the October 2026 Clusters announcement. The @modal.clustered decorator and rdma=True parameter bring multi-node placement and networking into a concise Python training workflow. Modal's launch materials also describe real-world deployment in 1X's model-training pipeline and Decagon's post-training workloads.
Fast Cold Starts and Efficient Initialization
Modal is engineered for fast cold starts and faster development feedback loops. For Functions, its optimized container filesystem makes files available as needed, helping large Images start without a lengthy full-image transfer. Reusable Images, preinstalled dependencies, and cold-start optimizations streamline repeated development cycles.
For compatible deployed Functions, Modal also offers opt-in GPU Memory Snapshots (Alpha) that preserve initialized GPU state for reuse. This capability is especially relevant to initialization-heavy inference. Model training additionally involves dataset preparation and training checkpoint restoration, so container initialization and time to the first training step are distinct measurements. Persistent Volumes provide durable storage for application-managed training checkpoints.
Code-First Infrastructure
Modal uses code to describe training environments, GPU resources, autoscaling, and Cluster execution. The Python SDK defines training Functions, while its JavaScript, TypeScript, and Go SDKs let other applications interact with Modal resources. The platform supports:
- Custom container Images for reproducible training dependencies
- GPU selection for a range of model sizes and training methods
- Autoscaling controls for elastic workload execution
- Persistent Volumes for checkpoints and shared artifacts
- Managed multi-node Clusters for distributed GPU workloads
For adjacent execution-heavy AI applications, Modal Sandboxes can run untrusted code in isolated environments regardless of programming language. This is a distinct execution interface from training Functions, and it extends Modal's infrastructure to coding agents and automated development tools.
Unified Platform for the AI Lifecycle
Modal supports training, inference, batch workloads, and Sandboxes across CPU and GPU compute. Training pipelines can prepare data, launch experiments, save artifacts, and connect to inference services through a consistent infrastructure environment.
That reach also includes secure execution for AI coding agents. In a Modal customer case study, Ramp describes background coding agents built on Modal Sandboxes. Sandboxes provide isolated execution for generated code, while training Functions and Clusters provide the GPU computation used in model development.
Enterprise Security and Compliance
Modal has completed a SOC 2 Type II audit. The platform uses gVisor-based isolation, with documented encryption in transit and at rest. Modal supports HIPAA-compliant workloads on Enterprise plans via a BAA. These capabilities support an integrated security approach for enterprise AI workloads.
Per-Second GPU Economics
Modal meters requested compute by the second and supports automatic scale-to-zero for Functions. This provides a flexible billing model for periodic retraining, short experiments, and large training jobs that use multiple coordinated GPU nodes. GPU, CPU, and memory resources are metered according to the compute requested or consumed, with platform storage and applicable network usage handled under their respective billing terms.
Modal is the strongest overall choice in this guide for teams that value custom training code, high-bandwidth distributed GPU infrastructure, fast iteration, and integration with inference and batch execution. The Cluster SDK reference describes the current @modal.clustered interface, while the Modal Clusters launch explains its PyTorch and NCCL integration.

