All posts
Model Training
Modal Team
15 min read

Best Serverless GPU Platforms for Model Training in 2026

Serverless GPU platforms simplify AI model training by managing compute provisioning and execution infrastructure. They let ML engineers focus on experiments, fine-tuning, distributed training, and evaluation rather than maintaining GPU fleets. The category includes serverless GPU functions, managed custom training jobs, and model-specific training APIs, each with a different execution model.

Modal Sandboxes in production

For teams that want custom training code alongside scalable inference and batch processing, Modal is the leading choice in this guide. Its training infrastructure combines code-defined workloads, on-demand NVIDIA GPUs, multi-node Clusters with RDMA networking, and per-second compute billing. Modal also provides CPU compute and secure Sandboxes for adjacent AI applications, making it a versatile platform beyond GPU training alone.

Key Takeaways

  • Code-first model development: Modal lets developers define training Functions, container Images, GPU resources, and distributed Clusters in Python. Its SDK ecosystem also includes JavaScript and TypeScript and Go clients for working with Modal resources. Sandboxes can execute programs in the languages their workloads require.
  • Distributed training with RDMA: Modal Clusters combine coordinated node scheduling with high-bandwidth communication. The October 2026 Clusters announcement describes RDMA fabric bandwidth of up to 6.4 Tbps per node. An earlier product update documented 3.2 Tbps InfiniBand for multi-node Clusters.
  • Fast cold starts and efficient iteration: Modal Function containers use an optimized filesystem to come online quickly without letting large Images unnecessarily slow startup. GPU Memory Snapshots can streamline initialization for compatible deployed inference Functions. Training startup also includes data and checkpoint preparation, which is distinct from container cold-start time.
  • Usage-based compute: Modal bills compute by the second and supports scale-to-zero Functions, aligning resource usage with experiments and scheduled training jobs. Billing is based on resources used or requested, including intentionally retained warm compute.
  • One platform across AI workflows: Modal combines model training, inference, batch processing, and isolated Sandboxes in an integrated infrastructure platform for CPU and GPU workloads.

1. Modal

Modal provides code-first serverless infrastructure for training, inference, batch computation, and isolated execution. Developers define GPU training Functions and their runtime environments through Python instead of provisioning servers or operating a Kubernetes cluster. Modal's JavaScript and TypeScript SDK and Go SDK provide additional ways to interact with deployed resources, while Modal Sandboxes support execution of code in different programming languages.

On October 1, 2026, Modal Clusters became generally available. The capability extends Modal's on-demand compute model to coordinated, multi-node GPU training and inference. Modal has highlighted production use cases including 1X's world-model pretraining and Decagon's post-training workloads.

How Does Modal Work?

Modal packages code and dependencies into reusable container Images and schedules workloads on available compute infrastructure. Developers can select NVIDIA GPU options such as B300, B200, H200, H100, and A100, alongside CPU resources. Training jobs can run on a single GPU, multiple GPUs on one node, or a coordinated multi-node Cluster.

  • Code-first infrastructure: Python decorators and custom Images define dependencies, hardware, and execution settings. JavaScript, TypeScript, and Go SDK clients integrate Modal resources into other application backends.
  • Fast cold starts: Engineered for fast cold starts and shorter feedback loops, Modal Functions use an optimized filesystem that makes files available as needed. This helps Function containers come online quickly, including when Images contain substantial dependencies.
  • Multi-node GPU Clusters: Modal uses the @modal.clustered decorator to schedule GPU containers together, with cluster size defined in code and bounded by workspace GPU limits. Each clustered container requests all GPUs on its host, and rdma=True enables RDMA networking on supported hardware.
  • GPU flexibility: Teams can match GPU memory and compute characteristics to fine-tuning, pretraining, evaluation, or inference, while using the same platform for CPU-based preparation and orchestration.
  • Per-second compute billing: Modal's billing model meters compute resources used or requested, and autoscaling lets Functions scale with their workload.

Distributed Training Capabilities

The Modal Clusters announcement and current Python SDK reference describe coordinated GPU training with @modal.clustered, including optional RDMA networking for PyTorch and NCCL.

  • RDMA networking: The Modal Clusters announcement describes RDMA fabric bandwidth of up to 6.4 Tbps per node. Modal also documented 3.2 Tbps InfiniBand in an earlier multi-node update. Automated configuration for PyTorch and NCCL supports high-bandwidth GPU communication during distributed training.
  • Coordinated scheduling: Cluster nodes are scheduled as a group and communicate over private networking. The Python rdma=True option enables RDMA placement and configuration on supported GPUs.
  • Framework flexibility: Modal provides PyTorch and NCCL integration, TensorFlow GPU examples, and customizable Images for compatible training frameworks and dependencies.
  • Persistent training artifacts: Modal Volumes, Cloud Bucket Mounts, and configurable execution retries support training code that writes checkpoints and resumes work from saved state.
  • Proven AI training applications: The Modal Clusters launch describes 1X using multi-node B300 Clusters for world-model pretraining and separate GPU jobs for evaluations.

Security and Compliance

Modal has completed a SOC 2 Type II audit. Its infrastructure uses gVisor-based workload isolation and documents encryption for data in transit and at rest. Modal supports HIPAA-compliant workloads on Enterprise plans via a Business Associate Agreement. These security capabilities complement its managed GPU infrastructure for production AI development.

Best For: ML engineers and AI teams seeking code-defined custom training, flexible NVIDIA GPU compute, RDMA-capable multi-node Clusters, and integrated infrastructure for training, inference, batch processing, and secure execution.

2. Google Gemini Enterprise Agent Platform

Google's Gemini Enterprise Agent Platform, renamed from Vertex AI on April 22, 2026, offers managed custom model training within Google Cloud. The service provisions infrastructure for submitted jobs, while teams define the training code, accelerators, worker pools, and data access. It integrates with BigQuery and Cloud Storage.

Key Features

  • Managed multi-worker training jobs with user-defined worker pools, GPU types, and replica counts
  • Cloud Storage and BigQuery data sources for ML workflows
  • PyTorch, TensorFlow, and custom training container support
  • Reduction Server optimization for NVIDIA GPU all-reduce in distributed training
  • Managed training infrastructure with Google Cloud security and governance features
  • Eligible H200, B200, and GB200 training compute configurations

Enterprise Focus

The platform supports organizations using BigQuery and Cloud Storage as part of custom training pipelines. Developers configure worker topology, accelerators, and regional resources for each job, while the managed infrastructure handles provisioning and release of compute. Data services, training jobs, and Google Cloud governance tools can be combined into a broader model development workflow.

Best For: Enterprises using Google Cloud that want managed custom training jobs and integration with BigQuery, Cloud Storage, and Google Cloud governance tools.

3. Azure Machine Learning Serverless Compute

Azure Machine Learning serverless compute provides managed, on-demand training job compute, including GPUs, within the Microsoft Azure ecosystem. It supports model training without maintaining a user-created Azure Machine Learning compute cluster and integrates with Azure identity, governance, and MLOps services.

Core Capabilities

  • On-demand serverless compute for training jobs, including command, AutoML, sweep, parallel, and pipeline workloads
  • PyTorch, TensorFlow, and MPI-based distributed training support
  • Microsoft Entra ID workspace authentication and user-assigned managed identities for serverless jobs
  • Managed network isolation and regional resource placement options
  • MLflow and Azure DevOps integrations within Microsoft's broader ML tooling

Compliance and Governance

Azure supports enterprise identity, policy integration, auditing, and networking controls. The cloud security shared-responsibility model and Azure Policy's SOC 2 control mappings describe how cloud platform services and customer configuration participate in enterprise governance. These capabilities fit regulated AI workflows within the scope of an organization's requirements.

Best For: Organizations building Azure-native GPU training jobs with enterprise identity, experiment tracking, governance tools, and Microsoft MLOps integrations.

4. Beam Cloud

Beam Cloud offers a managed, Python-oriented serverless GPU service associated with its open-source Beta9 runtime. Beam is Y Combinator-backed and provides GPU functions, scheduled tasks, and persistent storage for training and batch workflows. Its managed service and the independently deployable runtime are distinct options.

Key Features

  • Open-source Beta9 runtime and self-hosting option
  • Python SDK for defining GPU functions
  • Built-in cron scheduling for periodic GPU jobs, using UTC scheduling
  • Persistent volumes for datasets and checkpoints
  • Usage-based GPU execution billing, with compute resource selection determined by each workload

Developer Experience

Beam's scheduled GPU jobs support periodic retraining and batch experimentation, while volumes provide storage for datasets and model artifacts. Beam's published cost example describes resource-based billing across serverless GPU workloads. The platform provides a Python interface for teams developing custom training tasks.

Best For: Teams running Python-based GPU fine-tuning, scheduled retraining, and checkpointed batch experiments, including users interested in the Beta9 runtime.

5. Fal.ai

Fal.ai provides managed training APIs for selected generative image and video models. Its serverless inference product and dedicated fal Compute training machines are separate offerings alongside model-specific LoRA trainer APIs.

Specialized Capabilities

  • Wan 2.2 text-to-video and image-to-video LoRA trainer endpoints
  • Optional dedicated fal Compute instances with 8 H100 SXM GPUs as a separate machine-based training option
  • Model-specific training support for generative image and video models
  • Wan 2.2 trainer constraints and a minimum charge of 100 training steps for the documented trainer endpoints
  • Supported Wan 2.2 workflows include a returned .safetensors LoRA artifact for compatible inference
  • API-driven dataset submission, training status, and artifact retrieval

Media Model Focus

The Wan 2.2 guide describes dataset submission, LoRA training, and subsequent use of the adapter with a compatible inference model. The separately billed fal Compute product provides another execution model for custom training programs. These services are oriented toward different training formats, including structured trainer APIs and dedicated GPU machines.

Best For: Teams fine-tuning supported image and video model families through specialized LoRA trainer APIs, with dedicated compute as another option for custom training.

6. Cerebrium AI

Cerebrium AI, a Y Combinator Winter 2022 company, offers serverless GPU infrastructure for custom ML applications, training experiments, and real-time AI services. Cerebrium has announced ISO 27001 compliance and completion of a SOC 2 Type II audit. Its published memory snapshot information principally concerns workload initialization, which is distinct from restoring a model training checkpoint.

Platform Highlights

  • CPU/GPU memory snapshots for compatible workloads
  • Custom Python GPU workloads, including documented PyTorch/CUDA examples
  • SOC 2 Type II audit and ISO 27001 compliance announcement
  • Weights & Biases hyperparameter sweep example for training experiments
  • Support for custom ML pipelines and model applications

MLOps Integration

Cerebrium's W&B sweep example illustrates tracked hyperparameter experimentation with serverless GPU workloads. Its custom Python environments can run ML application code with compatible dependencies, and its snapshot feature offers a distinct form of initialized-state reuse. A dated 2025 walkthrough also describes the W&B sweep example.

Best For: Teams using Python-based GPU training experiments, parallel evaluations, and Weights & Biases experiment tracking alongside AI application deployment.

7. Together AI

Together AI's managed fine-tuning service provides model customization for supported language and multimodal models. Teams upload datasets, select models and methods, and run training without managing underlying GPU machines. Together AI has advertised training support for eligible models up to 1 trillion parameters. A completed fine-tuning job makes a fine-tuned model available; deployment requires a separate action.

Managed Fine-Tuning

  • Supervised fine-tuning and DPO with LoRA or full-model updates
  • Fine-tuning support for 100B+ parameter models in eligible configurations
  • Catalog of supported base models and fine-tuning options, including selected language, reasoning, tool-calling, and vision-language models
  • Token-based billing for training and validation
  • Deployment of a fine-tuned model to a dedicated inference endpoint through an additional serving workflow

Separate from these managed tuning methods, custom reinforcement-learning training was announced in closed beta in July 2026. This is a distinct training offering from Together AI's supervised and preference-tuning options.

Managed Model Customization

The managed fine-tuning workflow lets developers retrieve the trained model name and explicitly provision a serving endpoint. Hosting charges are independent of fine-tuning charges, reflecting the separate training and deployment services.

Best For: Teams seeking managed fine-tuning APIs for supported LLMs and multimodal models, with separately configured model-serving endpoints.

Why Modal Stands Out for Serverless GPU Training

High-Performance Distributed Training

Modal combines RDMA-capable Clusters, coordinated node scheduling, and integrated GPU orchestration. Its supported configurations offer RDMA fabric bandwidth of up to 6.4 Tbps per node, as specified in the October 2026 Clusters announcement. The @modal.clustered decorator and rdma=True parameter bring multi-node placement and networking into a concise Python training workflow. Modal's launch materials also describe real-world deployment in 1X's model-training pipeline and Decagon's post-training workloads.

Fast Cold Starts and Efficient Initialization

Modal is engineered for fast cold starts and faster development feedback loops. For Functions, its optimized container filesystem makes files available as needed, helping large Images start without a lengthy full-image transfer. Reusable Images, preinstalled dependencies, and cold-start optimizations streamline repeated development cycles.

For compatible deployed Functions, Modal also offers opt-in GPU Memory Snapshots (Alpha) that preserve initialized GPU state for reuse. This capability is especially relevant to initialization-heavy inference. Model training additionally involves dataset preparation and training checkpoint restoration, so container initialization and time to the first training step are distinct measurements. Persistent Volumes provide durable storage for application-managed training checkpoints.

Code-First Infrastructure

Modal uses code to describe training environments, GPU resources, autoscaling, and Cluster execution. The Python SDK defines training Functions, while its JavaScript, TypeScript, and Go SDKs let other applications interact with Modal resources. The platform supports:

For adjacent execution-heavy AI applications, Modal Sandboxes can run untrusted code in isolated environments regardless of programming language. This is a distinct execution interface from training Functions, and it extends Modal's infrastructure to coding agents and automated development tools.

Unified Platform for the AI Lifecycle

Modal supports training, inference, batch workloads, and Sandboxes across CPU and GPU compute. Training pipelines can prepare data, launch experiments, save artifacts, and connect to inference services through a consistent infrastructure environment.

That reach also includes secure execution for AI coding agents. In a Modal customer case study, Ramp describes background coding agents built on Modal Sandboxes. Sandboxes provide isolated execution for generated code, while training Functions and Clusters provide the GPU computation used in model development.

Enterprise Security and Compliance

Modal has completed a SOC 2 Type II audit. The platform uses gVisor-based isolation, with documented encryption in transit and at rest. Modal supports HIPAA-compliant workloads on Enterprise plans via a BAA. These capabilities support an integrated security approach for enterprise AI workloads.

Per-Second GPU Economics

Modal meters requested compute by the second and supports automatic scale-to-zero for Functions. This provides a flexible billing model for periodic retraining, short experiments, and large training jobs that use multiple coordinated GPU nodes. GPU, CPU, and memory resources are metered according to the compute requested or consumed, with platform storage and applicable network usage handled under their respective billing terms.

Modal is the strongest overall choice in this guide for teams that value custom training code, high-bandwidth distributed GPU infrastructure, fast iteration, and integration with inference and batch execution. The Cluster SDK reference describes the current @modal.clustered interface, while the Modal Clusters launch explains its PyTorch and NCCL integration.

Frequently Asked Questions

  • Serverless GPU computing lets teams run GPU-backed workloads while the platform handles infrastructure provisioning and job scheduling. Developers retain control over model code, data preparation, training configuration, and checkpoints according to the service's execution model. Resource allocation also follows platform-specific policies, including Azure workspace and subscription quotas and Google Cloud limits. Modal is especially well suited to custom workloads because its GPU Functions and Clusters combine on-demand compute with code-defined infrastructure and per-second billing.

  • A training job's startup involves several stages: compute scheduling, container initialization, dataset access, and any training checkpoint restoration. Cold-start time describes bringing a new execution environment into service, while time to the first training step also includes the other stages of the training workflow. Modal Functions are engineered for fast cold starts using an optimized filesystem, reusable Images, and container startup techniques. Compatible Functions can also use Memory Snapshots to preserve initialized state. Inference cold-start measurements and end-to-end training-startup measurements capture different stages of execution, so inference cold-start examples do not measure a full training job's initialization.

  • Important criteria include custom training-code support, GPU memory, distributed topology, RDMA networking, model and framework compatibility, data location, checkpoint storage, security requirements, and the total cost of training and serving. Modal is a strong choice for custom training loops and RDMA-capable Clusters. Managed trainer APIs, such as those offered by Together AI, suit supported model-specific fine-tuning workflows.

  • Yes. The integration approach depends on whether a service runs custom code or exposes predefined trainer APIs. Modal provides custom Images for training dependencies, PyTorch and NCCL support for distributed training, TensorFlow examples, and a Datadog integration for exporting customer Function logs and metrics. Frameworks and tools such as Weights & Biases, MLflow, and JAX can be part of compatible custom training environments.

  • Modal's GPU documentation lists B300, B200, H200, H100, A100, and other supported accelerator types for its GPU workloads. Google Gemini Enterprise Agent Platform's serverless training compute documentation covers eligible H200, B200, and GB200 configurations. Azure offers GPU virtual-machine families such as GB200, although availability in the broader Azure VM catalog is distinct from availability for a particular Azure ML serverless job. For teams building custom NVIDIA GPU training workloads, Modal offers a particularly cohesive path from single-node experiments to RDMA-enabled distributed training.

View Sandboxes Docs

Run your first sandbox in minutes.

$30 in free compute to get started.