Key Takeaways
- AI-specialized platforms can simplify GPU workflows: Platforms designed around AI can provide a more purpose-built developer experience, while general-purpose serverless platforms such as Google Cloud Run and Azure Container Apps also provide direct serverless GPU options
- Modal is engineered for fast cold starts: Modal combines a performance-focused container runtime, Memory Snapshots, and an optimized filesystem so containers can come online quickly without letting large images slow startup down
- GPU access varies by runtime: Google Cloud Run and Azure Container Apps provide direct serverless GPU options, while AWS Lambda and Cloudflare Workers commonly pair their core runtimes with separate managed AI services for GPU-backed inference
- Edge runtimes support distributed request handling: Cloudflare Workers uses V8 isolates across a globally distributed network, while Workers AI adds managed model inference
- Security requirements depend on the workload and data: Enterprise security reviews commonly consider SOC 2 Type 2 reports or comparable assurance, while HIPAA-regulated workloads also make BAA availability and documented product scope relevant
- Granular billing can reduce idle-compute costs: Per-second billing and scale-to-zero can improve economics for bursty or intermittent workloads, but total cost still depends on utilization, initialization overhead, transfer charges, discounts, and workload shape
1. Modal: The AI-Native Serverless Platform for Modern SaaS
Modal is an AI infrastructure platform for teams running compute-intensive and agentic workloads. It provides serverless Functions alongside infrastructure for inference, training, batch processing, notebooks, storage, and secure code execution. Its code-first model lets teams define workloads in Python and use Modal’s TypeScript/JavaScript and Go SDKs to work with remote resources.
How Does Modal Work?
Modal packages application workloads into containers and runs them in the cloud with automatic scaling. Modal pools capacity across major clouds and places workloads based on available capacity. When a workload needs an accelerator, developers can select a GPU type or configure an ordered GPU fallback list.
- Define Functions and remote workloads in code, then deploy applications with the Modal CLI.
- Scale CPU workloads elastically and use pooled capacity for GPU-backed workloads.
- Run untrusted or AI-generated code in Modal Sandboxes with gVisor-based isolation.
- Compose multi-step workflows with Functions, Sandboxes, snapshots, persistent Volumes, Queues, Dicts, and retry policies.
Modal's Deep Infrastructure for AI Workloads
Modal built its own filesystem, container runtime, scheduler, and image builder for cloud execution. That foundation supports several workload types through a shared platform rather than requiring separate infrastructure for every AI task.
- Serve models and run inference workloads on CPU or GPU resources.
- Run training and batch-processing jobs alongside online application workloads.
- Use Sandboxes for isolated execution of arbitrary languages and runtimes.
- Build and operate workloads with code-defined infrastructure and platform telemetry integrations.
Documented Results
Modal has described production workloads that use its infrastructure for background coding agents and large numbers of concurrent sandbox sessions.
- Modal reported that roughly half of merged pull requests across Ramp’s frontend and backend repositories were started by its Inspect background coding agent in February 2026. Read the Ramp case study.
- Modal engineering has reported support for 50,000 concurrent Sandboxes per customer, with gVisor-based isolation.
Security and Compliance: SOC 2 Type 2 and HIPAA Workloads
Modal Sandboxes use gVisor-based isolation and support configurable compute, networking, filesystem access, and snapshots. Modal states that it maintains SOC 2 Type II certification. HIPAA-compliant workloads are supported on Enterprise plans through a Business Associate Agreement for customers handling protected health information. Teams remain responsible for how their applications handle data, credentials, and sandbox permissions.
2. Google Cloud Run: Containerized Serverless for Scalable AI Applications
Google Cloud Run runs containerized applications as managed services, jobs, and worker pools within Google Cloud. Its container-based approach suits teams that want to package an AI SaaS component with its runtime and dependencies, then connect it to Google Cloud data and AI services.
Key Features
- Run OCI-compatible container images without converting an application to a provider-specific function format.
- Use request-driven services, jobs, and worker pools for different application execution patterns.
- Configure autoscaling and scale-to-zero for supported service configurations.
- Connect container workloads with Google Cloud services such as Vertex AI, BigQuery, and Cloud Storage.
Container Deployment and Management
Cloud Run’s container model allows teams to package application code, dependencies, and runtime requirements together. Services handle request-facing workloads, while Cloud Run Jobs support tasks that run to completion. Preview options include individually addressable instances and sandboxed execution for supported configurations.
Usage-Based Pricing and Scaling
Cloud Run scales services according to demand and supports scale-to-zero configurations. The available scaling and billing behavior depends on the selected Cloud Run product and configuration, so teams should estimate costs using their expected request volume, concurrency, resource allocation, and job duration.
Microservices Integration
Cloud Run can host individual containerized components that communicate with other Google Cloud services. AI SaaS teams can use this model to separate an API, model-facing service, and background job while integrating with services such as Vertex AI, BigQuery, and Cloud Storage. Cloud Run supports L4 and RTX PRO 6000 GPUs in supported regions and configurations.
3. Azure Container Apps: Managed Serverless for Microservices and AI
Azure Container Apps provides a managed environment for containerized applications and microservices within Azure. It can fit AI SaaS architectures that already use Azure services and need container-based deployment for APIs, workers, or supporting services. Azure Functions can also be hosted on Azure Container Apps for supported configurations.
Scaling and Networking
Container Apps offers a managed way to deploy services without directly operating the underlying container infrastructure. Teams can organize application components as separate containerized services and configure how they communicate and scale. Available scaling behavior, network access, and execution characteristics depend on the app configuration and Azure environment.
- Use containerized services for independently deployed API and worker components.
- Connect application components to Azure data and identity services according to the deployment’s access policies.
- Evaluate Azure Container Apps GPU profiles when an AI service needs supported accelerator compute; documented profiles include A100 and T4 for applicable workloads.
4. AWS Lambda: Event-Driven Serverless Functions for AI Workloads
AWS Lambda runs functions in response to events and is commonly used for API handlers, automation, and application integrations across AWS. For an AI SaaS product, Lambda can execute lightweight request and event-processing logic while calling managed model services or a separately hosted inference backend.
AWS Ecosystem Integration for AI Services
Lambda’s event-driven model can connect application code with AWS services through triggers and service integrations. Teams can use functions to process application events, coordinate steps in an AI workflow, or connect an API layer to a model endpoint. GPU-intensive model serving and training are generally separate workloads from standard Lambda function execution and may use other AWS compute services.
Startup and Pricing Options
Lambda is designed to run code on demand rather than requiring a continuously managed server for every function. Cost and scaling behavior depend on invocation volume, execution duration, memory configuration, and the surrounding AWS services. Teams should account for model inference, storage, networking, and other downstream service costs alongside function execution.
5. Cloudflare Workers: Edge Serverless for Distributed AI Applications
Cloudflare Workers provides distributed serverless compute using V8 isolates across Cloudflare’s network. It supports JavaScript and TypeScript, Python Workers, Rust, and WebAssembly-based workloads, with integrations across Cloudflare’s application, storage, and AI services. This model can suit products whose request handling benefits from a globally distributed runtime.
Managed AI Inference at the Edge
Cloudflare Workers AI provides access to hosted models for text, embeddings, vision, speech, and related AI workloads. Teams can call managed inference from Workers-based application logic rather than operating every model endpoint themselves. Model availability and usage-based pricing vary by workload.
Data and Security Integration
Workers integrates with Cloudflare services including R2, Vectorize, D1, and AI Gateway. These services can support application data, vector search, and AI request management around a Workers-based product. Teams should configure access controls and data handling for each service according to their product’s security requirements.
6. Vercel Functions: Frontend-Focused Serverless for AI-Powered UIs
Vercel Functions provides server-side execution integrated with Vercel deployments and web application workflows. It is a natural option for teams building AI SaaS products around Next.js or another supported web framework, especially when the frontend and backend should share a deployment workflow.
Development for AI-Driven User Interfaces
Vercel’s platform connects preview deployments and production deployments with web application development. Functions can stream responses, including generated model output, and supported frameworks receive platform-aware build and deployment behavior. Vercel’s AI SDK provides tooling for building AI features and agent-style application workflows.
API Endpoints for AI Interactions
Functions can provide regional server-side endpoints for application APIs, tool calls, and streamed AI responses. Vercel Sandbox adds isolated Firecracker microVM environments for executing untrusted or generated code. Teams with self-hosted GPU inference or training typically connect Functions to a separate backend for accelerator-intensive workloads.
7. IBM Cloud Code Engine: Unified Serverless for Diverse AI Workloads
IBM Cloud Code Engine is a managed serverless platform for running cloud-native application workloads in the IBM Cloud ecosystem. It can be evaluated for AI SaaS components that need managed deployment and integration with an organization’s IBM Cloud services. The right fit depends on the application’s runtime, data location, connectivity, and compute requirements.
AI Microservices and Batch Processing
An AI SaaS product can separate request-facing APIs from asynchronous processing and model-related work. When evaluating Code Engine for this architecture, map each component to the execution type and resource profile it requires, and confirm support for the specific runtimes, accelerators, and service integrations the application depends on.
Security and Hybrid Cloud Capabilities
For organizations with existing IBM Cloud or hybrid-cloud environments, evaluate Code Engine against the required identity, network, data residency, and compliance controls. Security assurances and available integrations should be checked against the specific service configuration and contractual requirements for the workload.
Why Modal Stands Out for AI SaaS Infrastructure
The platforms above cover different parts of a serverless architecture. Modal stands out for teams that want to run AI compute, serverless application logic, batch workloads, and isolated code execution through one AI-focused infrastructure platform.
Purpose-Built for AI Workloads
Modal’s platform combines infrastructure for inference, training, batch processing, notebooks, and Sandboxes, rather than focusing only on request-driven functions.
- Run model workloads and application compute on CPU or GPU resources.
- Use the same platform for scheduled or batch work and interactive services.
- Add isolated execution for user-submitted or agent-generated code with Modal Sandboxes.
SDKs Across Python, TypeScript, and Go
Modal provides code-first workflows through Python, TypeScript/JavaScript, and Go SDKs. Teams can define and operate remote workloads programmatically, while Sandboxes can run the language and runtime required by the workload.
Fast Cold Starts and Memory Snapshots
Modal is engineered for fast cold starts and faster feedback loops. Its optimized filesystem helps containers come online without large images slowing startup, and Memory Snapshots can reduce repeated initialization work for suitable workloads.
Native Sandboxes for AI Agents and Code Execution
Modal Sandboxes provide isolated environments for untrusted or AI-generated code, with gVisor-based isolation, configurable compute, filesystem APIs, snapshots, and networking controls. They support arbitrary programming languages and can be used as the execution layer for agents and AI products.
Full AI Lifecycle in One Platform
Modal brings inference, training, batch processing, serverless Functions, Sandboxes, and notebooks together. A team can build product APIs, run model workloads, process background tasks, and execute untrusted code using a common platform and developer workflow.
Proven Scale with Documented Results
Modal has published production examples of large-scale AI and agent workloads.
- Ramp’s Inspect agent is described in a Modal case study covering background coding-agent sessions in full development environments.
- Modal engineering has reported support for 50,000 concurrent Sandboxes per customer.
Enterprise Security and Compliance
Modal documents its security model, maintains SOC 2 Type II certification, and supports HIPAA-compliant workloads on Enterprise plans through a BAA for customers handling protected health information. Modal also provides Enterprise audit logs and supports exporting telemetry to an OpenTelemetry provider.
Explore Modal’s platform and documentation to see how its serverless infrastructure fits your AI workloads.
