Key Takeaways
- GPU-enabled sandboxes matter when the environment itself needs a GPU: Many code-execution rewards, including builds, package installs, and unit tests, remain CPU-oriented, and policy or reward-model inference can be served separately. GPU-equipped sandboxes are valuable when the environment, verifier, simulator, or in-sandbox reward computation must run on a GPU. Modal supports compute across CPUs and GPUs, and its standard Sandbox API accepts a GPU reservation drawing on a broad accelerator catalog
- Startup latency affects rollout throughput, but the percentage varies: Sandbox provisioning and per-action execution overhead can materially change rollout throughput and aggregate worker-hours. Published rollout-infrastructure research reports differences in cold-start latency and projected worker-hours across platforms, though the share of total training time is workload- and architecture-dependent
- Modal is engineered for fast cold starts: Fast cold starts support faster feedback loops, and an optimized filesystem helps containers come online quickly without letting large images slow startup down
- Isolation models create different trust boundaries, not a single ranking: gVisor implements a user-space application kernel, Firecracker and Kata use hardware-virtualized VMs, and Hyper-V is a hypervisor. Relative suitability depends on threat model, configuration, compatibility needs, and operational controls
- Unified platforms reduce handoffs: Running training, Sandboxes, and inference on one platform with shared Volumes and Secrets reduces cross-provider transfers and integration work
- Concurrency caps limit simultaneous rollout throughput: Lower concurrency may increase collection time or idle time even when the trainer can accumulate the same statistical batch over multiple waves. GRPO and PPO configure samples per prompt and global batch size independently of sandbox concurrency
1. Modal Sandboxes
Modal Sandboxes provide secure, scalable environments for running untrusted, model-generated code, which is the core primitive Code RL depends on. More than one billion Sandboxes have been launched on Modal, and Modal's engineering team has demonstrated one million concurrent Sandboxes created in under a minute on its new Sandbox system.
How Do Modal Sandboxes Work?
Modal's Sandbox infrastructure provides dynamically created containers built from custom images with arbitrary dependencies, which supports the rapid iteration cycles Code RL depends on:
- GPU-Capable Execution: The standard Sandbox API accepts a GPU reservation, and Modal's GPU guide documents T4, L4, A10, L40S, A100 (including 40GB and 80GB variants), RTX-PRO-6000, H100, H200, and B200 as selectable types, plus a
B200+ opt-in that lets Modal run the request on either a B200 or a B300 for access to a larger capacity pool - Fast Cold Starts: Modal is engineered for fast cold starts and faster feedback loops, with an optimized filesystem that helps containers come online quickly without letting large images slow startup down
- Code-First SDKs and All-Language Workloads: Modal's code-defined infrastructure is code-first, with Python, TypeScript, and Go client libraries for Sandbox operations. Code running inside a Sandbox is not limited to one programming language: the Sandbox runs whatever runtime or language the workload requires
- gVisor Isolation: Sandboxes are built on gVisor, which Modal describes as giving stronger isolation than most other container runtimes, and Sandboxes are not authorized to access other resources in your Modal workspace
- Snapshotting: Filesystem and Directory Snapshots preserve filesystem state for reproducible environments and default to a 30-day TTL. Directory Snapshots can capture a specific part of a Sandbox, such as separating user project files from platform-owned dependencies, and can be mounted after a Sandbox has started, which allows project-specific state to be attached to pre-warmed Sandboxes. Memory Snapshots capture memory, processes, and filesystem together and are in Alpha
- Network Controls: Sandboxes cannot accept inbound connections or reach Modal workspace resources by default. Outbound connections to public IP addresses are allowed by default and can be dropped entirely with
block_network=True - Server Exposure and Connection Tokens: A server running inside a Sandbox can be exposed through a URL using Sandbox tunnels, and connection tokens authenticate access to that server, which is useful when embedding Sandbox-backed previews in an application
- Filesystem APIs: A Filesystem API supports reading, writing, copying, listing, inspecting, and managing files in running Sandboxes, alongside separate per-Sandbox logs and metrics
What Makes Modal Ideal for Code RL?
- Very High Concurrency: Modal's July 2026 engineering post reports that its new Sandbox system can run millions of Sandboxes concurrently and demonstrates one million created in under a minute. The V2 Sandbox backend is recommended for more than 20 Sandbox creations per second or more than 10,000 concurrent Sandboxes, while the standard backend covers GPU reservations and Memory Snapshots
- Unified Platform: Training, Sandboxes, batch processing, inference, and storage run on the same core platform, and Modal Volumes can be mounted across Sandbox runs and other workloads so weights, datasets, and artifacts are shared rather than shipped between providers
- Fast Cold Starts: Modal is engineered for fast cold starts and faster feedback loops, with an optimized filesystem that helps containers come online quickly without letting large images slow startup down, which compounds across the many short-lived environments a rollout loop creates
- Documented RL and Agent Customers: Modal documents production RL use by Applied Compute, which runs RL environments thousands of times in parallel, and by Cognition, which uses Modal for RL infrastructure and millions of sandboxes. Ramp built a full-context background coding agent on Modal, and Modal also describes RL architectures scaling toward one million concurrent environments
Enterprise Security and Compliance
Modal has successfully completed a SOC 2 Type II audit; in its January 2, 2025 announcement, Modal reported that no deviations were found in that audit and that it planned annual renewals. Modal supports HIPAA-compliant workloads on Enterprise plans via a BAA. Audit logs and Okta SSO are available on Enterprise plans, and role-based access control is available on Team and Enterprise plans.
Best For: Teams needing GPU-capable RL environments on a standard Sandbox backend, organizations running unified ML pipelines from training through deployment, and enterprises that require a completed SOC 2 Type II audit for code execution infrastructure.
2. Daytona
Daytona publicly announced its realignment from developer environments to AI-agent runtimes on April 17, 2025, after beginning the internal shift in late 2024, and followed with its agent-native cloud launch later that month. The platform emphasizes sandbox startup and lifecycle management for high-churn rollouts.
Key Features
- Startup Latency: Daytona publishes startup latency figures for code execution. Earlier Daytona material separately described a narrower infrastructure startup component, which is a different measurement boundary and is not directly comparable
- Unlimited Persistence: Daytona advertises unlimited persistence of sandbox state, which is distinct from unlimited concurrent creation
- Sandbox Architecture: Daytona uses OCI/Docker-compatible Linux sandboxes by default and describes each sandbox as having a dedicated kernel, filesystem, and network stack, with additional VM, Windows, and GPU sandbox types available
- Bring Your Own Compute: Customers can operate runner infrastructure connected to Daytona's managed control plane. Daytona announced on June 11, 2026 that its production codebase was moving to closed source
- Multi-Language SDKs: Python, TypeScript, Go, Java, and Ruby client libraries
RL-Specific Capabilities
Daytona's architecture targets the high-churn pattern typical in Code RL, where many short-lived sandboxes execute model-generated code for reward computation. In a Daytona-published SWE-agent-style benchmark using small CPU environments, Daytona reported per-trajectory execution times and aggregate worker-hour totals for a large trajectory set. These are vendor-published figures and are workload-, resource-, region-, and methodology-specific.
Considerations
Managed-service concurrency and creation rates are governed by account tiers and resource limits, while BYOC capacity is controlled by the customer's own infrastructure. Daytona documents GPU sandboxes and configurable GPU types, and plan-level availability varies. Daytona also publishes RL infrastructure benchmarks and research guidance, and evaluation is best anchored to specific capabilities such as trainer integrations, rollout schedulers, trajectory stores, or reward APIs.
Best For: Teams focused on high-churn CPU rollouts, organizations that want to run runner capacity on their own infrastructure through BYOC, and developers requiring multi-language SDK support.
3. E2B
E2B provides purpose-built sandboxes for AI agents, with per-sandbox Firecracker microVM isolation giving each sandbox its own guest kernel, memory, and page cache. The platform offers native integrations with popular agent frameworks including LangChain and LlamaIndex.
Core Capabilities
- Firecracker MicroVMs: Per-sandbox microVMs with a dedicated guest kernel for untrusted code
- Agent-First SDKs: Native Python and TypeScript libraries designed for agent workflows
- Pause/Resume/Snapshot: Pausing preserves full filesystem and memory state, and paused sandboxes have no automatic time-to-live
- Framework Integrations: Support for LangChain, LlamaIndex, CrewAI, and OpenAI Agents SDK
- Template Loading: E2B documents loading a saved sandbox template; end-to-end application readiness varies with template and workload
Sandbox Lifecycle Management
E2B's pause/resume functionality handles multi-step agent workflows where sandbox state must persist between LLM inference calls. Pro sandboxes can run continuously for up to 24 hours, and longer workflows can use pause/resume, which preserves memory and filesystem state and resets the continuous-runtime window rather than requiring explicit checkpointing.
Considerations
E2B's public sandbox documentation currently describes CPU, memory, and disk configurations, and no generally available GPU sandbox option appears in that documentation. Pro plans include a documented concurrency allowance with self-service add-ons, Enterprise concurrency is custom, and E2B currently markets tens of thousands of concurrent sandboxes for RL, so the Pro figures are not a platform ceiling.
Best For: Teams building LLM agents that want per-sandbox microVM boundaries with a dedicated guest kernel, developers using LangChain or LlamaIndex, and workloads that benefit from full-state pause and resume.
4. CoreWeave Sandboxes
CoreWeave announced CoreWeave Sandboxes on May 14, 2026, targeting RL, agent tool use, and model evaluation with Weights & Biases integration. As of July 22, 2026, the product remains in public preview.
Platform Architecture
- Kata Isolation for Serverless CPU Sandboxes: CoreWeave documents Kata-based isolation for serverless CPU sandboxes and configurable runtime classes for CKS-hosted profiles
- W&B Weave Integration: Sandbox telemetry can be logged alongside training runs for unified RL observability
- Serverless and On-Cluster Modes: CoreWeave offers serverless access and an on-cluster mode that runs the managed Sandboxes execution layer on a customer's CKS capacity, with customers managing capacity, profiles, and namespaces
- GPU-Backed Profiles: CoreWeave supports GPU-backed sandbox profiles, and current documentation includes an H100 example
- Creation and Scaling: CoreWeave describes sandbox creation and scaling for agentic workloads
RL Training Integration
CoreWeave's Weights & Biases integration provides telemetry capture from sandbox executions, enabling dashboards that correlate model training metrics with code execution outcomes. This coupling supports debugging RL failures with correlated training and environment data.
Considerations
CoreWeave Sandboxes is one of the newest offerings in this comparison, announced May 14, 2026, shortly before Google announced GKE Agent Sandbox general availability on May 20, 2026. The product is currently in public preview, and the platform is oriented toward teams already on CoreWeave infrastructure. CKS-based deployments involve cluster capacity management.
Best For: Organizations already on CoreWeave infrastructure, teams standardized on Weights & Biases for experiment tracking, and developers who want tight training and rollout integration within a preview-stage offering.
5. Google Kubernetes Engine Agent Sandbox
GKE Agent Sandbox provides Kubernetes-native isolation for AI agent workloads. It is built on gVisor and supports a pluggable runtime architecture, with Kata Containers available through the open-source integration path.
Technical Architecture
- gVisor-Native with Pluggable Runtimes: gVisor is the native runtime; alternative runtimes come via the open-source project rather than an interchangeable managed toggle
- Kubernetes-Native: Full cluster control with NetworkPolicy, RBAC, and audit logging
- Warm-Pool Allocation: Using an integrated warm pool, Google describes per-cluster sandbox allocation from ready capacity. This is warm-pool allocation rather than unconditional from-zero provisioning
- GCP Integration: Native integration with Google Cloud services, managed identity, and networking
- RL Orchestration Examples: The open-source Agent Sandbox project includes RL and evaluation orchestration examples, including the
agent-sandbox-rl package
Enterprise Control
GKE Agent Sandbox provides Kubernetes governance capabilities, enabling teams to define isolation policies, network boundaries, and resource quotas. The service supports deployment across GKE Autopilot or Standard clusters.
Considerations
GKE Agent Sandbox is designed for teams operating Kubernetes clusters and managing cluster lifecycle. GKE supports GPU node pools, and GPU access from Agent Sandbox workloads depends on compatible node, driver, runtime, and sandbox configuration rather than node pool creation alone. A Python sandbox SDK and open-source RL orchestration examples exist, and teams integrate them with their own trainer and inference stack.
Best For: Enterprises requiring Kubernetes-native infrastructure control, organizations with existing GKE deployments, and teams needing fine-grained isolation policy configuration.
6. Azure Container Apps Sandboxes and Dynamic Sessions
Microsoft now offers two related capabilities. Azure Container Apps Sandboxes, announced in June 2026 and in public preview, is a first-class hardware-isolated microVM sandbox resource with snapshot and resume, OCI image support, cold starts, and scale to thousands. Microsoft describes it as the next evolution of Dynamic Sessions and identifies it as the underlying infrastructure for products including GitHub Copilot Cloud Sandbox, Foundry Hosted Agents, and Azure Container Apps Express. Dynamic Sessions remains available as pool-based, short-lived code and custom-container sessions with Hyper-V isolation.
Core Features
- Hyper-V Isolation: Dynamic Sessions runs each session with Hyper-V isolation; Azure Container Apps Sandboxes provides a dedicated microVM sandbox resource
- Prewarmed Session Pools: A new session is allocated from ready, unallocated capacity. This is warm-pool allocation rather than from-zero VM provisioning
- Azure Identity and Networking: Dynamic Sessions supports Entra authentication, managed identities for callers, and optional network controls. Access to services such as Key Vault can be designed through Azure identity and application integration rather than as a native session feature
- REST API: Language-agnostic API for session orchestration
- MCP Tooling: Dynamic Sessions offers preview platform-managed MCP tooling for predefined Python and shell session pools; standalone Container Apps can host custom MCP servers. MCP is the protocol and interface layer, while sandboxes are the execution and isolation layer, and the two address different concerns
Compliance and Security
Azure maintains SOC 2 attestations, offers a HIPAA BAA for in-scope services, and has FedRAMP-authorized offerings. Compliance coverage depends on the specific service, region, and dependent services in use, along with how the workload is configured.
Considerations
Azure Container Apps supports serverless GPUs for container apps, including A100 and T4 options, while current Dynamic Sessions documentation describes session pools without GPU-enabled configurations, so Container Apps GPU capability is separate from session pools. Azure Container Apps Sandboxes is in public preview. Both are oriented toward Azure-native stacks rather than multi-cloud deployments, and pool-based allocation involves capacity planning for burst workloads.
Best For: Organizations standardized on Azure infrastructure, enterprises requiring Hyper-V or microVM isolation within Azure's compliance framework, and teams building agents aligned with GitHub Copilot execution patterns.
7. Deno Sandbox
Deno Sandbox provides ephemeral Linux microVMs integrated with Deno Deploy, aimed at short-lived, isolated code execution. The product is currently in a pre-release phase.
Platform Characteristics
- Linux MicroVMs: Each sandbox is an isolated Linux microVM rather than a V8 isolate
- Multi-Language Execution: Official SDK examples cover JavaScript/TypeScript and Python, and arbitrary compatible Linux processes, commands, package managers, and binaries can run inside the VM
- 30-Minute Sandbox Lifetime: Individual sandboxes have a documented maximum lifetime of 30 minutes, while persistent regional volumes and snapshots can carry filesystem state across separate sandbox sessions
- Two Documented Regions: Current sandbox execution is available in Amsterdam and Chicago; stable workloads can subsequently be promoted to globally deployed Deno Deploy apps
- Security Model: Deno Sandbox uses hypervisor-isolated microVMs with network egress controls, secret substitution and redaction, filesystem isolation, and audit and observability features. Deno Runtime permissions may additionally constrain Deno programs inside the VM, though they are not the defining isolation mechanism of the sandbox service
Web-Native Design
Deno Sandbox fits web-native agent workloads where code requires secure, ephemeral execution close to an existing Deno Deploy application, with the option to promote stable workloads into globally deployed apps.
Considerations
Current public Deno Sandbox documentation lists CPU-only resource configurations without a GPU option. Default organizational concurrency limits apply during the pre-release phase. Sandbox execution is available in the two documented regions, and the 30-minute per-sandbox lifetime suits episode boundaries aligned with that constraint or the use of persistent volumes.
Best For: Teams running short-duration episodic workloads close to an existing Deno Deploy application, and developers working primarily in JavaScript, TypeScript, or Python.
Why Modal Stands Out for Code RL
GPU-Capable Sandboxes on a High-Scale Platform
Modal supports compute resources across CPUs and GPUs, and its standard Sandbox API accepts a GPU reservation across a broad accelerator catalog including T4, L4, A10, L40S, A100, RTX-PRO-6000, H100, H200, and B200, with a B200+ option that opts into either B200 or B300 capacity. Separately, Modal's next-generation V2 backend targets very high concurrency and is recommended above 10,000 concurrent Sandboxes, so GPU-backed rollouts run on the standard backend while the highest-concurrency CPU rollouts run on V2.
Unified ML Infrastructure
Modal's core platform provides training, Sandboxes, batch processing, inference, and storage on shared infrastructure. Shared Volumes let model weights, datasets, and artifacts be mounted across training runs and Sandbox evaluations, reducing cross-provider handoffs and integration work.
Code-First Developer Experience
Modal supports code-defined infrastructure with SDKs in Python, TypeScript, and Go, so Sandbox creation, execution, and teardown are expressed directly in application code. Code running inside a Sandbox is not restricted to any one language: the Sandbox executes whatever runtime the workload requires, which matters when RL environments span build tools, package managers, test harnesses, and compiled binaries.
Fast Cold Starts
Modal is engineered for fast cold starts and faster feedback loops, with an optimized filesystem that helps containers come online quickly without letting large images slow startup down. Enabling techniques such as memory snapshotting and that optimized filesystem keep container startup out of the critical path, which compounds across rollout loops that create and tear down environments continuously.
Enterprise Security and Compliance
Modal has completed a SOC 2 Type II audit, and its January 2, 2025 announcement reported no deviations in that audit. gVisor provides the isolation boundary for untrusted code. Audit logs and Okta SSO are available on Enterprise plans, RBAC on Team and Enterprise plans, and Modal supports HIPAA-compliant workloads on Enterprise plans via a BAA.
Observability
Modal provides per-Sandbox lifecycle visibility, logs, health information, and resource metrics. Its Datadog and OpenTelemetry integrations export audit logs, Function logs, and container metrics, and custom OpenTelemetry metrics and spans are available for customer-defined instrumentation. Teams can build correlated dashboards spanning training metrics and semantic sandbox outcomes through that instrumentation, and logs can also be inspected directly in the Modal dashboard.
For teams building Code RL systems, Modal's combination of GPU-capable Sandboxes, fast cold starts, unusually high documented Sandbox scale, unified platform primitives, and documented RL customers makes it a strong choice for serious reinforcement learning workloads. Explore the sandbox documentation to get started.
