Code reinforcement learning (Code RL) is increasingly used to train AI coding agents by executing model-generated actions or code in isolated environments and deriving rewards from tests, final environment state, or trajectory quality. The infrastructure choice for these sandboxes determines how many rollouts can run in parallel, how efficiently CPU and memory are used, and how consistently environments start under load, which together shape training throughput, cost efficiency, and iteration speed. A purpose-built AI sandbox platform can reduce the infrastructure overhead that often dominates RL rollout cycles. This guide examines seven sandbox environments used for Code RL workloads in 2026, starting with Modal, a scalable, high-performance sandbox platform for executing AI-generated code at production scale.
Key Takeaways
1. Modal Sandboxes
Modal is a scalable, high-performance sandbox platform for executing AI-generated code at production scale, including Code RL rollouts. Modal Sandboxes provide secure environments for running untrusted, model-generated code, which is the core primitive Code RL depends on. Modal reports running 3 billion Sandboxes as of October 2026, up from the one billion Sandboxes it reported at its Series C. Its engineering team has also demonstrated 1M+ Sandboxes created in under a minute on its new Sandbox system.
How Do Modal Sandboxes Work?
Modal's Sandbox infrastructure provides dynamically created containers built from custom images with arbitrary dependencies, which supports the rapid iteration cycles Code RL depends on:
Why Teams Choose Modal for Code RL
Enterprise Security and Compliance
Modal has successfully completed a SOC 2 Type II audit; in its January 2, 2025 announcement, Modal reported that no deviations were found in that audit and that it planned annual renewals. Modal supports HIPAA-compliant workloads on Enterprise plans via a BAA. Audit logs and Okta SSO are available on Enterprise plans, and role-based access control is available on Team and Enterprise plans. Compute isolation comes from VM and gVisor sandboxes, public APIs run over TLS 1.3, and data is encrypted both in transit and at rest. Modal supports region-pinning.
2. Daytona
Daytona offers isolated sandboxes and development environments with a focus on persistence. It publicly announced its realignment from developer environments to AI-agent runtimes on April 17, 2025, after beginning the internal shift in late 2024, and followed with its agent-native cloud launch later that month.
Capabilities
Considerations
Why Teams Choose Daytona
Daytona is a purpose-built sandbox provider with persistence included and quick starts at smaller scale. Code RL teams pick it as a point solution for stateful reward environments when their rollout counts don't need high-concurrency support.
3. E2B
E2B provides purpose-built sandboxes for AI agents, with per-sandbox Firecracker microVM isolation giving each sandbox its own guest kernel, memory, and page cache. The platform documents integrations with agent frameworks including LangChain and LlamaIndex.
Capabilities
Considerations
Why Teams Choose E2B
E2B is a purpose-built sandbox provider with an open-source core and BYOC support, and 1 billion sandboxes have run on the platform in total. Teams choose E2B for Code RL when they want a standalone sandbox point solution and their test-based rewards run on CPU.
4. CoreWeave Sandboxes
CoreWeave announced CoreWeave Sandboxes on May 14, 2026, targeting RL, agent tool use, and model evaluation with Weights & Biases integration. CoreWeave's documentation now lists serverless sandboxes as generally available.
Capabilities
Considerations
Why Teams Choose CoreWeave Sandboxes
CoreWeave Sandboxes is a code-execution service attached to CoreWeave's GPU cloud. Teams choose it for Code RL when their training already runs on CoreWeave and they want sandbox telemetry recorded next to their Weights & Biases experiments.
5. Google Kubernetes Engine Agent Sandbox
GKE Agent Sandbox provides Kubernetes-native isolation for AI agent workloads. It is built on gVisor and supports a pluggable runtime architecture, with Kata Containers available through the open-source integration path.
Capabilities
Considerations
Why Teams Choose GKE Agent Sandbox
GKE Agent Sandbox is a sandboxing layer for Google Kubernetes Engine. Teams choose it for Code RL when their rollout infrastructure already lives on GKE and they want sandboxes governed by the same cluster policies as their other workloads.
6. Azure Container Apps Sandboxes and Dynamic Sessions
Microsoft now offers two related capabilities. Azure Container Apps Sandboxes, previewed in June 2026 and generally available since September 2026, is a first-class hardware-isolated microVM sandbox resource with snapshot and resume, OCI image support, and scale to thousands. Microsoft describes it as the next evolution of Dynamic Sessions and identifies it as the underlying infrastructure for products including GitHub Copilot Cloud Sandbox, Foundry Hosted Agents, and Azure Container Apps Express. Dynamic Sessions remains available as pool-based, short-lived code and custom-container sessions with Hyper-V isolation.
Capabilities
Considerations
Why Teams Choose Azure Container Apps Sandboxes and Dynamic Sessions
Azure Container Apps Sandboxes and Dynamic Sessions are compute types inside Microsoft's Azure Container Apps service. Teams choose them for Code RL when their stack is standardized on Azure and they want rollout execution to stay within Azure's identity and compliance framework.
7. Deno Sandbox
Deno Sandbox provides ephemeral Linux microVMs integrated with Deno Deploy, aimed at short-lived, isolated code execution. The product is currently in a pre-release phase.
Capabilities
Considerations
Why Teams Choose Deno Sandbox
Deno Sandbox is an execution layer within Deno Deploy. Teams choose it for Code RL when episodes finish inside 30 minutes, the environment sits near an existing Deno Deploy application, and the workload doesn't need high concurrency.
Why Modal Stands Out for Code RL
Validated at Scale
Modal reports running 3 billion Sandboxes as of October 2026, up from the one billion Sandboxes it reported at its Series C. It has demonstrated creating 1M+ Sandboxes in under a minute and running over a million concurrent Sandboxes, and it regularly runs tens of thousands of concurrent customer Sandboxes. Its next-generation V2 backend is built for that level of throughput. Modal's public case studies document RL production usage by Applied Compute and Cognition, production coding-agent usage by Ramp, and RL architectures scaling toward one million concurrent environments.
CPU and Memory Bursting
Code RL workloads are uneven. One rollout checks a short function in milliseconds, the next installs dependencies and runs a full test suite, and many sandboxes wait idle while the policy model produces its next action. Under fixed-reservation pricing, teams size every sandbox for the heaviest episode, so most of the reserved capacity goes unused and costs rise relative to the reward computation performed. Modal Sandboxes start from a configured CPU and memory baseline and burst above it automatically, which lets teams provision for typical usage instead of peak usage. Modal bills per second on the greater of requested and actual resources, and bursting also reduces the risk of out-of-memory failures that would otherwise discard rollouts.
Consistent Starts Under Rollout Load
Modal delivers sub-second Sandbox starts that stay consistent at high concurrency, including with large, custom images that package compilers, test harnesses, and repository dependencies. Its custom-built scheduler, an optimized filesystem, and techniques such as memory snapshotting (currently in alpha for Sandboxes) keep container startup out of the critical path, which compounds across rollout loops that create and tear down environments continuously.
GPU-Capable Sandboxes When Environments Need Them
Many Code RL rewards run on CPU, but some environments, verifiers, or simulators need a GPU inside the sandbox. For those cases, Modal's standard Sandbox API accepts a GPU reservation across a broad accelerator catalog of 10+ types, including T4, L4, A10, L40S, A100, RTX-PRO-6000, H100, H200, B200, and B300. GPU Sandboxes keep full Sandbox functionality and need no capacity reservations.
Unified ML Infrastructure
Modal's core platform provides training, Sandboxes, batch processing, inference, and storage on shared infrastructure. Shared Volumes let model weights, datasets, and artifacts be mounted across training runs and Sandbox evaluations, reducing cross-provider handoffs and integration work.
Code-First Developer Experience
Modal provides code-first SDKs in Python, TypeScript, and Go, so Sandbox creation, execution, and teardown are expressed directly in application code. Code running inside a Sandbox is not restricted to any one language: the Sandbox executes whatever runtime the workload requires, which matters when RL environments span build tools, package managers, test harnesses, and compiled binaries.
Enterprise Security and Compliance
Modal has completed a SOC 2 Type II audit, and its January 2, 2025 announcement reported no deviations in that audit. VM sandboxes and gVisor sandboxes provide the isolation boundary for untrusted code, public APIs use TLS 1.3, and data is encrypted in transit and at rest. Audit logs and Okta SSO are available on Enterprise plans, RBAC on Team and Enterprise plans, and Modal supports HIPAA-compliant workloads on Enterprise plans via a BAA. Modal supports region-pinning.
Observability
Modal provides per-Sandbox lifecycle visibility, logs, health information, and resource metrics. Its Datadog and OpenTelemetry integrations export audit logs, Function logs, and container metrics, and custom OpenTelemetry metrics and spans are available for customer-defined instrumentation. Teams can build correlated dashboards spanning training metrics and semantic sandbox outcomes through that instrumentation, and logs can also be inspected directly in the Modal dashboard.
For teams building Code RL systems, Modal's combination of rollout-scale concurrency, automatic CPU and memory bursting, consistent starts under load, GPU-capable Sandboxes when environments need them, unified platform primitives, and documented RL customers makes it a strong choice for serious reinforcement learning workloads. Explore the sandbox documentation to get started.
Frequently Asked Questions
Code RL (code reinforcement learning) is a training approach where a model generates code or actions that are executed, with rewards derived from the outcome: whether tests pass, whether the final environment state is correct, or how the trajectory scores. Sandboxes matter because they isolate untrusted, model-generated code from the host and from other tenants. Sandbox provisioning and per-action execution overhead also affect rollout throughput, though the share of total training time is workload- and architecture-dependent.
Modal runs the GPU-heavy parts of an RL pipeline on its dedicated training and inference products, which handle policy updates and reward-model serving. When the environment itself needs a GPU, Modal's standard Sandbox API accepts a GPU reservation across a broad accelerator catalog of 10+ types, including B300, without capacity reservations. CoreWeave documents GPU sandboxes, Daytona documents GPU sandboxes that are ephemeral, and GKE supports GPU node pools subject to runtime and driver compatibility. E2B and Deno currently document CPU-only sandbox configurations. Note that reward-model and policy inference can also be served outside the sandbox, so in-sandbox GPUs are only required when the environment itself needs one.
Platforms use different isolation models. Modal offers VM sandboxes and gVisor sandboxes. gVisor, also used by GKE, implements a user-space application kernel that intercepts system calls, and Modal describes it as giving stronger isolation than most other container runtimes. Firecracker microVMs, used by E2B, give each sandbox its own guest kernel. Kata Containers, used by CoreWeave's serverless CPU sandboxes, use lightweight hardware-virtualized VMs. Hyper-V backs Azure Dynamic Sessions. Daytona uses OCI-compatible Linux sandboxes with a dedicated kernel, filesystem, and network stack by default. These create different trust boundaries and compatibility profiles; the appropriate choice depends on your threat model, configuration, and operational controls rather than a fixed ranking.
Not inherently. MCP is the protocol and interface layer, while sandboxes are the execution and isolation layer. Many MCP servers are lightweight wrappers around APIs, databases, SaaS tools, or file systems, and connector-style servers that proxy APIs, retrieve data, expose SaaS actions, or wrap databases and file systems typically do not need isolated execution environments. Sandboxing becomes important when MCP-enabled systems execute AI-generated code, run shells, launch browsers, manipulate files dynamically, or process untrusted workloads on behalf of models. For those execution-heavy MCP workloads, Modal provides VM and gVisor isolation, high concurrency, automatic CPU and memory bursting, and consistent starts under load.
Modal delivers sub-second Sandbox starts that stay consistent at high concurrency, which is where its speed advantage shows. An optimized filesystem helps containers come online quickly without letting large images slow startup down, and memory snapshotting, currently in alpha for Sandboxes, further shortens the path from request to running environment. In Code RL, where rollout loops create and discard environments continuously, that startup behavior directly affects throughput.
Yes. Modal provides code-first SDKs in Python, TypeScript, and Go, and code inside a Sandbox can run in any language the workload requires. E2B offers integrations with LangChain, LlamaIndex, and CrewAI. CoreWeave provides Weights & Biases-linked RL telemetry. GKE offers a Python sandbox SDK and open-source RL orchestration examples. Azure Dynamic Sessions offers preview platform-managed MCP tooling for Python and shell pools. Integration approaches vary across platforms.
Scalability varies significantly. Modal's July 2026 engineering post reports that its new system created 1M+ Sandboxes in under a minute and runs over a million Sandboxes concurrently, and Modal regularly runs tens of thousands of concurrent customer Sandboxes. E2B documents a per-plan concurrency allowance with self-service add-ons and custom Enterprise limits. Daytona governs concurrency through account tiers and limits or customer-controlled BYOC capacity. GKE advertises per-cluster warm-pool allocation. Deno documents default concurrency limits during its pre-release phase. Higher concurrency shortens wall-clock rollout collection, though the statistical batch can also be accumulated over multiple waves.
Modal charges for active resources. Each Code RL sandbox is created with a CPU and memory request, can burst past that request automatically when a sample compiles or runs heavy tests, and is billed per second on the greater of what it requested and what it actually used. Because that extra headroom is always available, a rollout is less likely to be lost to an out-of-memory error, and teams can size sandboxes for an average episode while still covering the heavy ones. Most sandbox providers bill a fixed reservation and don't let a sandbox exceed it, so RL teams have to request enough for their largest episode on every rollout and pay for capacity that sits idle across most of the run.
