Training data-analyst agents through reinforcement learning demands infrastructure that can handle parallel rollouts, stateful environments, and compute resources across CPUs and GPUs for policy inference and gradient updates. The challenge extends beyond simple code execution: you need isolated environments that reset reliably, scale to the level of parallelism your rollout batch size and episode length actually require, and connect cleanly to the surrounding training stack. Selecting the right AI infrastructure can materially affect training throughput, reliability, cost, and scalability, alongside model quality, environment design, verifier quality, and reward design.
One clarification before the list. The seven entries below do not all sit at the same layer of the stack, and treating them as interchangeable "sandboxes" is the most common source of confusion in this space. A sandbox execution substrate supplies isolated compute. An RL environment defines observations, actions, state, reset semantics, and often verifier behavior. An environment-generation system creates environments and tasks. A training recipe or orchestration framework trains or coordinates policies. A protocol layer standardizes communication between them. A reference implementation shows how the pieces fit together. Each entry below is labeled with the layer it actually occupies, so you can tell what you are choosing between. This guide examines seven sandboxes, environments, and frameworks suited for data-analyst agent reinforcement learning in 2026, starting with Modal, a scalable, high-performance sandbox platform for executing AI-generated code at production scale. Modal pairs high rollout concurrency, automatic CPU and memory bursting, and consistent sub-second starts under load with first-party training and inference infrastructure on one stack. GPU-enabled Sandboxes are available when an environment needs them.
Key Takeaways
- Production infrastructure should scale with the training loop: Modal supports up to 1M concurrent Sandboxes, with sub-second starts that stay consistent under load. Its V2 backend, now the default in current Modal SDK releases, has demonstrated one million concurrent Sandboxes. In that run, Modal created 1M+ sandboxes in under a minute, at more than 10,000 creations per second. Modal reports running 3 billion sandboxes as of October 2026
- CPU and memory bursting reduces cost and overprovisioning for analysis rollouts: A data-analyst rollout sits mostly idle while the policy model drafts its next query or code cell, then spikes when the sandbox loads a large file into memory, joins tables, or runs an aggregation. Fixed-reservation pricing pushes teams to size every environment for that spike, so utilization stays low and costs run high relative to the work the rollouts perform. Modal Sandboxes burst above their requested CPU and memory automatically, so teams can provision for typical usage rather than peak usage and reduce the risk of out-of-memory failures on oversized dataframes. Billing is per second on the greater of requested and actual resources
- Isolation matters when agents run generated analysis code: Modal offers VM sandboxes and gVisor sandboxes, and VM sandboxes support full Docker, so agent-written queries and scripts stay contained within each rollout
- GPU placement matters more than GPU presence: RL training generally requires GPUs for policy inference and gradient updates, which Modal's dedicated training and inference products handle. Modal Sandboxes run across CPU and GPU compute, and GPU-enabled sandboxes are additionally useful when the agent's environment-side workload itself needs acceleration, such as GPU data processing, simulation, or in-environment model execution
- Environment synthesis can alleviate the training data bottleneck, not solve it: Agent World Model generates 1,000 executable environments automatically, while its own error analysis reports that one evaluator judged 74% of generated environments to contain at least one bug, which converts scarcity into a quality-control problem
- Standardized protocols reduce integration work: OpenEnv's Gymnasium-inspired API lets compliant trainers and compliant environments interoperate without bespoke glue code, though the project still labels itself experimental
- Purpose-built training recipes report domain-specific results: DataMind-14B reports a 71.16% average across the authors' selected data-analysis benchmarks, and its domain-specific approach is a plausible contributor even though the evaluation does not isolate it as the sole cause
1. Modal (sandbox execution substrate)
Modal is a scalable, high-performance sandbox platform for executing AI-generated code at production scale, including the rollout environments behind data-analyst agent reinforcement learning. The platform provides dynamically defined containers with consistent sub-second starts under load, automatic CPU and memory bursting, and support for up to 1M concurrent Sandboxes, with GPU-enabled Sandbox execution when an environment needs it. Modal's V2 Sandbox backend, now the default in current Modal SDK releases, is built for extreme concurrency and has demonstrated one million concurrent Sandboxes created in under a minute.
Key Capabilities
- High concurrency: Modal has demonstrated creating 1M+ sandboxes in under a minute, at more than 10,000 creations per second. It supports up to 1M concurrent Sandboxes without you operating your own fixed container cluster. A custom-built scheduler keeps starts consistent as rollout batches grow, and Modal regularly runs tens of thousands of concurrent customer Sandboxes. Account-level concurrency scales with plan, from Starter and Team through custom capacity on Enterprise, as listed in current pricing. The V2 backend behind this throughput is the default for new Sandboxes, and Sandboxes that request GPUs are created on the previous backend automatically
- Automatic CPU and memory bursting: Teams set a baseline CPU and memory request, and Sandboxes burst above it when a rollout loads a large dataset or runs a heavy query. Billing is per second, based on the greater of requested and actual resources, so you can tune requested CPU and memory to match the typical shape of your rollout workload rather than its peak. This active-resourcing model differs from the fixed-reservation billing most Sandbox providers use
- Consistent fast starts under load: Sub-second starts stay consistent at high concurrency, which keeps feedback loops short. An optimized filesystem helps containers come online quickly without letting large images slow startup down. Filesystem, directory, and memory snapshots further reduce repeated setup work between rollouts, and directory snapshots let you capture only part of a Sandbox, such as separating project files from platform-owned dependencies
- CPU and GPU-enabled Sandboxes: Sandboxes run across CPU and GPU compute, and when an environment needs acceleration you can attach H100, H200, B200, B300, A100, and other accelerators from 10+ GPU types to isolated Sandboxes without reservations. Sandbox resource documentation confirms GPU Sandbox support, and GPU Sandboxes run on the gVisor runtime
- Unified platform: Training, inference, and Sandbox execution run on the same core platform, reducing the overhead of coordinating multiple vendors
- Code-first SDK: Define environments programmatically, with configuration expressed as code. Modal provides code-first SDKs in Python, TypeScript, and Go, and code running inside a Sandbox is not limited to one programming language: a Sandbox can run whatever runtime or language the workload requires
Infrastructure Architecture
Modal built its own container runtime, scheduler, filesystem, and container image builder, and describes its core platform as cloud infrastructure designed specifically for AI workloads. That investment is what makes the combination of consistent fast starts under load, high Sandbox concurrency, and first-party training infrastructure possible on one stack. On security, Modal isolates compute in VM sandboxes and gVisor sandboxes, and VM sandboxes support full Docker. Modal's security documentation states that Modal has successfully completed a SOC 2 Type 2 audit, that public APIs use TLS 1.3, and that user data is encrypted in transit and at rest. Modal supports HIPAA-compliant workloads on Enterprise plans via a BAA. Modal supports region-pinning.
Documented Results
Modal has demonstrated creating 1M+ sandboxes in under a minute and running over a million concurrent sandboxes. It reports running 3 billion sandboxes as of October 2026. For data-analyst agent RL, that maps onto collecting rollouts across large numbers of environments concurrently. Applied Compute runs reinforcement learning at scale on Modal. Ramp is a production, agent-specific example: it runs background coding agents on Modal Sandboxes, and those agents generate code changes and write them back into commits or pull requests, a pattern documented in Modal's write-up on Ramp's background coding agent. The same loop of generate, execute, inspect, and iterate is what a data-analyst agent performs during RL rollouts.
Why Teams Choose Modal for Data-Analyst Agent RL
- Rollout batches that widen without re-architecting: Modal has created 1M+ sandboxes in under a minute and run over a million of them concurrently, so a data-analyst RL run can grow each training step from a pilot batch to thousands of SQL and notebook environments while starts stay consistent
- Active-resource pricing for dataframe spikes: Analyst rollouts wait while the policy model writes the next query, then spike when a wide table loads into memory or a join materializes, so teams request modest sandboxes and let them burst instead of reserving memory for the largest dataset in the task set
- Data-stack images that start quickly: Custom images that bundle data libraries, database drivers, plotting tools, and task files still start in under a second at high concurrency
- Training and serving beside the environments: Modal's training and inference products run on the same platform as its Sandboxes, so policy updates, rollout generation, and environment execution share one stack
- GPU when an environment calls for it: GPU Sandboxes on 10+ GPU types are available without reservations for environment-side acceleration, while CPU Sandboxes handle the bulk of SQL and notebook rollouts
2. DataMind (data-synthesis and agent-training recipe)
DataMind is an open-source data-synthesis and agent-training recipe focused on data-analytic agents, presented at ICLR 2026 as a scalable approach to post-training data scaling and multi-turn RL. Developed by researchers affiliated with Zhejiang University and Alibaba Group, it targets agents that work across heterogeneous data formats including CSV files, SQLite databases, and Excel spreadsheets. It is not a sandbox product, and it runs on top of whatever execution infrastructure you provide.
Capabilities
- Combined training pipeline: DataMind supports both supervised fine-tuning (SFT) and reinforcement learning stages in a unified workflow.
- Multi-turn analysis via replayed code context: Rather than depending on a persistent live global-variable pool, DataMind preserves prior code chunks and concatenates them to reproduce the effect of global execution while maintaining isolation.
- Domain-specific dataset: The paper reports 11,707 filtered multi-turn trajectories across diverse data-analysis tasks and 18 task categories, and the current Hugging Face dataset lists 12,187 rows.
- Reported benchmark results: DataMind-14B reports a 71.16% average across the authors' selected data-analysis benchmarks, outperforming the compared general-purpose and proprietary baselines in that evaluation. The domain-specific training approach is a plausible contributor to that result, though the evaluation does not isolate it experimentally as the sole cause.
- Venues: The project repository distinguishes three separate projects and venues: DataMind was accepted at ICLR 2026, DataMind-Analysis at AAAI 2026, and DataPRM at KDD 2026.
- Licensing: Licensing differs by artifact, with the DataMind-7B and DataMind-14B model repositories under Apache-2.0 and DataMind-12K currently published under MIT.
Considerations
- Related process-level reward work: DataPRM provides step-level verification for multi-step analysis tasks, and it is a separate project in the same research line, not an intrinsic component of the ICLR 2026 DataMind recipe.
- Compute requirements: The code can be deployed on compatible compute infrastructure subject to its software dependencies and hardware requirements, and the reported training setup itself used substantial GPU resources. Modal provides training infrastructure spanning CPU and GPU compute, supports common frameworks including PyTorch and Hugging Face TRL, and lets you define the training environment as code in Python, TypeScript, or Go.
Why Teams Choose DataMind
DataMind is an open-source training recipe for data-analytic agents rather than an execution platform. Teams choose DataMind when they want a published SFT-plus-RL methodology, released models, and a trajectory dataset to reproduce or extend on compute they select separately.
3. Agent World Model (environment-generation system)
Agent World Model (AWM), developed by researchers associated with Snowflake AI Research and UNC-Chapel Hill, addresses a major bottleneck in agent RL: the scarcity of diverse training environments. AWM automatically synthesizes executable environments backed by SQLite databases, generating tasks at a scale that would be costly and labor-intensive to build manually. It generates environments; it does not supply the isolated compute they run on.
Capabilities
- Automatic synthesis: AWM creates 1,000 executable SQL-backed environments from seed specifications.
- Broad tool coverage: AWM generates 35,062 tools across 10,000 tasks with MCP interface compatibility, where MCP is the protocol and interface layer that describes how tools are exposed, distinct from the execution and isolation layer that a sandbox provides.
- Database-backed state: AWM uses actual SQLite database instances whose schemas and sample data are synthetically generated, which provides explicit, persistent state transitions that can be checked programmatically rather than relying on purely LLM-simulated world state.
- End-to-end pipeline: The system automatically generates scenarios, tasks, database schemas, interfaces, and verifiers.
- Reported experiments: The authors' RL experiments used 1,024 parallel environment instances per training step, an experimental configuration rather than a platform guarantee. The Arctic-AWM models (4B/8B/14B) were trained on a 526-environment, 3,315-task subset of the synthesized corpus, selected because of compute budget, and show improved out-of-distribution performance in the authors' evaluations, evidence consistent with generalization beyond the training tasks.
- Research status: AWM reports an ICML 2026 acceptance, and its repository records that its infrastructure was merged into OpenEnv in May 2026. OpenEnv has since moved to
huggingface/OpenEnvunder broader multi-stakeholder governance, so as of August 2026 that integration sits inside a community-governed project rather than a Meta or PyTorch-only ecosystem.
Considerations
- Verification quality: Verification itself still uses code and/or LLM judging, and the paper discusses imperfections in environment and verifier signals, including an error analysis in which one evaluator judged 74% of generated environments to contain at least one bug.
- Research context: AWM is a recent large-scale approach to synthetic executable-environment generation for agentic RL, alongside other 2026 environment-generation research.
Why Teams Choose Agent World Model
AWM is an environment-generation system rather than a sandbox runtime. Teams choose it when environment scarcity is the bottleneck and they want large batches of SQL-backed tasks produced automatically, with time set aside for checking the generated environments.
4. OpenEnv (protocol and interoperability layer)
OpenEnv provides a standardized protocol for RL environments, functioning as the connective tissue between sandboxes, environments, and training systems. It was initially launched through collaboration between Meta/PyTorch and Hugging Face in 2025, and as of 2026 it is coordinated under a broader community governance model. It is an emerging open protocol and framework intended to standardize interfaces between agent environments and training systems.
Capabilities
- Gymnasium-inspired API: OpenEnv offers a familiar reset/step/state interface adapted for network-based environments.
- Multiple providers: The current README documents provider options including Docker, Docker Swarm, UV, Daytona, and Azure Container Apps Sandbox, with Kubernetes support described as planned. Some individual environments use their own backends: the Jupyter environment runs on E2B, which is environment-specific rather than a generic provider backend.
- Environment Hub: OpenEnv provides centralized discovery for community-built environments.
- Environment catalog: OpenEnv's current catalog lists roughly 40 environments spanning coding (REPL, Jupyter, Git), games (Atari, Chess), finance (FinQA, FinRL), and web (BrowserGym) domains.
- Ecosystem integration: Current OpenEnv material documents integrations and examples for TRL, SkyRL, and TorchForge. The 2025 launch post also described integration work with verl, which is best treated as historical context rather than a documented current integration.
Considerations
- Experimental status: The project's own documentation describes it as experimental, so its API surface continues to evolve.
- Runtime parity: OpenEnv can provide a common interface across compliant environments and trainers, subject to environment and runtime parity.
- FinQA scope note: The current FinQA environment documentation designates the FinQA dataset as evaluation only and indicates it is not intended for training.
Why Teams Choose OpenEnv
OpenEnv is a protocol and interoperability layer, not an execution backend. Teams choose OpenEnv when they want their analyst environments and trainers to communicate through one reset/step interface and can accept an API that is still evolving.
5. FinQA RL with OpenEnv and Daytona (reference implementation)
This entry is a Daytona implementation guide, not a sandbox product. It documents end-to-end RL training for financial question-answering agents using OpenEnv's FinQA environment with Daytona sandboxes, and remains a detailed public reference implementation for data-analyst agent RL.
Capabilities
- 500-sandbox GRPO run: The guide uses Qwen3-14B with LoRA adapters across 500 concurrent sandboxes, with four GPUs serving vLLM generation and the trainer while the rollout sandboxes execute the SQL environment separately.
- Multi-turn tool-calling: Agents interact with SQL databases to answer questions about SEC 10-K filing data.
- Documented accuracy improvement: The guide reports performance rising from roughly 21% to 52% over 10 iterations, a documented experimental result.
- Technical patterns: The guide covers operational concerns for running RL at scale: sandbox pooling, stale WebSocket cleanup, staggered launches, lag-1 pipeline overlap for GPU efficiency, and batched vLLM generation.
- Warm pools: Warm pools of pre-started sandboxes are a common latency-optimization pattern generally, because they let upfront work such as starting the sandbox, launching a server, pulling a repo, or installing dependencies happen before the end user is waiting.
- Learning outcomes: Analysis of the trained model shows improvements in SQL quoting conventions, numeric formatting, and error recovery.
Considerations
- Dataset designation: OpenEnv's current FinQA documentation designates the FinQA dataset as evaluation only rather than as training data.
- Transfer scope: The guide itself notes that the model did not learn a fundamentally new database-navigation strategy, and it documents dataset-specific conventions and failure modes, so it does not establish cross-task or cross-dataset transfer.
Why Teams Choose the FinQA Reference Implementation
This entry is a vendor-published implementation guide rather than a sandbox product. Teams follow it when they want a worked SQL-agent RL example with operational patterns such as sandbox pooling and pipeline overlap, keeping the FinQA dataset designation in mind.
6. Harbor (task and environment orchestration framework)
Harbor is an independent agent-environment framework with an experimental integration in Hugging Face TRL. Its core concepts explicitly separate tasks and environments from container runtimes and cloud sandbox providers, so Harbor orchestrates sandboxes rather than being one.
Capabilities
- Task-suite abstraction: Each Harbor task bundles instruction, environment, and verifier into a self-contained unit.
- TRL integration: A HarborSpec wires a task suite into GRPOTrainer slots via
train_dataset,environment_factory, andreward_funcs. - External agent pattern: Harbor supports in-process vLLM generation for efficient training loops.
- Multi-backend support: Harbor works with Docker, E2B, Daytona, GKE, Modal, and Runloop. Modal publishes a direct example that runs Harbor evals on Modal Sandboxes, using the
harbor[modal]dependency and Modal as the selected Harbor environment. - Swappable backends and harnesses: Harbor's design lets you swap sandbox backends, for example running the same task suite on Modal or on Docker, and experiment with different harnesses (bash, jupyter, terminal_notes) without rewriting training code.
- Data agent tasks: Example task suites include data agent environments designed for analyst workflows.
Considerations
- Single-suite scope: Individual task suites can be configured modularly, and the current TRL documentation states that a HarborSpec represents one task suite plus one harness, with multi-suite training outside the present scope.
- Experimental integration: The integration sits under TRL's experimental surface, and sandbox provisioning is currently sequential.
Why Teams Choose Harbor
Harbor is an orchestration framework that sits above sandbox providers rather than replacing them. Teams choose Harbor when they train with TRL and want task, environment, and verifier definitions they can move between backends, within the current single-task-suite scope.
7. OpenEnv Jupyter (notebook environment, E2B-backed)
OpenEnv Jupyter is an environment implementation rather than a sandbox provider: its current v0.4.1 documentation says each episode creates a fresh E2B Code Interpreter sandbox. It provides a notebook-native surface for data-analyst agent RL, maintaining state across tool calls for multi-step data exploration.
Capabilities
- Stateful sessions: Variables, imports, files, and plots persist across tool calls within an episode, enabling iterative data analysis workflows.
- MCP tool interface: The documented tools are
add_and_execute_code_cell,edit_and_execute_current_cell,execute_shell_command, a notebook-cell summarization tool, andfinal_answer. MCP defines how those tools are exposed to the model, while the isolated execution of the code they run is a separate infrastructure concern. - Tool coverage: The environment exposes five focused tools covering notebook execution, shell interaction, state and context handling, and answer submission.
- Setup/verify hooks: Hooks configure task-specific state and validation logic.
- Web UI included: An interactive notebook interface at the
/webendpoint supports debugging agent behavior. - Notebook workflow alignment: Unlike terminal-based environments, OpenEnv Jupyter matches notebook-oriented exploratory data-analysis workflows, where analysis is built up incrementally: loading data, exploring distributions, creating visualizations, and iterating on transformations.
Considerations
- E2B-backed execution: Each episode runs in a fresh E2B Code Interpreter sandbox, so the environment's execution layer is tied to E2B.
- Workflow scope: Analysts also work in IDEs, SQL clients, BI tools, terminals, and spreadsheets, so notebook alignment is a fit for a specific workflow rather than for the occupation as a whole.
Why Teams Choose OpenEnv Jupyter
OpenEnv Jupyter is a notebook environment built on E2B sandboxes, not a sandbox provider itself. Teams choose it when the behavior they are training is notebook-style exploratory analysis and they want stateful cells, setup hooks, and a debugging UI in one environment.
Why Modal Stands Out for Data-Analyst Agent RL
Rollout Concurrency at Production Scale
Data-analyst agent RL depends on running many environments in parallel so that rollout collection keeps pace with the trainer. Modal supports up to 1M concurrent Sandboxes and regularly runs tens of thousands of concurrent customer Sandboxes. Its V2 Sandbox architecture is designed for extreme concurrency and has demonstrated one million concurrent Sandboxes. All 1M+ of those sandboxes were created in under a minute, at more than 10,000 creations per second. A custom-built scheduler keeps rollout starts consistent as concurrency climbs.
CPU and Memory Bursting for Analysis Rollouts
Data-analyst rollouts are bursty. An environment can sit idle for long stretches while the policy model generates its next SQL query or code cell, then need far more memory when it reads a large file, pivots a wide table, or renders a chart. Fixed-reservation pricing makes teams size every environment for the heaviest dataset in the task set, so most of the reserved capacity goes unused and costs climb relative to the work the rollouts perform. Modal's active-resourcing model works differently: Sandboxes start from the CPU and memory they request and burst above it automatically, so teams provision for typical usage instead of peak usage. Billing is per second on the greater of requested and actual resources, and bursting also lowers the risk of out-of-memory failures that would cut an episode short and waste the rollout.
Infrastructure Built for AI Workloads
Modal's custom-built infrastructure includes a purpose-designed container runtime, scheduler, filesystem, and image builder. That investment shows up as consistent sub-second starts under heavy concurrent load, including with large custom images, and fast environment turnover, which compounds when training requires thousands of environment resets per hour. Filesystem and memory snapshots further reduce repeated setup work, and directory snapshots can be mounted after a Sandbox has started, which supports patterns such as attaching project-specific state to pre-warmed environments.
GPU-Enabled Sandboxes and Training on One Stack
Modal runs Sandboxes, training, and inference on one platform. That maps onto data-analyst agent RL, where you need environment execution and model training coordinated tightly. Policy updates and rollout generation run on Modal's dedicated training and inference products, and Modal's own RL architecture write-up separates GPU-intensive training, GPU-backed inference for rollout generation, and isolated environment containers, noting that high Sandbox concurrency is what keeps the inference GPUs fed. When the environment-side workload itself needs CUDA for accelerated data processing, simulation, or in-environment model execution, Modal supports GPU-backed Sandboxes across 10+ GPU types without reservations. GPU Sandboxes run on the gVisor runtime, and the same Sandbox API runs CPU-only analyst workloads just as readily.
Unified Platform Reduces Vendor Sprawl
Data-analyst agent development requires training infrastructure, inference endpoints, and sandbox environments. Modal delivers all three on one platform, so you can train models, serve inference, and run sandboxed evaluation inside the same infrastructure ecosystem. Sandboxes, Functions, inference endpoints, and training jobs are distinct primitives with clearly defined roles, and keeping them in one ecosystem reduces cross-vendor integration while giving you consistent observability across the pipeline.
Code-First Developer Experience
Modal's SDK lets you define environments programmatically, with configuration expressed as code. Modal provides code-first SDKs in Python, TypeScript, and Go, and Sandboxes are not tied to a single language: code inside a Sandbox can run whatever runtime or language the workload requires. This aligns with how ML engineers prefer to work: infrastructure as code that can be version-controlled, tested, and iterated alongside training scripts.
Enterprise-Ready Security
Modal has successfully completed a SOC 2 Type 2 audit, isolates compute in VM sandboxes and gVisor sandboxes, and supports HIPAA-compliant workloads on Enterprise plans via a BAA. Modal supports region-pinning. Sandboxes also support tunnels and connection tokens, which authenticate access to servers running inside a Sandbox, a useful control when sandbox-backed analysis surfaces are embedded in an application.
For teams building production data-analyst agents, Modal can serve as the scalable Sandbox execution backend beneath agent and evaluation frameworks such as Harbor, with automatic CPU and memory bursting and consistent starts under load. Explore the sandbox documentation to start building high-concurrency rollout environments for your agent training.

