Repo-level coding agents benefit from models that can reason over large portions of a codebase, not just individual files. Long-context open-weight code LLMs (models distributed as downloadable weights, which is a narrower claim than the OSI Open Source AI Definition) have emerged as a foundation for these agents, supporting code generation, bug fixing, and refactoring across complex software projects. Longer context windows can reduce retrieval and chunking requirements for small and medium repositories, though large repositories still require repository indexing, retrieval, context selection, or summarization. Even the most capable model is only as effective as the infrastructure running it. Choosing the right serverless GPU infrastructure determines whether your repo agent can scale from prototype to production without infrastructure headaches. This guide examines seven essential components for building production-ready repo agents in 2026, starting with Modal, the AI infrastructure platform that can run open-weight and custom code models for deployment as production repo agents.
Key Takeaways
- Infrastructure shapes model effectiveness: The most capable code LLM delivers limited value without infrastructure that handles substantial VRAM footprints (from a few GB to hundreds of GB depending on model, precision, and context), fast container startup, and automatic scaling
- MoE architectures reduce per-token compute: Models like Qwen3-Coder-Next reach 70.6% on SWE-bench Verified with SWE-Agent while activating only 3B of 80B parameters per token, which lowers per-token computation, though the full model must still be resident in memory unless weights are offloaded
- Context window size varies significantly: Qwen3-Coder-Next supports 262,144 tokens natively, while the Qwen3-Coder-30B and 480B releases document extension to 1M tokens with YaRN, and Kimi-Dev-72B and Yi-Coder-9B are configured for 131,072 tokens
- License terms affect commercial deployment: Apache 2.0 models (Devstral Small 2, Qwen3-Coder-30B, Yi-Coder-9B, and the Qwen3-Coder-Next and 480B releases) permit commercial use, modification, and redistribution subject to the license's notice, attribution, patent, and trademark conditions; Kimi-Dev-72B is MIT licensed, subject to the Qwen-2.5-72B license agreement for its base model
- Secure code execution is non-negotiable: Repo agents execute AI-generated code, requiring gVisor-isolated sandboxes to contain untrusted workloads and reduce their ability to affect the host or neighboring workloads
- Scale-to-zero economics enable cost-effective experimentation: Per-second billing with scale-to-zero means teams pay for compute while containers run and nothing when scaled to zero, avoiding the fixed cost of permanently allocated GPUs
1. Modal
Modal provides the serverless AI infrastructure layer, with compute across CPUs and GPUs, that makes deploying long-context code LLMs practical for production repo agents. Rather than competing with the models themselves, Modal serves as the foundation enabling teams to deploy, scale, and secure their chosen model without managing GPU clusters or container orchestration.
How Does Modal Work?
Modal turns code into serverless cloud workloads through a simple code-first API, with SDKs in Python, TypeScript, and Go. For code LLMs, this means:
- GPU provisioning: Specify a GPU type and count (for example T4, A100, or H100, and up to eight GPUs on a single machine) and Modal provisions and scales that configuration. Modal's documentation also lists additional GPU types such as L4, A10, L40S, H200, B200, and B300. For supported model families, Modal Auto Endpoints can select a compatible managed serving recipe
- Fast container startup: Modal is engineered for fast container startup and fast cold starts. End-to-end model readiness also includes weight transfer from Modal Volumes and engine initialization, and caching, snapshotting, and warm containers keep model readiness fast for large models
- Code-first deployment: Define deployments in code and deploy vLLM or SGLang-based inference endpoints without writing Kubernetes manifests or Dockerfiles
- gVisor-isolated sandboxes: Execute AI-generated code from repo agents in gVisor-isolated environments designed to contain untrusted workloads and reduce their ability to affect the host or neighboring workloads. Sandboxes are not tied to a single programming language: code inside a Sandbox can run whatever runtime or language the workload requires, and Sandboxes can be created and driven from Modal's Python, TypeScript/JavaScript, and Go SDKs
Infrastructure Capabilities for Code LLMs
Modal's architecture addresses several challenges of deploying long-context code models:
- Fast cold starts: Engineered for fast cold starts and faster feedback loops, with an optimized filesystem that helps containers come online quickly without letting large images slow startup down
- Multi-GPU provisioning: Request multiple GPUs on a single machine (up to eight through the documented GPU interface) for large models such as Qwen3-Coder-480B. The application specifies GPU type and count and configures the inference engine's tensor-parallel settings; Modal provisions the requested resources without requiring Kubernetes or SLURM
- Memory snapshotting: Modal Memory Snapshots materially reduce initialization latency by restoring previously initialized process state instead of rebuilding it from scratch. GPU Memory Snapshots further reduce cold-start latency for GPU workloads
- OpenAI-compatible endpoints: Expose inference as an OpenAI Chat Completions-compatible API that can integrate with agent frameworks supporting configurable OpenAI-compatible providers, such as Claude Code, Cline, and SWE-Agent
- Optimized attention support: Modal supports custom inference environments capable of using optimized attention implementations (for example FlashAttention) through engines such as vLLM, SGLang, and TensorRT-LLM, with performance gains that depend on the model, engine configuration, GPU, workload, and kernel version
Security and Compliance
Production repo agents require enterprise-grade security. Modal delivers:
- SOC 2 Type II: Modal announced in January 2025 that it had completed a SOC 2 Type II audit with no deviations, and its current security documentation continues to state that it has completed a SOC 2 Type 2 audit. Customers can request the current report through Modal's Security Portal
- HIPAA: Modal supports HIPAA-compliant workloads on Enterprise plans via a BAA
- gVisor sandboxing: Modal's standard compute and Sandbox runtime use gVisor for container isolation. Modal also offers a VM Sandbox runtime for workloads that require a full Linux kernel
- OIDC-based authentication: Secure integration with AWS, GCP, and other services without long-lived credentials
Best For: Teams building production repo agents that need to deploy open-weight or custom code LLMs with autoscaling, gVisor-isolated code execution, and per-second billing without operating GPU clusters.
2. Qwen3-Coder-Next
Qwen3-Coder-Next is a strong candidate when low active-parameter compute is a priority for repo-level coding agents. This 80B parameter model uses a hybrid Mixture-of-Experts (MoE) architecture that activates only 3B parameters per token, delivering strong performance for its active-parameter compute.
Architecture and Efficiency
- 512 routed experts, 10 active: The MoE design routes each token through 10 of 512 routed experts, plus one shared expert
- 262,144-token native context: Reduces chunking for small and medium repositories. The current official model card does not document a 1M-token YaRN configuration for this release (the Qwen3-Coder 30B and 480B cards do document it)
- VRAM: The model has 80B stored parameters, so a four-bit deployment may fit its weights on an 80GB GPU (roughly 40GB for weights alone), but total memory depends on the quantization implementation, context length, KV-cache precision, batching, and inference engine. Active parameter count reduces per-token computation; it does not mean only 3B parameters must be stored
- Throughput: Decode speed depends on hardware, quantization, inference engine, parallelism, context length, and batch size. The official model card and technical report do not establish a general tokens-per-second figure
Benchmark Performance
Qwen3-Coder-Next achieves 70.6% on SWE-bench Verified with SWE-Agent (71.1% with MiniSWE-Agent and 71.3% with OpenHands), under the technical report's 300-turn evaluation configuration. Additional results include:
- 42.7% on SWE-bench Pro with SWE-Agent (38.7% with MiniSWE-Agent) under the same 300-turn configuration
- 62.8% on SWE-bench Multilingual with SWE-Agent (56.2% with MiniSWE-Agent and 64.3% with OpenHands)
- 66.2% on Aider-Polyglot
Every SWE-bench result should name the scaffold and evaluation configuration, since results are not model-only constants.
Deployment Considerations
A four-bit deployment may fit the model's weights on a single 80GB GPU, but actual VRAM depends on quantization, context length, KV-cache precision, batching, and engine. Modal's GPU interface provisions the GPU configuration the developer specifies, and for supported model families Modal Auto Endpoints can select a compatible serving recipe. Teams can benchmark quantization and parallelism options on Modal to find a configuration that fits their latency and memory targets.
Best For: Teams that prioritize low active-parameter compute for high-volume agent workloads, where MoE efficiency can reduce per-token cost relative to dense models of comparable capability.
3. Qwen3-Coder-480B-A35B-Instruct
Qwen3-Coder-480B-A35B-Instruct was one of the largest dedicated open-weight code models at its 2025 release. It is no longer the largest available coding-focused model by total parameter count: Moonshot AI's Kimi K2.7 Code is a coding-focused MoE model that lists 1 trillion total parameters. The 480B model targets organizations working with complex, multi-step coding tasks on large codebases.
Architecture Scale
- 480B total parameters, 35B active: A large MoE code model, with 160 experts and 8 active per token
- 262,144-token native context: With documented extension to 1M tokens via YaRN
- VRAM: Roughly 960GB for BF16 weights, or approximately 240GB before runtime overhead at four-bit weight quantization. Actual multi-GPU memory depends on precision, context length, batching, and inference engine; multi-GPU deployment is required
- Throughput and relative accuracy: These depend on scaffold, precision, and workload. Qwen3-Coder-Next surpasses the 480B model on several published coding evaluations, including LiveCodeBench v6, OJBench, Codeforces rating, and Aider-Polyglot
Benchmark Results
Published SWE-bench Verified results for this model vary by scaffold and evaluation protocol. Moonshot AI's Kimi K2 evaluation table attributes 69.6% to Qwen3-Coder-480B under its cited setup. The 74.2% figure with SWE-Agent belongs to GLM-4.7 in the Qwen3-Coder-Next technical report, not to Qwen3-Coder-480B, and that same table lists MiniMax-M2.1 at 74.8%, so 74.2% is not the highest even within that evaluation. Because scaffolds and protocols differ, no single figure should be presented as a universal score. The scale can help on:
- Complex multi-file refactoring tasks
- Deep codebase reasoning requiring extensive context
- Enterprise codebases with intricate dependencies
Infrastructure Requirements
Serving a model of this size requires multi-GPU deployment. Modal can provision requested multi-GPU machines and provides cluster primitives for distributed workloads. Developers configure GPU count, tensor parallelism, and model-server settings in code, and Modal provisions the requested resources.
Best For: Enterprise teams with complex codebases that need a very large MoE model, particularly for automated code review and large-scale refactoring initiatives.
4. Devstral-Small-2-24B-Instruct-2512
Devstral-Small-2-24B-Instruct-2512 from Mistral AI offers strong agentic coding capability under a permissive Apache 2.0 license.
Architecture and License
- 24B dense parameters: No MoE complexity, straightforward deployment
- 256K context window: Roughly 262,144 tokens for code understanding
- Apache 2.0 license: Permits commercial use, modification, and redistribution subject to the license's notice, attribution, patent, and trademark conditions
- VRAM: Mistral describes the model as light enough to run in reduced-precision configurations on a single 24GB RTX 4090 or a 32GB unified-memory Mac. Serving the full 262,144-token context may require more memory; Mistral's vLLM example uses tensor parallel size 2
Benchmark Results
Devstral Small 2 achieves 68.0% on SWE-bench Verified, competitive with much larger models. Additional figures from Mistral's published table:
- 22.5% on Terminal-Bench 2: Command-line and system interaction capability
- 55.7% on SWE-bench Multilingual: Cross-language performance
(The 61.3% SWE-bench Multilingual figure sometimes cited belongs to the larger Devstral 2 123B model, not to Devstral Small 2. The 40.5% Terminal-Bench figure does not match the value in Mistral's Devstral Small 2 table, which reports 22.5%.)
Deployment Simplicity
The dense architecture eliminates MoE routing overhead, making inference behavior more predictable. For teams prioritizing license clarity and deployment simplicity, Devstral Small 2 is a permissively licensed 24B option worth evaluating.
Best For: Organizations that want Apache 2.0 licensing for commercial products, particularly those building developer tools where license clarity affects customer adoption.
5. Qwen3-Coder-30B-A3B-Instruct
Qwen3-Coder-30B-A3B-Instruct is a smaller Qwen coder variant with lower weight-memory requirements and Apache 2.0 licensing.
Efficiency Advantages
- 30.5B total parameters, 3.3B active: 128 experts with 8 active per token
- 262,144-token native context: With documented extension to 1M tokens via YaRN
- VRAM: Four-bit weights are roughly 15 to 19GB depending on quantization format, which can fit an A100 40GB or high-end consumer GPUs. Practical VRAM increases with context length, runtime overhead, and batching
- Throughput: Depends on GPU, quantization, context length, engine, parallelism, and batching; no general tokens-per-second figure is established
- Apache 2.0 license: Commercial use, modification, and redistribution subject to the license's conditions
Cost-Performance Balance
For teams processing millions of tokens monthly, the combination of a small active-parameter count and permissive licensing makes this a candidate for high-volume workloads where per-token cost matters. The lower weight-memory footprint also enables deployment on less expensive GPU tiers, with real cost efficiency tracking your own hardware prices, latency targets, and task success rates.
Best For: Cost-conscious teams needing production-grade code generation at scale, particularly those with existing A100 40GB capacity or seeking to minimize weight-memory footprint.
6. Kimi-Dev-72B
Kimi-Dev-72B from Moonshot AI takes a specialized approach to coding capability through reinforcement learning on Docker test suites.
Specialized Training
- 72B dense parameters: Based on a dense Qwen2.5-72B architecture, without MoE complexity
- 131,072-token context window: Shorter than the Qwen coder models but sufficient for many repositories
- MIT license: Released under the MIT license, subject to the Qwen-2.5-72B license agreement for its base model
- VRAM: The released BF16 weights total roughly 145GB and do not fit on a single 80GB A100 in that form. A four-bit quantized version may approach 36 to 40GB for weights, before KV cache and runtime overhead
Training Methodology
Unlike models trained primarily on code completion, Kimi-Dev-72B underwent reinforcement learning using Docker-based test execution, with reward based on passing the full test suite. This approach optimizes for real-world software engineering tasks where tests validate code correctness.
Benchmark Performance
The model card reports 60.4% on SWE-bench Verified, a solid agentic result at its original release, though below the strongest published 2026 agentic-coding numbers. The specialized RL training may offer advantages on specific task types not captured by standard benchmarks.
Best For: Teams interested in models trained specifically for test-driven development workflows, or those seeking an MIT-licensed alternative to the Qwen family.
7. Yi-Coder-9B-Chat
Yi-Coder-9B-Chat from 01.ai provides an accessible entry point for teams exploring repo-level coding agents without significant infrastructure investment.
Accessibility Advantages
- 9B dense parameters: Smallest footprint among the models in this list
- 131,072-token context window: Adequate for many single-repository contexts
- Apache 2.0 license: Commercial use, modification, and redistribution subject to the license's conditions
- VRAM: The released BF16 weights total roughly 18GB, which alone exceed a 16GB T4. Four-bit quantized versions may use about 5GB for weights and can run on a T4 or consumer GPU at constrained context and batch settings
- Throughput: The smallest parameter count in this list is likely to offer lower inference cost, though throughput depends on hardware, precision, context length, inference engine, and batching
Use Case Fit
While benchmark performance trails larger models, Yi-Coder-9B suits scenarios where:
- Rapid iteration matters more than maximum accuracy
- Infrastructure costs must remain minimal
- Local development and testing precedes cloud deployment
- Teams are evaluating agent architectures before scaling
For teams using Modal Notebooks, Yi-Coder-9B is a lightweight option for interactive development and prototyping with modest GPU overhead.
Best For: Individual developers and small teams prototyping repo agents, or organizations needing fast, lightweight code assistance for less complex tasks.
Why Modal Stands Out for Deploying Long-Context Code LLMs
Managed Recipes and Workload-Aware Configuration
For supported model families, Modal Auto Endpoints select a compatible serving recipe. For custom deployments, developers choose the GPU configuration and can benchmark alternative quantization, parallelism, and memory-management strategies on Modal. Active parameter count alone does not determine VRAM needs, since most or all expert weights may still need to reside in memory depending on serving architecture and offloading strategy. Managed recipes and workload benchmarking help teams avoid unnecessarily expensive GPU configurations.
Scale-to-Zero Economics for Bursty Agent Workloads
Repo agents experience highly variable load, with intense activity during code review cycles and minimal usage overnight. Modal's scale-to-zero capability means teams pay for compute while containers run and can scale to zero when no containers are active, helping avoid the fixed cost of permanently allocated GPUs. Because billing is per second across CPU and GPU resources, savings track directly with real utilization, traffic distribution, warm-container settings, and GPU choice, so teams can model them from their own workload characteristics.
Secure Sandboxes for Agentic Code Execution
Repo agents execute AI-generated code to test solutions and validate changes. Modal's gVisor-isolated Sandboxes contain untrusted generated code and begin without access to other Modal resources. Teams decide exactly which Secrets, storage, and credentials a Sandbox receives, so access to sensitive resources stays explicitly scoped by design. Sandboxes also support both common agent architectures: running the agent inside the Sandbox, which is a quick starting point for internal coding agents, and running the agent outside the Sandbox, which gives platforms with proprietary agent logic a cleaner separation of concerns. Ramp, for example, built a background coding agent on Modal that generates code changes and writes them back as commits and pull requests.
Code-First Developer Experience
Modal's code-first SDKs, available in Python, TypeScript, and Go, reduce the infrastructure complexity that typically accompanies LLM deployment. A concise, code-defined declaration specifies compute resources across CPUs and GPUs, container configuration, and scaling behavior. This simplicity accelerates the path from prototype to production, allowing teams to focus on agent logic rather than infrastructure orchestration.
Enterprise-Grade Security and Compliance
For organizations deploying code LLMs on proprietary codebases, security matters. Modal's SOC 2 Type II audit (announced January 2025 with no deviations), HIPAA-compliant workloads on Enterprise plans via a BAA, and gVisor-based isolation provide a security foundation for enterprise teams.
