Small open-source code LLMs have transformed how teams build and deploy AI agents. Several open-weight models ranging from 7B to 30B parameters now report competitive results on selected agentic coding benchmarks under particular agent and evaluation configurations. For engineering teams building coding agents, selecting the right model, and the right infrastructure to run it, shapes whether your agent can handle real-world software engineering challenges at production scale. This guide examines seven options for building AI agents with open-source code LLMs, starting with Modal, the AI infrastructure platform that makes deploying these models practical for production workloads.
Key Takeaways
- Small models report competitive benchmark results: Several small open-weight models report competitive results on selected coding benchmarks under particular agent and evaluation configurations. For example, Ornith-1.0-9B reports 69.4% on SWE-Bench Verified under its documented OpenHands harness, a result that partly reflects the model-plus-agent system rather than model capability alone
- Infrastructure determines production viability: Running code LLMs for AI agents benefits from fast cold starts, elastic autoscaling, and on-demand GPU access. Modal is engineered for fast cold starts and faster feedback loops through techniques like memory snapshotting and an optimized filesystem
- MoE architectures can reduce arithmetic work: Qwen3-Coder-30B-A3B activates only 3.3B of its 30.5B parameters per forward pass, which can reduce arithmetic work relative to a similarly sized dense model; end-to-end serving cost still depends on memory, quantization, hardware, context length, and utilization
- Reinforcement learning post-training shows strong benchmark results: RL-post-trained small models such as DeepCoder-14B and Ornith-1.0-9B have reported competitive results on selected coding benchmarks, including near-o3-mini-Low performance for DeepCoder on one LiveCodeBench evaluation slice
- Secure sandboxed execution is essential: AI agents generating and running code require isolated environments, and Modal's standard container-based sandboxing uses gVisor for compute isolation, backed by SOC 2 Type II compliance. Modal also offers VM Sandboxes for workloads that need a full Linux kernel
- Context window size affects how much a model can see at once: The 262,144-token native windows in several of these models can fit more repository content in one request, although whether a repository fits, and whether the model uses that context effectively, depends on repository size, tokenization, inference memory, and the evaluation setup
1. Modal
Modal provides the AI infrastructure that makes running small open-source code LLMs practical for production AI agents. While the other six entries on this list are models you deploy, Modal is the platform that handles deployment, scaling, and secure execution.
How Modal Enables AI Agent Development
Modal's platform addresses the core infrastructure challenges teams face when building coding agents:
- Fast cold starts: Engineered for fast cold starts and faster feedback loops, with an optimized filesystem that helps containers come online quickly without letting large images slow startup down
- Instant autoscaling: Scale from zero to thousands of containers across CPUs and GPUs without capacity planning or reservations
- Elastic GPU access: Access H100s, B200s, A100s, and other accelerators on demand
- Secure sandboxed execution: Run AI-generated code with gVisor-based isolation for standard container compute, with VM Sandboxes available when a full Linux kernel is required
Why Infrastructure Matters for Code LLMs
Deploying code LLMs for AI agents involves unique challenges beyond standard inference workloads:
- Multi-step reasoning: Agents execute multiple inference calls per task, requiring low-latency infrastructure
- Tool calling: Models need to interact with file systems, terminals, and external services
- Code execution: Generated code must run in isolated environments to prevent security issues
- Long contexts: Repository-scale work benefits from models with 64K to 256K token windows
Modal's inference infrastructure handles these requirements through memory snapshotting, an optimized filesystem for fast startup, and primitives like Volumes for persistent storage, Queues and Dicts for coordination and shared state, and Tunnels for encrypted network access to running containers.
Documented Results
Modal's production deployments demonstrate performance across demanding AI workloads:
- Physical Intelligence reports that running its robotic-control inference on Modal kept network overhead to roughly 10-15ms
- Ramp built a full-context background coding agent on Modal, using Sandboxes to generate code changes that are written back into commits and pull requests
- Lovable uses Modal Sandboxes as preview environments for generated apps and websites
- Modal customers use the platform across generative AI inference, LLM fine-tuning, computational biology, and media processing
Best For: Teams building production AI agents who want infrastructure that handles deployment complexity while providing fast cold starts, elastic GPU scaling, and secure sandboxed code execution.
2. Ornith-1.0-9B
Ornith-1.0-9B from DeepReinforce-AI introduces self-scaffolding reinforcement learning to reach strong reported results at 9B parameters. Rather than just generating code, the model is trained to generate and improve its own execution workflow.
Core Capabilities
- Parameters: 9B dense
- Context Window: 262,144 tokens
- License: MIT
- Key Innovation: Self-scaffolding RL, a training objective designed to help the model build and refine its own execution scaffold
Performance Benchmarks
Ornith-1.0-9B reports results that are competitive with much larger models under its documented harness:
- 69.4% on SWE-Bench Verified, reported using OpenHands with a 256K context window and specified sampling settings; the comparison with Qwen3.5-35B (70%), a model roughly 4x larger, partly reflects the surrounding agent system rather than model capability alone
- 43.1% on Terminal-Bench 2.1 under its documented harness, more than doubling Qwen3.5-9B's 21.3%
- Produces explicit <think> reasoning traces that compatible serving runtimes can expose separately; these traces should not be assumed to be fully faithful explanations of internal reasoning
- Can be served through recent vLLM or SGLang releases using an OpenAI-compatible API with parsed tool calls
Why It Matters for AI Agents
The self-scaffolding training objective is designed to improve the search trajectories and execution scaffolds Ornith uses for multi-step tasks. This is relevant for agentic workflows where the model must plan, execute, observe results, and iterate.
As commentary, AI Tutorial Engineer Mehul Gupta described Ornith-1.0-9B as one of the most interesting open-source releases for agentic coding, highlighting its ability to generate and improve its own execution scaffold. This is an attributed opinion rather than independent technical validation of the benchmark or architecture claims.
Deployment Requirements
- Runs on a single 80GB GPU (approximately 19GB in bf16)
- Batching behavior is provided by the selected inference runtime (for example, vLLM or SGLang)
- Works with standard inference frameworks (vLLM, SGLang)
Deploy Ornith-1.0-9B with Modal's GPU infrastructure to leverage instant autoscaling and fast cold starts for responsive agent interactions.
Best For: Teams building self-improving coding agents that want strong multi-step reasoning within a 9B parameter budget.
3. Devstral-Small-2-24B-Instruct
Mistral AI's Devstral-Small-2-24B-Instruct is designed for agentic software-engineering workflows and can run on consumer hardware. The 24B dense architecture provides a balance between capability and deployment flexibility that many teams look for.
Core Capabilities
- Parameters: 24B dense
- Context Window: 256K tokens
- License: Apache 2.0
- Architecture: Uses Scalable-Softmax attention and RoPE scaling from Ministral 3
Performance Benchmarks
Devstral-Small-2 reports results competitive with models many times its size in Mistral's evaluation:
- 68.0% on SWE-Bench Verified in Mistral's evaluation, comparable to GLM 4.6 (355B) despite being roughly 15x smaller
- 55.7% on SWE-Bench Multilingual
- Approaches flagship Devstral 2 (123B) at 72.2%
Deployment Flexibility
A key advantage of Devstral-Small-2 is hardware accessibility:
- The official FP8 checkpoint can run locally on a single RTX 4090 or a Mac with 32GB RAM. Practical context length and throughput depend on precision, runtime, and available memory (a 24B model in BF16 would need roughly 48GB for weights alone, before KV cache and runtime overhead)
- Runs agentic workflows without enterprise GPU clusters
- Official Mistral Vibe CLI integration for terminal-based coding agents
Why It Matters for AI Agents
The combination of 256K context, vision capabilities (multimodal), and modest local hardware requirements makes Devstral-Small-2 practical for teams scaling from prototype to production. The Apache 2.0 license permits commercial use, modification, and distribution, subject to its notice, attribution, patent, and other license conditions.
Use Modal Notebooks for iterative development with Devstral-Small-2, then deploy to production with Modal's inference infrastructure.
Best For: Teams seeking agentic coding with flexible deployment options and a permissive license.
4. Qwen3-Coder-30B-A3B-Instruct
Qwen3-Coder-30B-A3B-Instruct uses a Mixture-of-Experts (MoE) architecture that activates only 3.3B parameters per forward pass out of 30.5B total. This design can reduce arithmetic work relative to a similarly sized dense model, which is attractive for cost-sensitive production deployments.
Core Capabilities
- Parameters: 30.5B total, 3.3B active (MoE with 128 experts, 8 active)
- Context Window: 262,144 tokens native, extendable toward 1M with YaRN
- License: Apache 2.0
- Key Feature: Provides a function-call format for agent integrations such as Qwen Code and Cline
Performance and Efficiency
The MoE architecture changes the arithmetic profile of inference:
- 30.5B total parameters with only 3.3B active per token, which can reduce arithmetic work relative to a similarly sized dense model; total weights must still be stored and moved through the serving system
- Qwen reports strong performance among open models on agentic coding, browser-use, and foundational coding evaluations
- 262,144-token native context can fit more repository content in one request, extendable toward 1M tokens with YaRN. The official card notes that context may need to be reduced to 32,768 tokens when memory is insufficient
Repository-Scale Work
For AI agents that work across large codebases, a longer context can expose the model to more files at once. The 262,144-token native window may support tasks such as:
- Cross-file refactoring
- Architecture-aware code generation
- Dependency analysis and updates
Task success and maximum usable context vary with the target repositories and deployment hardware, since a long window establishes an input-capacity ceiling rather than guaranteeing that any given repository fits or that distant context is used reliably.
SiliconFlow describes the model as offering strong performance and efficiency.
Scale Qwen3-Coder deployments efficiently with Modal's batch processing for offline code analysis tasks.
Best For: Teams that want to expose more repository content per request with MoE efficiency for cost-conscious production deployment.
5. DeepCoder-14B-Preview
DeepCoder-14B-Preview from Agentica-Org is a notable example of open-source reinforcement learning for code generation. The model reports results close to o3-mini's Low setting on a specific benchmark slice while publishing extensive training assets.
Core Capabilities
- Parameters: 14B dense
- Context Window: 32K training, generalizes to 64K
- License: MIT
- Training Method: GRPO+ with iterative context lengthening (16K to 32K)
Performance Benchmarks
DeepCoder demonstrates that an open RL recipe can produce competitive benchmark results:
- 60.6% Pass@1 on the specified LiveCodeBench v5 slice (questions from August 1, 2024 through February 1, 2025), close to o3-mini-2025-01-31 at its Low setting (60.9%). See also the project write-up
- 92.6% on HumanEval+
- 1936 Codeforces rating (95.3 percentile)
Reproducibility Assets for RL
What makes DeepCoder notable beyond benchmarks is the breadth of its released assets. The release includes:
- Training code
- Training data references
- Weights & Biases logs
- The GRPO+ implementation
As commentary, AI analyst Asif Razzaq described DeepCoder-14B as closing the gap with leading models like o3-mini using just 14 billion parameters. This is an attributed opinion rather than independent verification of general parity.
Why It Matters for AI Agents
DeepCoder publishes unusually extensive training and evaluation assets, improving reproducibility and allowing teams to inspect and adapt the RL recipe. These assets support reproduction and adaptation of the methodology, though they do not let a team reconstruct exactly how particular learned reasoning behaviors emerged.
Fine-tune DeepCoder variants for specialized agent tasks using Modal's training infrastructure with access to H100 and B200 clusters.
Best For: Teams that want a reproducible RL training methodology with competitive coding benchmark results.
6. Seed-Coder-8B-Instruct
ByteDance-Seed's Seed-Coder-8B-Instruct uses a model-centric data curation pipeline that predominantly relies on LLM-based scoring and filtering rather than hand-crafted rules, reaching strong reported results at the 8B scale.
Core Capabilities
- Parameters: 8B dense
- Context Window: 32,768 tokens
- License: MIT
- Key Innovation: Model-centric data curation that predominantly relies on LLM-based scoring and filtering, reducing reliance on hand-crafted language-specific rules
Performance Benchmarks
In the authors' 2025 evaluation against the listed approximately 8B baselines, Seed-Coder reported the strongest aggregate results across several tasks:
- 84.8% on HumanEval, 85.2% on MBPP
- 53.3% on BigCodeBench (Full), 26.4% on BigCodeBench (Hard)
- Strong reported results on CodeEditorBench across debugging, translation, and polishing
Software Engineering Capabilities
The ByteDance Seed Research Team reports that Seed-Coder achieves an 11.2% resolved rate on OpenHands, the strongest reported result among the approximately 8B baselines it evaluated, without pre-defined workflows.
Why It Matters for AI Agents
The model-centric curation approach suggests that LLM-based filtering can be an effective alternative to extensive hand-crafted filtering rules. Isolating dataset size from curation quality would require controlled ablations, so this points toward a promising training direction rather than a settled conclusion that a smaller dataset beat a larger manually curated one.
Deploy Seed-Coder-8B with Modal's platform to leverage the multi-cloud capacity pool and optimized container runtime.
Best For: Teams that want strong code editing results at the 8B scale, particularly for debugging and code transformation tasks.
7. SWE-agent-LM-7B
SWE-agent-LM-7B from the SWE-bench team is purpose-built for the SWE-agent framework. Fine-tuned from Qwen2.5-Coder-7B-Instruct on automatically generated software engineering tasks, it represents a new approach to training coding agents.
Core Capabilities
- Parameters: approximately 7.6B, conventionally named 7B (fine-tuned from Qwen2.5-Coder-7B-Instruct)
- Context Window: 32,768 tokens (per the model's config.json)
- License: Apache 2.0
- Training Data: The Hugging Face model card lists 5K Claude 3.7 Sonnet trajectories from SWE-smith, while the NeurIPS 2025 SWE-smith paper reports a 2K training set for the 7B model and a 5K set for the 32B model; these sources currently conflict
Performance on SWE Benchmarks
- With the SWE-agent system at Pass@1, 11.7% on SWE-bench Lite and 15.2% on SWE-bench Verified; the paper reports a 2K training set for this model and excludes verifier-based or multiple-attempt systems from that comparison
- Optimized specifically for SWE-agent scaffolding
SWE-smith Data Pipeline
The SWE-smith research introduces automated task generation for software engineering agents. Rather than manually curating training examples, the pipeline generates diverse tasks at scale from real repositories.
Why It Matters for AI Agents
SWE-agent-LM-7B shows that automated data generation can produce effective training signals for coding agents. At approximately 7.6B parameters, it is the smallest model on this list, so its comparatively small parameter count may make local or high-throughput deployment easier. Whether it is suitable for a given scenario depends on:
- Quantization and device memory for edge or local deployment
- Latency and throughput requirements where cost-per-request matters
- Task-quality thresholds for teams with limited GPU budgets
Run SWE-agent-LM-7B evaluations at scale with Modal's batch processing to queue up to 1M inputs with built-in observability.
Best For: Teams specifically targeting the SWE-agent framework, or teams evaluating a comparatively small model for software engineering agents subject to quantization, memory, latency, and task-quality requirements.
Why Modal Stands Out for AI Agent Infrastructure
An AI-Oriented Platform
Modal's AI-oriented platform includes an AI-native container runtime, an optimized filesystem, workload scheduling, and an integrated container-image system. This architecture is designed to reduce the infrastructure work associated with bursty workloads across CPUs and GPUs and with model deployment:
- Memory snapshotting for faster model loading
- Optimized filesystem designed for large model weights, helping containers come online quickly without letting large images slow startup down
- Multi-cloud capacity pool for elastic access to compute resources across CPUs and GPUs
Rather than requiring teams to assemble these capabilities directly from general-purpose cloud services, Modal packages them into a code-first serverless platform, with SDKs and code-defined infrastructure available in Python, TypeScript, and Go.
Fast Cold Starts
AI agents make multiple inference calls per task. Modal is engineered for fast cold starts and faster feedback loops. Its optimized filesystem helps containers come online quickly without letting large images slow startup down, and packaging or pre-downloading large model weights keeps startup fast.
Secure Sandboxes for AI-Generated Code
Running code that an LLM generates creates security risks. Modal's Sandboxes provide:
- gVisor-based compute isolation for standard container execution, with VM Sandboxes available for workloads that require a full Linux kernel
- Fast scheduling for 100,000+ concurrent sandboxes
- Dynamically defined containers
- Filesystem APIs, sandbox tunnels, and connection tokens for authenticating access to sandbox servers
This matters because coding agents don't just write code; they execute it. Secure sandboxed execution is a priority for production deployments.
Enterprise Security and Compliance
Modal's security posture meets common enterprise requirements:
- SOC 2 Type II certified with no deviations found
- HIPAA support for Enterprise customers under a Business Associate Agreement
- TLS 1.3 for public APIs
- Rust-based worker runtime for security-critical infrastructure
Instant Autoscaling Without Capacity Planning
Building coding agents means unpredictable load patterns. Some tasks require seconds of inference; others need hours of compute across hundreds of GPUs. Modal's autoscaling handles this across CPUs and GPUs without reservations or quotas, scaling from zero to 1,000+ GPUs as workload demands.
Complete Platform for Agent Development
Beyond inference, Modal provides the primitives coding agents use:
For teams building AI agents with small open-source code LLMs, Modal provides a broad infrastructure stack, from development through production deployment at scale. Explore the documentation to see how Modal's primitives map to your agent architecture.
Explore Modal's documentation to see how its primitives map to your agent architecture.
View Modal Docs