Small open-source code LLMs have transformed how teams build and deploy AI agents. Several open-weight models ranging from 7B to 30B parameters now report competitive results on selected agentic coding benchmarks under particular agent and evaluation configurations. For engineering teams building coding agents, selecting the right model, and the right infrastructure to run it, shapes whether your agent can handle real-world software engineering challenges at production scale. This guide examines seven options for building AI agents with open-source code LLMs, starting with Modal, the AI infrastructure platform that makes deploying these models practical for production workloads.
Key Takeaways
1. Modal
Modal provides the AI infrastructure that makes running small open-source code LLMs practical for production AI agents. While the other six entries on this list are models you deploy, Modal is the platform that handles deployment, scaling, and secure execution.
How Modal Enables AI Agent Development
Modal's platform addresses the core infrastructure challenges teams face when building coding agents:
Why Infrastructure Matters for Code LLMs
Deploying code LLMs for AI agents involves unique challenges beyond standard inference workloads:
Modal's inference infrastructure handles these requirements through memory snapshotting, an optimized filesystem for fast startup, and primitives like Volumes for persistent storage, Queues and Dicts for coordination and shared state, and Tunnels for encrypted network access to running containers.
Documented Results
Modal's production deployments demonstrate performance across demanding AI workloads:
Best For: Teams building production AI agents who want infrastructure that handles deployment complexity while providing fast cold starts, elastic GPU scaling, and secure sandboxed code execution.
2. Ornith-1.0-9B
Ornith-1.0-9B from DeepReinforce-AI introduces self-scaffolding reinforcement learning to reach strong reported results at 9B parameters. Rather than just generating code, the model is trained to generate and improve its own execution workflow.
Core Capabilities
Performance Benchmarks
Ornith-1.0-9B reports results that are competitive with much larger models under its documented harness:
Why It Matters for AI Agents
The self-scaffolding training objective is designed to improve the search trajectories and execution scaffolds Ornith uses for multi-step tasks. This is relevant for agentic workflows where the model must plan, execute, observe results, and iterate. As commentary, AI Tutorial Engineer Mehul Gupta described Ornith-1.0-9B as one of the most interesting open-source releases for agentic coding, highlighting its ability to generate and improve its own execution scaffold. This is an attributed opinion rather than independent technical validation of the benchmark or architecture claims.
Deployment Requirements
Deploy Ornith-1.0-9B with Modal's GPU infrastructure to leverage instant autoscaling and fast cold starts for responsive agent interactions. Best For: Teams building self-improving coding agents that want strong multi-step reasoning within a 9B parameter budget.
3. Devstral-Small-2-24B-Instruct
Mistral AI's Devstral-Small-2-24B-Instruct is designed for agentic software-engineering workflows and can run on consumer hardware. The 24B dense architecture provides a balance between capability and deployment flexibility that many teams look for.
Core Capabilities
Performance Benchmarks
Devstral-Small-2 reports results competitive with models many times its size in Mistral's evaluation:
Deployment Flexibility
A key advantage of Devstral-Small-2 is hardware accessibility:
Why It Matters for AI Agents
The combination of 256K context, vision capabilities (multimodal), and modest local hardware requirements makes Devstral-Small-2 practical for teams scaling from prototype to production. The Apache 2.0 license permits commercial use, modification, and distribution, subject to its notice, attribution, patent, and other license conditions. Use Modal Notebooks for iterative development with Devstral-Small-2, then deploy to production with Modal's inference infrastructure. Best For: Teams seeking agentic coding with flexible deployment options and a permissive license.
4. Qwen3-Coder-30B-A3B-Instruct
Qwen3-Coder-30B-A3B-Instruct uses a Mixture-of-Experts (MoE) architecture that activates only 3.3B parameters per forward pass out of 30.5B total. This design can reduce arithmetic work relative to a similarly sized dense model, which is attractive for cost-sensitive production deployments.
Core Capabilities
Performance and Efficiency
The MoE architecture changes the arithmetic profile of inference:
Repository-Scale Work
For AI agents that work across large codebases, a longer context can expose the model to more files at once. The 262,144-token native window may support tasks such as:
Task success and maximum usable context vary with the target repositories and deployment hardware, since a long window establishes an input-capacity ceiling rather than guaranteeing that any given repository fits or that distant context is used reliably. SiliconFlow describes the model as offering strong performance and efficiency. Scale Qwen3-Coder deployments efficiently with Modal's batch processing for offline code analysis tasks. Best For: Teams that want to expose more repository content per request with MoE efficiency for cost-conscious production deployment.
5. DeepCoder-14B-Preview
DeepCoder-14B-Preview from Agentica-Org is a notable example of open-source reinforcement learning for code generation. The model reports results close to o3-mini's Low setting on a specific benchmark slice while publishing extensive training assets.
Core Capabilities
Performance Benchmarks
DeepCoder demonstrates that an open RL recipe can produce competitive benchmark results:
Reproducibility Assets for RL
What makes DeepCoder notable beyond benchmarks is the breadth of its released assets. The release includes:
As commentary, AI analyst Asif Razzaq described DeepCoder-14B as closing the gap with leading models like o3-mini using just 14 billion parameters. This is an attributed opinion rather than independent verification of general parity.
Why It Matters for AI Agents
DeepCoder publishes unusually extensive training and evaluation assets, improving reproducibility and allowing teams to inspect and adapt the RL recipe. These assets support reproduction and adaptation of the methodology, though they do not let a team reconstruct exactly how particular learned reasoning behaviors emerged. Fine-tune DeepCoder variants for specialized agent tasks using Modal's training infrastructure with access to H100 and B200 clusters. Best For: Teams that want a reproducible RL training methodology with competitive coding benchmark results.
6. Seed-Coder-8B-Instruct
ByteDance-Seed's Seed-Coder-8B-Instruct uses a model-centric data curation pipeline that predominantly relies on LLM-based scoring and filtering rather than hand-crafted rules, reaching strong reported results at the 8B scale.
Core Capabilities
Performance Benchmarks
In the authors' 2025 evaluation against the listed approximately 8B baselines, Seed-Coder reported the strongest aggregate results across several tasks:
Software Engineering Capabilities
The ByteDance Seed Research Team reports that Seed-Coder achieves an 11.2% resolved rate on OpenHands, the strongest reported result among the approximately 8B baselines it evaluated, without pre-defined workflows.
Why It Matters for AI Agents
The model-centric curation approach suggests that LLM-based filtering can be an effective alternative to extensive hand-crafted filtering rules. Isolating dataset size from curation quality would require controlled ablations, so this points toward a promising training direction rather than a settled conclusion that a smaller dataset beat a larger manually curated one. Deploy Seed-Coder-8B with Modal's platform to leverage the multi-cloud capacity pool and optimized container runtime. Best For: Teams that want strong code editing results at the 8B scale, particularly for debugging and code transformation tasks.
7. SWE-agent-LM-7B
SWE-agent-LM-7B from the SWE-bench team is purpose-built for the SWE-agent framework. Fine-tuned from Qwen2.5-Coder-7B-Instruct on automatically generated software engineering tasks, it represents a new approach to training coding agents.
Core Capabilities
Performance on SWE Benchmarks
SWE-smith Data Pipeline
The SWE-smith research introduces automated task generation for software engineering agents. Rather than manually curating training examples, the pipeline generates diverse tasks at scale from real repositories.
Why It Matters for AI Agents
SWE-agent-LM-7B shows that automated data generation can produce effective training signals for coding agents. At approximately 7.6B parameters, it is the smallest model on this list, so its comparatively small parameter count may make local or high-throughput deployment easier. Whether it is suitable for a given scenario depends on:
Run SWE-agent-LM-7B evaluations at scale with Modal's batch processing to queue up to 1M inputs with built-in observability. Best For: Teams specifically targeting the SWE-agent framework, or teams evaluating a comparatively small model for software engineering agents subject to quantization, memory, latency, and task-quality requirements.
Why Modal Stands Out for AI Agent Infrastructure
An AI-Oriented Platform
Modal's AI-oriented platform includes an AI-native container runtime, an optimized filesystem, workload scheduling, and an integrated container-image system. This architecture is designed to reduce the infrastructure work associated with bursty workloads across CPUs and GPUs and with model deployment:
Rather than requiring teams to assemble these capabilities directly from general-purpose cloud services, Modal packages them into a code-first serverless platform, with SDKs and code-defined infrastructure available in Python, TypeScript, and Go.
Fast Cold Starts
AI agents make multiple inference calls per task. Modal is engineered for fast cold starts and faster feedback loops. Its optimized filesystem helps containers come online quickly without letting large images slow startup down, and packaging or pre-downloading large model weights keeps startup fast.
Secure Sandboxes for AI-Generated Code
Running code that an LLM generates creates security risks. Modal's Sandboxes provide:
This matters because coding agents don't just write code; they execute it. Secure sandboxed execution is a priority for production deployments.
Enterprise Security and Compliance
Modal's security posture meets common enterprise requirements:
Instant Autoscaling Without Capacity Planning
Building coding agents means unpredictable load patterns. Some tasks require seconds of inference; others need hours of compute across hundreds of GPUs. Modal's autoscaling handles this across CPUs and GPUs without reservations or quotas, scaling from zero to 1,000+ GPUs as workload demands.
Complete Platform for Agent Development
Beyond inference, Modal provides the primitives coding agents use:
For teams building AI agents with small open-source code LLMs, Modal provides a broad infrastructure stack, from development through production deployment at scale. Explore the documentation to see how Modal's primitives map to your agent architecture.
Explore Modal's documentation to see how its primitives map to your agent architecture.
