Modal logo
FeaturedGetting started Hello, worldSimple web scraperServing web endpointsLarge language models (LLMs) Deploy an OpenAI-compatible LLM service with vLLMCut Ministral 3 cold start times by 10x with snapshotsMaximize tokens per second in batch processing with vLLMServe an ultra-low-latency chatbot with SGLangEfficient LLM Finetuning with UnslothRun a multimodal RAG chatbot to answer questions about PDFsFine-tune an LLM to replace your CEODeploy a stateless MCP with FastMCPImages, video, & 3D Edit images with Flux KontextFine-tune Wan2.1 video models on your faceRun Flux fast with torch.compileFine-tune Flux with LoRAAnimate images with LTX-VideoGenerate video clips with LTX-VideoRun Stable Diffusion with a CLI, API, and web UIAudio Deploy a Moshi voice chatbotStream transcripts at the speed of speech using Kyutai STTMake music with ACE-StepGenerate speech with ChatterboxRun high throughput batched transcription with WhisperFine-tune Whisper to recognize new wordsReal-time communication (WebRTC) Serverless WebRTCWebRTC quickstart with FastRTCComputational biology Fold proteins with Chai-1Build a protein-folding dashboardFold proteins with Boltz-2Modal Sandboxes Run a background coding agent with OpenCodeBuild a scalable AI coding platformRun a LangGraph agent's code in a secure GPU sandboxControl a sandboxed computer with an LLMBuild a stateful, sandboxed code interpreterRun Node.js, Ruby, and more in a SandboxRun a sandboxed Jupyter notebookReinforcement Learning Train a model to solve math problems using GRPO and verlTrain a model to solve coding problems using GRPO and TRLEmbeddings Embed millions of documents with TEITurn satellite images into vectors and store them in MongoDBParallel processing and job scheduling Deploy a Hacker News SlackbotRun a Document OCR job queueServe a Document OCR web appTraining models from scratch Train an SLM with early-stopping grid search over hyperparametersRun long, resumable training jobsHosting popular libraries YOLO: Fine-tune and serve computer vision modelsBlender: Build a 3D render farmStreamlit: Run and deploy Streamlit appsComfyUI: Run Flux on ComfyUI as an APISQLite: Publish explorable data with DatasetteAlgolia: Build docsearch with a crawlerConnecting to other APIs Discord: Deploy and run a Discord BotGoogle Sheets: Sync databases and APIs to a Google SheetOpenAI: Run a RAG Q&A chatbotTailscale: Add Modal Apps to your VPNPrometheus: Publish custom metrics with PushgatewayManaging data Mount S3 buckets in Modal appsBuild your own data warehouse with DuckDB, DBT, and ModalCreate a LoRA Playground with Modal, Gradio, and S3Miscellaneous
View on GitHub

Build a coding agent with Modal Sandboxes and LangGraph

This example demonstrates how to build an LLM coding “agent” that can generate and execute Python code, using documentation from the web to inform its approach.

Naturally, we use the agent to generate code that runs language models.

The agent is built with LangGraph, a library for building directed graphs of computation popular with AI agent developers, and uses models from the OpenAI API.

Setup 

You will need two Modal Secrets to run this example: one to access the OpenAI API and another to access the LangSmith API for logging the agent’s behavior.

To create them, head to the Secrets dashboard, select “Create new secret”, and use the provided templates for OpenAI and LangSmith.

Creating a Sandbox 

We execute the agent’s code in a Modal Sandbox, which allows us to run arbitrary code in a safe environment. In this example, we will use the transformers library to generate text with a pre-trained model. Let’s create a Sandbox with the necessary dependencies.

We also need a way to run our code in the sandbox. For this, we’ll write a simple wrapper around the Modal Sandbox exec method. We use exec because it allows us to run code without spinning up a new container. And we can reuse the same container for multiple runs, preserving state.

Constructing the agent’s graph 

Now that we have the sandbox to execute code in, we can construct our agent’s graph. Our graph is defined in the edges and nodes modules associated with this example. Nodes are actions that change the state. Edges are transitions between nodes.

The idea is simple: we start at the node generate, which invokes the LLM to generate code based off documentation. The generated code is executed (in the sandbox) as part of an edge called check_code_execution and then the outputs are passed to the LLM for evaluation (the evaluate_execution node). If the LLM determines that the code has executed correctly — which might mean that the code raised an exception! — we pass along the decide_to_finish edge and finish.

We now set up the graph and compile it. See the src module for details on the content of the graph and the nodes we’ve defined.

Running the Graph 

Now let’s call the agent from the command line!

We define a local_entrypoint that runs locally and triggers execution on Modal.

You can invoke it by executing following command from a folder that contains the codelangchain directory from our examples repo:

If things are working properly, you should see output like the following: