Modal logo
FeaturedGetting started Hello, worldSimple web scraperServing web endpointsLarge language models (LLMs) Deploy an OpenAI-compatible LLM service with vLLMRun DeepSeek-R1 and Phi-4 with llama.cppLow-latency, serverless TensorRT-LLMRun Vision-Language Models with SGLangEfficient LLM Finetuning with UnslothRun a multimodal RAG chatbot to answer questions about PDFsDeploy and benchmark the Tokasaurus high-throughput LLM serverFine-tune an LLM to replace your CEOImages, video, & 3D Edit images with Flux KontextFine-tune Wan2.1 video models on your faceRun Flux fast with torch.compileFine-tune Flux with LoRAAnimate images with LTX-VideoGenerate video clips with LTX-VideoRun Stable Diffusion with a CLI, API, and web UIAudio Deploy a Moshi voice chatbotStream transcripts at the speed of speech using Kyutai STTMake music with ACE-StepGenerate speech with ChatterboxRun high throughput batched transcription with WhisperFine-tune Whisper to recognize new wordsReal-time communication (WebRTC) Serverless WebRTCWebRTC quickstart with FastRTCComputational biology Fold proteins with Chai-1Build a protein-folding dashboardFold proteins with Boltz-2Modal Sandboxes Build a scalable AI coding platformRun a LangGraph agent's code in a secure GPU sandboxControl a sandboxed computer with an LLMBuild a stateful, sandboxed code interpreterRun Node.js, Ruby, and more in a SandboxRun a sandboxed Jupyter notebookReinforcement Learning Train a model to solve math problems using GRPO and verlTrain a model to solve coding problems using GRPO and TRLEmbeddings Embed millions of documents with TEITurn satellite images into vectors and store them in MongoDBParallel processing and job scheduling Transcribe podcasts with WhisperDeploy a Hacker News SlackbotRun a Document OCR job queueServe a Document OCR web appTraining models from scratch Train an SLM with early-stopping grid search over hyperparametersRun long, resumable training jobsHosting popular libraries YOLO: Fine-tune and serve computer vision modelsBlender: Build a 3D render farmStreamlit: Run and deploy Streamlit appsComfyUI: Run Flux on ComfyUI as an APISQLite: Publish explorable data with DatasetteAlgolia: Build docsearch with a crawlerConnecting to other APIs Discord: Deploy and run a Discord BotGoogle Sheets: Sync databases and APIs to a Google SheetOpenAI: Run a RAG Q&A chatbotTailscale: Add Modal Apps to your VPNPrometheus: Publish custom metrics with PushgatewayManaging data Mount S3 buckets in Modal appsBuild your own data warehouse with DuckDB, DBT, and ModalCreate a LoRA Playground with Modal, Gradio, and S3Miscellaneous

Modal Vibe: A scalable AI coding platform

The Modal Vibe repo demonstrates how you can build a scalable AI coding platform on Modal.

Users of the application can prompt an LLM to create sandboxed applications that service React through a UI.

Each application lives on a Modal Sandbox and contains a webserver accessible through Modal Tunnels.

For a high-level overview of Modal Vibe, including performance numbers and why they matter, see the accompanying blog post. For details on the implementation, read on.

How it’s structured 

Architecture diagram for Modal Vibe

  • main.py is the entrypoint that runs the FastAPI controller that serves the web app and manages the sandbox apps.
  • core contains the logic for SandboxApp model and LLM logic.
  • sandbox contains a small HTTP server that gets put inside every Sandbox that’s created, as well as some sandbox lifecycle management code.
  • web contains the Modal Vibe website that users see and interact with, as well as the api server that manages Sandboxes.

How to run 

First, set up the local environment:

python3 -m venv venv && source venv/bin/activate && pip install -r requirements.dev.txt

Deploy 

To deploy to Modal, copy .env.example to a file called .env and add your ANTHROPIC_API_KEY. Also, create a Modal Secret called anthropic-secret so our applications can access it.

Then, deploy the application with Modal:

modal deploy -m main

Local Development 

Run a load test:

modal run main.py::create_app_loadtest_function --num-apps 10

Delete a sandbox:

modal run main.py::delete_sandbox_admin_function --app-id <APP_ID>

Run an example sandbox HTTP server:

python -m sandbox.server