Modal logo
FeaturedGetting started Hello, worldSimple web scraperServing web endpointsLarge language models (LLMs) Deploy an OpenAI-compatible LLM service with vLLMCut Ministral 3 cold start times by 10x with snapshotsMaximize tokens per second in batch processing with vLLMServe an ultra-low-latency chatbot with SGLangEfficient LLM Finetuning with UnslothRun a multimodal RAG chatbot to answer questions about PDFsFine-tune an LLM to replace your CEODeploy a stateless MCP with FastMCPImages, video, & 3D Edit images with Flux KontextFine-tune Wan2.1 video models on your faceRun Flux fast with torch.compileFine-tune Flux with LoRAAnimate images with LTX-VideoGenerate video clips with LTX-VideoRun Stable Diffusion with a CLI, API, and web UIAudio Deploy a Moshi voice chatbotStream transcripts at the speed of speech using Kyutai STTMake music with ACE-StepGenerate speech with ChatterboxRun high throughput batched transcription with WhisperFine-tune Whisper to recognize new wordsReal-time communication (WebRTC) Serverless WebRTCWebRTC quickstart with FastRTCComputational biology Fold proteins with Chai-1Build a protein-folding dashboardFold proteins with Boltz-2Modal Sandboxes Run a background coding agent with OpenCodeBuild a scalable AI coding platformRun a LangGraph agent's code in a secure GPU sandboxControl a sandboxed computer with an LLMBuild a stateful, sandboxed code interpreterRun Node.js, Ruby, and more in a SandboxRun a sandboxed Jupyter notebookReinforcement Learning Train a model to solve math problems using GRPO and verlTrain a model to solve coding problems using GRPO and TRLEmbeddings Embed millions of documents with TEITurn satellite images into vectors and store them in MongoDBParallel processing and job scheduling Deploy a Hacker News SlackbotRun a Document OCR job queueServe a Document OCR web appTraining models from scratch Train an SLM with early-stopping grid search over hyperparametersRun long, resumable training jobsHosting popular libraries YOLO: Fine-tune and serve computer vision modelsBlender: Build a 3D render farmStreamlit: Run and deploy Streamlit appsComfyUI: Run Flux on ComfyUI as an APISQLite: Publish explorable data with DatasetteAlgolia: Build docsearch with a crawlerConnecting to other APIs Discord: Deploy and run a Discord BotGoogle Sheets: Sync databases and APIs to a Google SheetOpenAI: Run a RAG Q&A chatbotTailscale: Add Modal Apps to your VPNPrometheus: Publish custom metrics with PushgatewayManaging data Mount S3 buckets in Modal appsBuild your own data warehouse with DuckDB, DBT, and ModalCreate a LoRA Playground with Modal, Gradio, and S3Miscellaneous
View on GitHub

Fine-tune open source YOLO models for object detection

Example by @Erik-Dunteman and @AnirudhRahul.

The popular “You Only Look Once” (YOLO) model line provides high-quality object detection in an economical package. In this example, we use the YOLOv10 model, released on May 23, 2024.

We will:

  • Download two custom datasets from the Roboflow computer vision platform: a dataset of cats and a dataset of dogs

  • Fine-tune the model on those datasets, in parallel, using the Ultralytics package

  • Run inference with the fine-tuned models on single images and on streaming frames

For commercial use, be sure to consult the Ultralytics software license options, which include AGPL-3.0.

Set up the environment 

Modal runs your code in the cloud inside containers. So to use it, we have to define the dependencies of our code as part of the container’s image.

We also create a persistent Volume for storing datasets, trained weights, and inference outputs. For more on storing model weights on Modal, see this guide.

We attach both of these to a Modal App.

Download a dataset 

We’ll be downloading our data from the Roboflow computer vision platform, so to follow along you’ll need to:

You’re also free to bring your own dataset with a config in YOLOv10-compatible yaml format.

We’ll be training on the medium size model, but you’re free to experiment with other model sizes.

Train a model 

We train the model on a single A100 GPU. Training usually takes only a few minutes.

Run inference on single inputs and on streams 

We demonstrate two different ways to run inference — on single images and on a stream of images.

The images we use for inference are loaded from the test set, which was added to our Volume when we downloaded the dataset. Each image read takes ~50ms, and inference can take ~5ms, so the disk read would be our biggest bottleneck if we just looped over the image paths. To avoid it, we parallelize the disk reads across many workers using Modal’s .map, streaming the images to the model. This roughly mimics the behavior of an interactive object detection pipeline. This can increase throughput up to ~60 images/s, or ~17 milliseconds/image, depending on image size.

We use the @enter feature of modal.Cls to load the model only once on container start and reuse it for future inferences. We use a generator to stream images to the model.

Running the example 

We’ll kick off our parallel training jobs and run inference from the command line.

This runs the training in quick_check mode, useful for debugging the pipeline and getting a feel for it. To do a longer run that actually meaningfully improves performance, use:

Addenda 

The rest of the code in this example is utility code.