Modal logo
FeaturedGetting started Hello, worldSimple web scraperServing web endpointsLarge language models (LLMs) Deploy an OpenAI-compatible LLM service with vLLMCut Ministral 3 cold start times by 10x with snapshotsMaximize tokens per second in batch processing with vLLMServe an ultra-low-latency chatbot with SGLangEfficient LLM Finetuning with UnslothRun a multimodal RAG chatbot to answer questions about PDFsFine-tune an LLM to replace your CEODeploy a stateless MCP with FastMCPImages, video, & 3D Edit images with Flux KontextFine-tune Wan2.1 video models on your faceRun Flux fast with torch.compileFine-tune Flux with LoRAAnimate images with LTX-VideoGenerate video clips with LTX-VideoRun Stable Diffusion with a CLI, API, and web UIAudio Deploy a Moshi voice chatbotStream transcripts at the speed of speech using Kyutai STTMake music with ACE-StepGenerate speech with ChatterboxRun high throughput batched transcription with WhisperFine-tune Whisper to recognize new wordsReal-time communication (WebRTC) Serverless WebRTCWebRTC quickstart with FastRTCComputational biology Fold proteins with Chai-1Build a protein-folding dashboardFold proteins with Boltz-2Modal Sandboxes Run a background coding agent with OpenCodeBuild a scalable AI coding platformRun a LangGraph agent's code in a secure GPU sandboxControl a sandboxed computer with an LLMBuild a stateful, sandboxed code interpreterRun Node.js, Ruby, and more in a SandboxRun a sandboxed Jupyter notebookReinforcement Learning Train a model to solve math problems using GRPO and verlTrain a model to solve coding problems using GRPO and TRLEmbeddings Embed millions of documents with TEITurn satellite images into vectors and store them in MongoDBParallel processing and job scheduling Deploy a Hacker News SlackbotRun a Document OCR job queueServe a Document OCR web appTraining models from scratch Train an SLM with early-stopping grid search over hyperparametersRun long, resumable training jobsHosting popular libraries YOLO: Fine-tune and serve computer vision modelsBlender: Build a 3D render farmStreamlit: Run and deploy Streamlit appsComfyUI: Run Flux on ComfyUI as an APISQLite: Publish explorable data with DatasetteAlgolia: Build docsearch with a crawlerConnecting to other APIs Discord: Deploy and run a Discord BotGoogle Sheets: Sync databases and APIs to a Google SheetOpenAI: Run a RAG Q&A chatbotTailscale: Add Modal Apps to your VPNPrometheus: Publish custom metrics with PushgatewayManaging data Mount S3 buckets in Modal appsBuild your own data warehouse with DuckDB, DBT, and ModalCreate a LoRA Playground with Modal, Gradio, and S3Miscellaneous
View on GitHub

Make music with ACE-Step

In this example, we show you how you can run ACE Studio’s ACE-Step music generation model on Modal.

We’ll set up both a serverless music generation service and a web user interface.

Setting up dependencies 

We start by defining the environment our generation runs in. This takes some explaining since, like most cutting-edge ML environments, it is a bit fiddly.

This environment is captured by a container image, which we build step-by-step by calling methods to add dependencies, like apt_install to add system packages and pip_install to add Python packages.

Note that we don’t have to install anything with “CUDA” in the name — the drivers come for free with the Modal environment and the rest gets installed pip. That makes our life a lot easier! If you want to see the details, check out this guide in our docs.

In addition to source code, we’ll also need the model weights.

ACE-Step integrates with the Hugging Face ecosystem, so setting up the models is straightforward. ACEStepPipeline internally uses the Hugging Face model hub to download the weights if not already present.

But Modal Functions are serverless: instances spin down when they aren’t being used. If we want to avoid downloading the weights every time we start a new instance, we need to store the weights somewhere besides our local filesystem.

So we add a Modal Volume to store the weights in the cloud. For more on storing model weights on Modal, see this guide.

We don’t need to change any of the model loading code — we just need to make sure the model gets stored in the right directory.

To do that, we set an environment variable that Hugging Face expects (and another one that speeds up downloads, for good measure) and then run the load_model Python function.

While we’re at it, let’s also define the environment for our UI. We’ll stick with Python and so use FastAPI and Gradio.

This is a totally different environment from the one we run our model in. Say goodbye to Python dependency conflict hell!

Running music generation on Modal 

Now, we write our music generation logic.

  • We make an App to organize our deployment.
  • We load the model at start, instead of during inference, with modal.enter, which requires that we use a Modal Cls.
  • In the app.cls decorator, we specify the Image we built and attach the Volume. We also pick a GPU to run on — here, an NVIDIA L40S.

We can then generate music from anywhere by running code like what we have in the local_entrypoint below.

You can execute it with a command like:

Pass in --help to see options and how to use them.

Hosting a web UI for the music generator 

With the Gradio library, we can create a simple web UI in Python that calls out to our music generator, then host it on Modal for anyone to try out.

To deploy both the music generator and the UI, run