Modal logo
FeaturedGetting started Hello, worldSimple web scraperServing web endpointsLarge language models (LLMs) Deploy an OpenAI-compatible LLM service with vLLMCut Ministral 3 cold start times by 10x with snapshotsMaximize tokens per second in batch processing with vLLMServe an ultra-low-latency chatbot with SGLangEfficient LLM Finetuning with UnslothRun a multimodal RAG chatbot to answer questions about PDFsFine-tune an LLM to replace your CEODeploy a stateless MCP with FastMCPImages, video, & 3D Edit images with Flux KontextFine-tune Wan2.1 video models on your faceRun Flux fast with torch.compileFine-tune Flux with LoRAAnimate images with LTX-VideoGenerate video clips with LTX-VideoRun Stable Diffusion with a CLI, API, and web UIAudio Deploy a Moshi voice chatbotStream transcripts at the speed of speech using Kyutai STTMake music with ACE-StepGenerate speech with ChatterboxRun high throughput batched transcription with WhisperFine-tune Whisper to recognize new wordsReal-time communication (WebRTC) Serverless WebRTCWebRTC quickstart with FastRTCComputational biology Fold proteins with Chai-1Build a protein-folding dashboardFold proteins with Boltz-2Modal Sandboxes Run a background coding agent with OpenCodeBuild a scalable AI coding platformRun a LangGraph agent's code in a secure GPU sandboxControl a sandboxed computer with an LLMBuild a stateful, sandboxed code interpreterRun Node.js, Ruby, and more in a SandboxRun a sandboxed Jupyter notebookReinforcement Learning Train a model to solve math problems using GRPO and verlTrain a model to solve coding problems using GRPO and TRLEmbeddings Embed millions of documents with TEITurn satellite images into vectors and store them in MongoDBParallel processing and job scheduling Deploy a Hacker News SlackbotRun a Document OCR job queueServe a Document OCR web appTraining models from scratch Train an SLM with early-stopping grid search over hyperparametersRun long, resumable training jobsHosting popular libraries YOLO: Fine-tune and serve computer vision modelsBlender: Build a 3D render farmStreamlit: Run and deploy Streamlit appsComfyUI: Run Flux on ComfyUI as an APISQLite: Publish explorable data with DatasetteAlgolia: Build docsearch with a crawlerConnecting to other APIs Discord: Deploy and run a Discord BotGoogle Sheets: Sync databases and APIs to a Google SheetOpenAI: Run a RAG Q&A chatbotTailscale: Add Modal Apps to your VPNPrometheus: Publish custom metrics with PushgatewayManaging data Mount S3 buckets in Modal appsBuild your own data warehouse with DuckDB, DBT, and ModalCreate a LoRA Playground with Modal, Gradio, and S3Miscellaneous
View on GitHub

Run Flux on ComfyUI as an API

In this example, we show you how to turn a ComfyUI workflow into a scalable API endpoint.

Quickstart 

To run this simple text-to-image Flux Schnell workflow as an API:

  1. Deploy ComfyUI behind a web endpoint:
  1. In another terminal, run inference:

example comfyui image

The first inference will take ~1m since the container needs to launch the ComfyUI server and load Flux into memory. Successive calls on a warm container should take a few seconds.

Installing ComfyUI 

We use comfy-cli to install ComfyUI and its dependencies.

Downloading custom nodes 

We’ll also use comfy-cli to download custom nodes, in this case the popular WAS Node Suite.

Use the ComfyUI Registry to find the specific custom node name to use with this command.

See this post for more examples on how to install popular custom nodes like ComfyUI Impact Pack and ComfyUI IPAdapter Plus.

Downloading models 

comfy-cli also supports downloading models, but we’ve found it’s faster to use hf_hub_download directly by:

  1. Enabling faster downloads
  2. Mounting the cache directory to a Volume

By persisting the cache to a Volume, you avoid re-downloading the models every time you rebuild your image. For more on storing model weights on Modal, see this guide.

Lastly, copy the ComfyUI workflow JSON to the container.

Running ComfyUI interactively 

Spin up an interactive ComfyUI server by wrapping the comfy launch command in a Modal Function and serving it as a web server.

At this point you can run modal serve 06_gpu_and_ml/comfyui/comfyapp.py and open the UI in your browser for the classic ComfyUI experience.

Remember to close your UI tab when you are done developing. This will close the connection with the container serving ComfyUI and you will stop being charged.

Running ComfyUI as an API 

To run a workflow as an API:

  1. Stand up a “headless” ComfyUI server in the background when the app starts.

  2. Define an infer method that takes in a workflow path and runs the workflow on the ComfyUI server.

  3. Create a web handler api as a web endpoint, so that we can run our workflow as a service and accept inputs from clients.

We group all these steps into a single Modal cls object, which we’ll call ComfyUI.

This serves the workflow_api.json in this repo. When deploying your own workflows, make sure you select the “Export (API)” option in the ComfyUI menu:

comfyui menu

More resources