Modal logo
FeaturedGetting started Hello, worldSimple web scraperServing web endpointsLarge language models (LLMs) Deploy an OpenAI-compatible LLM service with vLLMCut Ministral 3 cold start times by 10x with snapshotsMaximize tokens per second in batch processing with vLLMServe an ultra-low-latency chatbot with SGLangEfficient LLM Finetuning with UnslothRun a multimodal RAG chatbot to answer questions about PDFsFine-tune an LLM to replace your CEODeploy a stateless MCP with FastMCPImages, video, & 3D Edit images with Flux KontextFine-tune Wan2.1 video models on your faceRun Flux fast with torch.compileFine-tune Flux with LoRAAnimate images with LTX-VideoGenerate video clips with LTX-VideoRun Stable Diffusion with a CLI, API, and web UIAudio Deploy a Moshi voice chatbotStream transcripts at the speed of speech using Kyutai STTMake music with ACE-StepGenerate speech with ChatterboxRun high throughput batched transcription with WhisperFine-tune Whisper to recognize new wordsReal-time communication (WebRTC) Serverless WebRTCWebRTC quickstart with FastRTCComputational biology Fold proteins with Chai-1Build a protein-folding dashboardFold proteins with Boltz-2Modal Sandboxes Run a background coding agent with OpenCodeBuild a scalable AI coding platformRun a LangGraph agent's code in a secure GPU sandboxControl a sandboxed computer with an LLMBuild a stateful, sandboxed code interpreterRun Node.js, Ruby, and more in a SandboxRun a sandboxed Jupyter notebookReinforcement Learning Train a model to solve math problems using GRPO and verlTrain a model to solve coding problems using GRPO and TRLEmbeddings Embed millions of documents with TEITurn satellite images into vectors and store them in MongoDBParallel processing and job scheduling Deploy a Hacker News SlackbotRun a Document OCR job queueServe a Document OCR web appTraining models from scratch Train an SLM with early-stopping grid search over hyperparametersRun long, resumable training jobsHosting popular libraries YOLO: Fine-tune and serve computer vision modelsBlender: Build a 3D render farmStreamlit: Run and deploy Streamlit appsComfyUI: Run Flux on ComfyUI as an APISQLite: Publish explorable data with DatasetteAlgolia: Build docsearch with a crawlerConnecting to other APIs Discord: Deploy and run a Discord BotGoogle Sheets: Sync databases and APIs to a Google SheetOpenAI: Run a RAG Q&A chatbotTailscale: Add Modal Apps to your VPNPrometheus: Publish custom metrics with PushgatewayManaging data Mount S3 buckets in Modal appsBuild your own data warehouse with DuckDB, DBT, and ModalCreate a LoRA Playground with Modal, Gradio, and S3Miscellaneous
View on GitHub

Animate images with Lightricks LTX-Video via CLI, API, and web UI

This example shows how to run LTX-Video on Modal to generate videos from your local command line, via an API, and in a web UI.

Generating a 5 second video takes ~1 minute from cold start. Once the container is warm, a 5 second video takes ~15 seconds.

Here is a sample we generated:

Basic setup 

All Modal programs need an App — an object that acts as a recipe for the application.

Configuring dependencies 

The model runs remotely, on Modal’s cloud, which means we need to define the environment it runs in.

Below, we start from a lightweight base Linux image and then install our system and Python dependencies, like Hugging Face’s diffusers library and torch.

Storing model weights on Modal 

We also need the parameters of the model remotely. They can be loaded at runtime from Hugging Face, based on a repository ID and a revision (aka a commit SHA).

Hugging Face will also cache the weights to disk once they’re downloaded. But Modal Functions are serverless, and so even disks are ephemeral, which means the weights would get re-downloaded every time we spin up a new instance.

We can fix this — without any modifications to Hugging Face’s model loading code! — by pointing the Hugging Face cache at a Modal Volume. For more on storing model weights on Modal, see this guide.

Storing model outputs on Modal 

Contemporary video models can take a long time to run and they produce large outputs. That makes them a great candidate for storage on Modal Volumes as well. Python code running outside of Modal can also access this storage, as we’ll see below.

Implementing LTX-Video inference on Modal 

We wrap the inference logic in a Modal Cls that ensures models are loaded and then moved to the GPU once when a new instance starts, rather than every time we run it.

The run function just wraps a diffusers pipeline. It saves the generated video to a Modal Volume, and returns the filename.

We also include a web wrapper that makes it possible to trigger inference via an API call. For details, see the /docs route of the URL ending in inference-web.modal.run that appears when you deploy the app.

Generating videos from the command line 

We add a local entrypoint that calls the Inference.run method to run inference from the command line. The function’s parameters are automatically turned into a CLI.

Run it with

You can also pass --help to see the full list of arguments.

Generating videos via an API 

The Modal Cls above also included a fastapi_endpoint, which adds a simple web API to the inference method.

To try it out, run

copy the printed URL ending in inference-web.modal.run, and add /docs to the end. This will bring up the interactive Swagger/OpenAPI docs for the endpoint.

Generating videos in a web UI 

Lastly, we add a simple front-end web UI (written in Alpine.js) for our image to video backend.

This is also deployed when you run

The Inference class will serve multiple users from its own auto-scaling pool of warm GPU containers automatically, and they will spin down when there are no requests.