Job queues & batch processing
An infinitely scalable job queue and a batch job system that can provision thousands of containers on a schedule, all on Modal.
Get startedSpawn tasks from your backend and poll for the results later. No need to manage Celery or your own queue.
Modal functions can be run on a schedule with a single line of code. Configure retries, timeouts and back-off in the same place.
@stub.function(
schedule=modal.Period(days=1),
retries=3,
timeout=600,
)
def perform_heavy_computation():
...Distribute your work over thousands of containers with a call to .map(). Monitor and debug your runs from the unified web interface.
Modal allows you to chain together functions that have disparate hardware requirements and image definitions. Go from running a GPU endpoint to an entire pipeline with pre- and post-processing that runs on CPU-only containers.
Try it out
Document OCR job queue
Use Modal as a job queue to run OCR transcription on documents on-demand.
DuckDB parallel processing
Analyze the NYC taxi dataset in parallel with DuckDB.
Hackernews Slack Alerts
Deploy a cron job that sends Slack alerts when certain terms are mentioned on Hackernews.
“Modal has made it incredibly easy for a team of researchers and scientists to deploy data science and data processing pipelines on cloud workloads. It means that it’s possible for an R&D team to own the software that scientists are developing, as opposed to handing it off to a team of engineers working on infrastructure.”