📷 Faster memory snapshotting

Memory snapshotting, our beta feature for improving cold starts, just got a major upgrade. For models like Stable Diffusion, this can cut cold start times by half! We’ve made significant optimizations to both speed and consistency.

🕰️ Async job queues

We’re expanding Modal’s capabilities for batch processing and job queues. Soon you’ll be able to queue up to 1 million inputs (up from the current 2k limit). Want early access? We’re looking for beta testers - please reach out at support@modal.com.

🔑 OpenID Connect (OIDC)

You can now use short-lived identity tokens to authenticate against external resources like S3 buckets or secret managers. Where available, OIDC can be used in place of Modal Secrets, allowing you to avoid storing long-lived credentials.

⭐ Faster builds with eStargz

Modal now supports the eStargz compression format, significantly speeding up image builds from external registries like ECR and Docker Hub. Enable it by adding a few flags to your docker buildx build command.

👩‍💻 Client Updates

Run pip install --upgrade modal to get the latest updates. Here are some highlights:

• CLI: modal launch now accepts a --detach flag to persist apps after client disconnect
• Sandbox: exec now supports streaming output with text and bufsize arguments
• Sandbox: exec adds timeout, workdir, and secrets parameters

🚀 New model partnerships

We’re making it easier to run open-source models. Recent collaborations include:

• Genmo’s Mochi 1 - Fine-tuning a SOTA text-to-video model
• Chai Discovery’s Chai-1 - Molecular structure prediction model

🤝 AWS strategic partnership

We’ve announced a multi-year strategic partnership with AWS to provide flexible, instantly accessible GPUs. You can now use Modal to draw down AWS marketplace committed spend. Coming soon: bonus Modal credits for AWS Activate startups!

📺 New video content

Our YouTube channel is live with tutorials on:

• Running ComfyUI in production
• Setting up vLLM inference servers
• And more!

🍭 Fun tidbits

• Our “Modal Wrapped” revealed: 40B+ inputs processed across 7M+ apps in 2024!

• We partnered with Cometeer for branded coffee pod giveaways. Follow us on Twitter for future promotions!