Model APIs
Create your own frontier
Access the latest in open weight intelligence, optimized for performant inference on any workload.
Open intelligence served on production-grade infra
Run open weight models on optimized endpoints with OpenAI-compatible APIs. Backed by the latest inference serving techniques and a multi-cloud compute plane.
'%20d='M24.49%2037.3C38.73%2015.35%2059.28%200%2082.85%200c13.65%200%2027.22%204.04%2041.39%2015.61%2015.5%2012.65%2032.02%2033.48%2052.63%2067.81l7.39%2012.32c17.84%2029.72%2027.99%2045.01%2033.93%2052.22%207.64%209.26%2012.99%2012.02%2019.94%2012.02%2017.63%200%2022.03-16.2%2022.03-34.74l27.4-.86c0%2019.38-3.82%2033.62-10.32%2044.87C270.96%20180.13%20258.72%20191%20238.13%20191c-12.8%200-24.14-2.78-36.68-14.61-9.64-9.08-20.91-25.21-29.58-39.71L146.08%2093.6c-12.94-21.62-24.81-37.74-31.68-45.04-7.39-7.85-16.89-17.33-32.05-17.33-12.27%200-22.69%208.61-31.41%2021.78z'%20/%3e%3cpath%20fill='url(%23meta-vertical-gradient)'%20d='M82.35%2031.23c-12.27%200-22.69%208.61-31.41%2021.78-12.33%2018.61-19.88%2046.33-19.88%2072.95%200%2010.98%202.41%2019.41%205.56%2024.51l-26.48%2017.44C3.34%20156.6%200%20141.76%200%20124.85c0-30.75%208.44-62.8%2024.49-87.55C38.73%2015.35%2059.28%200%2082.85%200z'%20/%3e%3c/svg%3e "Meta")
Deploy the latest open weight models the day they launch.
Get startedDeploy frontier models
Pay only for the tokens you use, with intelligent routing to maximize KV cache hits and minimize cost.
Production-grade quality
Evals verified by leading model labs for production serving performance.
Multi-cloud reliability
Served from Modal's multi-cloud, autoscaling compute pool for consistent reliability.
Deploy the latest open weight models the day they launch.
Get startedDeploy frontier models
Pay only for the tokens you use, with intelligent routing to maximize KV cache hits and minimize cost.
Production-grade quality
Evals verified by leading model labs for production serving performance.
Multi-cloud reliability
Served from Modal's multi-cloud, autoscaling compute pool for consistent reliability.
Production-grade performance
Kernels tuned for inference
Our FlashAttention-4 contributions add KV parallelism and support irregular memory access for decode-heavy workloads.
KV cache-aware routing
Intelligent, stateful routing keeps cache hit rate high for long-running workloads, reducing time to first token and serving costs.
Balance work across replicas
Load-aware placement splits concurrent sessions across replicas and sends new sessions to new capacity during scale-ups.
Designed for security and data privacy
Zero Data Retention (ZDR) by default
Zero data retention means we never store prompts or responses from Model APIs.
No data use
Your data is never used to train models. Inference runs entirely on Modal, so nothing is shared with model labs.
Encrypted in transit
All traffic to and from Model APIs is encrypted in transit with TLS
Built on the core of inference research
Optimizing the FlashAttention-4 kernel
Key/value parallelism and irregular memory access make the kernel faster for decode-heavy inference.
Speculation is all you need
How we train custom speculators for state-of-the-art inference performance.