🚀 Introducing L40S GPUs

NVIDIA L40S GPUs are now available on Modal at $1.95/hr! With 48GB of DDR6 RAM and impressive CUDA and Tensor Core performance, the L40S offers significant advantages over our popular A10 GPUs:

• 2x more memory for running larger models and longer contexts
• Up to 40% faster for memory-bound tasks
• Over 100% speedup for compute-bound jobs using 16bit Tensor Cores

Try it now by adding this decorator to your function: @app.function(gpu="L40S")

🔒 Proxy Auth Tokens

Modal now supports Proxy Auth tokens for authenticating access to web endpoints! This means you can gate access to web endpoints and prevent unwanted usage from incurring charges.

@app.function(gpu="h100")
@modal.web_endpoint(requires_proxy_auth=True, docs=False)
def expensive_secret():
    return "I didn't care for 'The Godfather'. It insists upon itself."

📷 File System API and Disk Snapshotting for Sandboxes

The new Filesystem API makes it seamless to read and write files in your Sandbox, and is especially good for getting files in and out of a Sandbox interactively.

sb = modal.Sandbox.create(app=app)

filepath = "/home/foo.bin"
with sb.open(filepath, "wb") as f:
    f.write(b"Hello, World!")

f = sb.open(filepath)
print(f.read())

We are also introducing disk capabilities for Sandboxes, expanding on our existing function snapshotting feature. This enables you to:

1. Create snapshots of your Sandbox’s entire state
2. Branch off from any snapshot to create new Sandbox instances
3. Eliminate cold-start times by restoring from snapshots

👩‍💻 Client Updates

Run pip install --upgrade modal to get the latest updates. Here are some of the highlights:

• Images: When using Image.from_dockerfile() or image.dockerfile_commands(), the system will now automatically look for and use a .dockerignore file.
• Images:FilePatternMatcher has a new constructor from_file which allows you to read file matching patterns from a file instead of having to pass them in directly.
• Volumes: Modal Volumes can now be renamed via the CLI (modal volume rename) or SDK (modal.Volume.rename).
• Sandboxes: Sandboxes now support fsnotify-like file watching and accept larger write payloads up to 1 GiB
• Environment: TheApp.run context manager has a new environment_name parameter.
• VSCode: You can now point modal launch vscode at an arbitrary Dockerhub base image:

modal launch vscode --image=nvidia/cuda:12.4.0-devel-ubuntu22.04

🔐 SOC 2 Type 2 Certification

We’re pleased to announce the completion of our SOC 2 Type 2 certification. If you would like to see the report or have more questions, please email security@modal.com.

📚 GPU Glossary

We work a lot with GPUs, and if you do too you probably know how hard it can be to find the information you need in the public documentation. So we put together a handy GPU Glossary that collects together quick explanations and high-quality resources for everything from Tensor Cores and Warp Schedulers to Compute Capabilities and the CUDA Toolkit.

🧬 New computational bio, OCR, and image diffusion resources

• ESM3: recent model from Evolutionary Scale that can not only predict protein structures from sequences but also generate new proteins. Protein folding dashboard example on Modal.
• GOT: a 580M parameter OCR model that can better handle a variety of content formats. Example on Modal.
• ICYMI, we hosted a webinar recently covering best practices on productionizing diffusion models. Here’s the video.

🍭 Fun Tidbits

• We’ve been hosting exclusive dinners for biotech founders and engineers! Reach out if you’d like to join our next one.