Maintain a pool of warm Sandboxes that are healthy and ready to serve requests

This example demonstrates how to build a pool of “warm” Modal Sandboxes, and deploy a Modal web endpoint that let’s you claim a Sandbox from the pool, getting a URL to the server running in the Sandbox.

Maintaining a pool of warm Sandboxes is useful for example if your Sandboxes need to do significant work after being created, like downloading code, installing dependencies, or running tests, before they are ready to serve requests.

It uses a Modal Queue to store references to the warm Sandboxes, and functionality to maintain the pool by adding and removing Sandboxes, checking the current size, etc.

The pool keeps track of the time to live for each Sandbox, and will always return a Sandbox with enough time left.

It’s structured into two Apps:

example-sandbox-pool is the main App that contains all the control logic for maintaining the pool, exposing ways to claim Sandboxes, etc.
example-sandbox-pool-sandboxes houses all the actual Sandboxes, and nothing else.

The implementation borrows from pawalt’s Sandbox pool example gist. 🙏

Here we define the image that will be used to run the server that runs in the Sandbox. In this simple example, we just run the built in Python HTTP server that returns a directory listing.

In this example Sandboxes live for 5 minutes, and we assume that they are used for 2 minutes, meaning that if a Sandbox has less than 2 minutes left it’s considered to be expiring too soon and will be terminated.

You’ll want to adjust these values depending on your use case.

Main implementation

We keep track of all warm Sandboxes in a Modal Queue of SandboxReference objects.

Health check

We add a simple health check that just ensures that the server in the Sandbox is running and responding to requests.

If you just want to ensure the sandbox is running you could for example check sb.poll() is not None instead.

Adding a Sandbox to the pool

This function creates and adds a new Sandbox to the pool. It runs a health check on the Sandbox before adding it.

We deploy the Sandboxes in a separate Modal App called example-sandbox-pool-sandboxes, to separate the control app (logs, etc.) from the Sandboxes.

We also have a utility function that can be .spawn()ed to terminate Sandboxes.

Claiming a Sandbox from the pool

We expose two ways to claim a Sandbox from the pool and get a URL to the server:

a web endpoint
a Function that can be called using the Modal SDK for Python, Go, or JS.

The web endpoint is deployed as a Modal web endpoint, and calls the claim_sandbox Function using claim_sandbox.local(), meaning that it’s called in the same process as the web endpoint.

The Function can be called using the Modal SDK for Python, Go, or JS.

Maintaining the pool

This function grows or shrinks the pool to SANDBOX_POOL_SIZE. It first removes any expiring or unhealthy sandboxes, then adjusts the pool size to reach the target.

It runs on a schedule to ensure the pool doesn’t drift too far from the target size.

Local commands for interacting with the pool

Deploy the app

This also runs the maintain_pool function to ensure the pool is at the correct size without having to wait for the first scheduled maintenance run.

Run it with python 13_sandboxes/sandbox_pool.py deploy.

Check the current state of the pool

Run it with python 13_sandboxes/sandbox_pool.py check.

Claiming a Sandbox from the pool and print its URL

This is implemented as if you wanted to call the Function from a Python backend application using the Modal SDK, i.e. using .from_name() to get the Function, etc.

Run it with python 13_sandboxes/sandbox_pool.py claim.

Run a demo of the Sandbox pool.