Reserving CPU and memory

Modal jobs are reserved 128 MiB of memory and 0.125 CPU cores by default. However, if there is free memory or CPU capacity on a worker, containers are free to spike above these limits. You can also guarantee higher limits by reserving more resources.

Billing

For both CPU and memory, you are billed for the greater of (reservation, usage).

Disk requests are billed by increasing the memory request at a 20:1 ratio. For example, requesting 500 GiB of disk will increase the memory request to 25 GiB, if it is not already set higher.

CPU cores

If you have code that must run on a larger number of cores, you can request that using the cpu argument. This allows you to specify a floating-point number of CPU cores:

import modal

app = modal.App()

@app.function(cpu=8.0)
def my_function():
    # code here will have access to at least 8.0 cores
    ...

Memory

If you have code that needs more guaranteed memory, you can request it using the memory argument. This expects an integer number of megabytes:

import modal

app = modal.App()

@app.function(memory=32768)
def my_function():
    # code here will have access to at least 32 GiB of RAM
    ...

How much can I request?

For both CPU and memory, a maximum is enforced at function creation time to ensure your application can be scheduled for execution. Requests exceeding the maximum will be rejected with an InvalidError.

As the platform grows, we plan to support larger CPU and memory reservations.

Resource limits

CPU limits

Modal containers have a soft CPU limit that is set at 4 physical cores above the CPU request. Given that the default CPU request is 0.1 cores the soft CPU limit is 4.1 cores. Above this limit the host will begin to throttle the CPU usage of the container.

Memory limits

Modal containers can have a hard memory limit which will ‘Out of Memory’ (OOM) kill containers which attempt to exceed the limit. This functionality is useful when a container has a serious memory leak. You can set the limit and have the container killed to avoid paying for the leaked GBs of memory.

mem_request = 1024
mem_limit = 2048
@app.function(
    memory=(mem_request, mem_limit),
)
def f():
    ...

Specify this limit using the memory parameter on Modal Functions.

Disk limits

Running Modal containers have access to many GBs of SSD disk, but the amount of writes is limited by:

1. The size of the underlying worker’s SSD disk capacity
2. A per-container disk quota that is set in the 100s of GBs.

Hitting either limit will cause the container’s disk writes to be rejected, which typically manifests as an OSError.

Increased disk sizes can be requested with the ephemeral_disk parameter. The maximum disk size is 3.0 TiB (3,145,728 MiB). Larger disks are intended to be used for dataset processing.