Modal logo
IntroductionCustom container images Custom containersPrivate registriesGPUs and other resources GPU accelerationUsing CUDA on ModalReserving CPU and memoryScaling out Scaling outDicts and queuesJob processingConcurrent inputs on a single container (beta)Secrets and environment variables SecretsEnvironment variablesDeployment Apps, Stubs, and entrypointsManaging deploymentsInvoke deployed functionsContinuous deploymentScheduling and cron jobsWeb endpoints Web endpointsStreaming endpointsWeb endpoint URLsRequest timeoutsTunnels (beta)Data sharing and storage Passing local dataVolumesMounting local files and directoriesStoring model weightsDataset ingestionCloud bucket mounts (Beta)Network file systems (superseded)Dynamic sandboxesPerformance Cold start performanceGeographic latencyReliability and robustness Failures and retriesPreemptionTimeoutsTroubleshootingSecurity and privacyIntegrations Connecting Modal to your Vercel accountConnecting Modal to your Datadog accountConnecting Modal to your OpenTelemetry providerOkta SSOOther topics File and project structureDeveloping and debuggingModal user account setupWorkspacesEnvironmentsJupyter notebooksAsynchronous API usageGlobal variablesRegion selectionContainer lifecycle and parametersS3 Gateway endpoints

Reserving CPU and memory

Modal jobs are reserved 128 MiB of memory and 0.1 CPU cores by default. However, if there is free memory or CPU capacity on a worker, containers are free to spike above these limits.

Keep in mind that these reservations should be set to the minimum value required to have your application function correctly.

CPU cores

If you have code that must run on a larger number of cores, you can request that using the cpu argument. This allows you to specify a floating-point number of CPU cores:

import modal

app = modal.App()

@app.function(cpu=8.0)
def my_function():
    # code here will have access to at least 8.0 cores
    ...

Memory

If you have code that needs more guaranteed memory, you can request it using the memory argument. This expects an integer number of megabytes:

import modal

app = modal.App()

@app.function(memory=32768)
def my_function():
    # code here will have access to at least 32 GiB of RAM
    ...

How much can I request?

For both CPU and memory, a maximum is enforced at function creation time to ensure your application can be scheduled for execution. Requests exceeding the maximum will be rejected with an InvalidError.

As the platform grows, we plan to support larger CPU and memory reservations.

Resource limits

CPU limits

Modal containers have a soft CPU limit that is set at 4 physical cores above the CPU request. Given that the default CPU request is 0.1 cores the soft CPU limit is 4.1 cores. Above this limit the host will begin to throttle the CPU usage of the container.

Memory limits

Modal containers can have a hard memory limit which will ‘Out of Memory’ (OOM) kill containers which attempt to exceed the limit. This functionality is useful when a container has a serious memory leak. You can set the limit and have the container killed to avoid paying for the leaked GBs of memory.

mem_request = 1024
mem_limit = 2048
@app.function(
    memory=(mem_request, mem_limit),
)
def f():
    ...

Specify this limit using the memory parameter on Modal Functions.

Disk limits

Running Modal containers have access to many GBs of SSD disk, but the amount of writes is limited by:

  1. The size of the underlying worker’s SSD disk capacity
  2. A per-container disk quota that is set in the 100s of GBs.

Hitting either limit will cause the container’s disk writes to be rejected, which typically manifests as an OSError.

Increased disk sizes can be requested with the ephemeral_disk parameter. The maximum disk size is 3.0 TiB (3,145,728 MiB). Larger disks are intended to be used for dataset processing. Be aware that requesting larger disk size prevents also specifying other configuration requests (RAM, GPU, region).