Modal logo
IntroductionCustom container images Defining ImagesUsing existing container imagesFast pull from registryGPUs and other resources GPU accelerationUsing CUDA on ModalReserving CPU and memoryScaling out Scaling outInput concurrencyBatch processingJob queuesDynamic batching (beta)Multi-node clusters (beta)Deployment Apps, Functions, and entrypointsManaging deploymentsInvoking deployed functionsContinuous deploymentRunning untrusted code in FunctionsModal Sandboxes SandboxesRunning commandsNetworking and securityFile accessSnapshotsModal NotebooksSecrets and environment variables SecretsEnvironment variablesScheduling and cron jobsWeb endpoints Web endpointsStreaming endpointsWeb endpoint URLsRequest timeoutsProxy Auth TokensNetworking TunnelsProxies (beta)Cluster networkingData sharing and storage Passing local dataVolumesStoring model weightsCloud bucket mountsDictsQueuesDataset ingestionPerformance Cold start performanceMemory SnapshotGeographic latencyReliability and robustness Failures and retriesPreemptionTimeoutsGPU healthTroubleshootingSecurity and privacyIntegrations Using OIDC to authenticate with external servicesConnecting Modal to your Datadog accountConnecting Modal to your OpenTelemetry providerOkta SSOSlack notifications (beta)Workspace & account settings WorkspacesEnvironmentsModal user account setupService usersOther topics JavaScript/Go SDKsModal 1.0 migration guideFile and project structureDeveloping and debuggingDeveloping Modal code with LLMsJupyter notebooksAsynchronous API usageGlobal variablesRegion selectionContainer lifecycle hooksParametrized functionsS3 Gateway endpointsGPU Metrics

Preemption

All Modal Functions are subject to preemption by default. If a preemption event interrupts a running Function, Modal will gracefully terminate the Function and restart it on the same input.

Preemptions are rare, but it is always possible that your Function is interrupted. Long-running Functions such as model training Functions should take particular care to tolerate interruptions, as likelihood of interruption increases with Function run duration.

Preparing for interruptions 

Design your applications to be fault and preemption tolerant. Modal will send an interrupt signal to your container when preemption occurs. This will cause the Function’s exit handler to run, which can perform any cleanup within its grace period.

Other best practices for handling preemptions include:

  • Divide long-running operations into small tasks or use checkpoints so that you can save your work frequently.
  • Ensure preemptible operations are safely retryable (ie. idempotent).

Non-preemptible Functions 

If you require Functions that are guaranteed not to be preempted, you may set the nonpreemptible parameter (available starting in client version v1.2.3) to True in the @app.function() or @app.cls() decorator. Note that a 3x multiplier will be applied to the list price for CPU and Memory usage when nonpreemptible is set to True.

Note: The nonpreemptible parameter is not supported for GPU Functions.

Non-preemptible Sandboxes 

Modal Sandboxes are not subject to preemption, except in the case where a region, cloud, or gpu requirement is specified. This is because of availability and scheduling latency constraints.