Modal logo
IntroductionCustom container images Defining ImagesUsing existing container imagesFast pull from registryGPUs and other resources GPU accelerationUsing CUDA on ModalConfiguring CPU, memory, and diskScaling out Scaling outInput concurrencyBatch processingJob queuesDynamic batchingMulti-node clusters (beta)Deployment Apps, Functions, and entrypointsManaging deploymentsInvoking deployed functionsContinuous deploymentRunning untrusted code in FunctionsModal Sandboxes SandboxesRunning commandsNetworking and securityFile accessSnapshotsDocker in Sandboxes (Alpha)Modal NotebooksSecrets and environment variables SecretsEnvironment variablesScheduling and cron jobsWeb endpoints Web endpointsStreaming endpointsWeb endpoint URLsRequest timeoutsProxy Auth TokensNetworking TunnelsProxies (beta)Cluster networkingData sharing and storage Passing local dataVolumesStoring model weightsCloud bucket mountsDictsQueuesDataset ingestionPerformance Cold start performanceMemory SnapshotsHigh-performance LLM inferenceGeographic latencyReliability and robustness Failures and retriesPreemptionTimeoutsGPU healthTroubleshootingSecurity and privacyIntegrations Using OIDC to authenticate with external servicesConnecting Modal to your Datadog accountConnecting Modal to your OpenTelemetry providerOkta SSOCustom SAML SSOSlack notifications (beta)Workspace & account settings WorkspacesEnvironmentsModal user account setupService usersRole-Based Access Control (RBAC)BillingOther topics Feature maturityJavaScript/Go SDKsModal 1.0 migration guideFile and project structureDeveloping and debuggingDeveloping Modal code with LLMsJupyter notebooksAsynchronous API usageGlobal variablesRegion selectionContainer lifecycle hooksParametrized functionsS3 Gateway endpointsGPU Metrics

Geographic Latency

Modal’s container cluster is multi-cloud and multi-region. The vast majority of containers are located in the continental USA, but we do run containers across the globe.

By default, all inputs to Modal containers go through our control plane in Virginia, USA (us-east) before being sent to a container for execution. Cloudping.co provides good estimates of the latency between regions. For example, the round-trip latency between AWS us-east (Virginia, USA) and us-west (California, USA) is around 60ms.

You can observe the location identifier of a container via an environment variable. Logging this environment variable alongside latency information can reveal when geography is impacting your application performance.

Optimizing latency 

Modal has a variety of tools to optimize network latency, even down to ~10ms in extreme cases like real-time robotics. One such tool is region selection, which tells Modal to only schedule your container in a certain region, e.g. us-east. Note that inputs still go through our control plane in this setting.

To optimize latency further, please contact us on Slack or at support@modal.com.