Modal logo
Guide Introduction to Modal Getting started Functions Functions Invocation methods Function lookups Container lifecycle management Parametrized Functions Dynamic configuration Sandboxes Sandboxes Running commands Networking and security File access Snapshots Exit Snapshots BetaPricing and resources Sidecars BetaSecret injection BetaEndpoints Endpoints Supported models Shared Endpoints Dedicated Endpoints Endpoint candidates Endpoint integrations Training Quickstart Tinker-compatible API BetaMulti-node training Images Defining Images Using existing container images Named images Fast pull from registry GPUs and other resources GPU acceleration Using CUDA on Modal Configuring CPU, memory, and disk Scaling out Scaling out Input concurrency Batch processing Job queues Dynamic batching Multi-node Clusters Deployment Apps, Functions, and entrypoints Managing deployments Continuous deployment Scheduled Functions HTTP Applications Servers Sticky Sessions Web Functions Streaming endpoints Web Function URLs Request timeouts Proxy Tokens Networking Tunnels Proxies BetaCluster networking Data sharing and storage Passing local data Volumes Storing model weights Cloud bucket mounts Dicts Queues Dataset ingestion Secrets and environment variables Secrets Environment variables Performance Cold start performance Memory Snapshots High-performance LLM inference Reliability and robustness Failures and retries Preemption Timeouts GPU health Troubleshooting Security and privacy Security and privacy Data residency Customer-supplied encryption keys AlphaAudit logs Integrations Using OIDC to authenticate with external services Connecting Modal to your Datadog account Connecting Modal to your OpenTelemetry provider Slack notifications BetaWorkspace & account settings Workspaces Environments Service users User groups Role-Based Access Control (RBAC) Billing Network egress billing Budgets Identity and provisioning Okta SSO Microsoft Entra SSO Google Workspace SSO Custom SAML SSO SCIM Integration Other topics Feature maturity Restricted Functions JavaScript/Go SDKs BetaModal 1.0 migration guide File and project structure Developing and debugging Jupyter notebooks Asynchronous API usage Global variables Region selection GPU Metrics Search syntax

Shared Endpoints

Shared Endpoints serve a subset of supported models from Modal-managed pools. Modal manages the hardware and autoscaling, and usage is billed per token.

Create a Shared Endpoint 

Pass --shared to create a Shared Endpoint from the CLI:

modal endpoint create --shared --model moonshotai/Kimi-K3

Shared Endpoints can also be created from the Endpoints tab in the dashboard.

Once the Endpoint is ready, the dashboard provides its URL, model name, and request examples. Shared Endpoints always require a proxy token.

Concurrency limits 

Shared Endpoints have model-specific limits on concurrent in-flight requests. The limit is shared across all Shared Endpoints using the same model in your Workspace. Requests above the limit receive an HTTP 429 response; retry them with exponential backoff and jitter.

Use a Dedicated Endpoint when you need isolated or configurable capacity.

Request routing 

Shared Endpoints route each request to the container most likely to have its prompt prefix cached. For multi-turn conversations and agent loops, send a Modal-Routing-Affinity-Key header with every request of a conversation so its requests stay on the same container. See Affinity routing for how to choose the header value.

Pricing 

Token rates for each model are listed in supported models, and are shown when you create an Endpoint and in its Usage view. Credits included with your plan cannot be used for Shared Endpoint usage. To cap out-of-pocket charges, see spend limits.

Shared Endpoint requests route through us-west and always require authentication. Modal manages their infrastructure and capacity.