Modal logo
Introduction Functions Functions Invocation methods Function lookups Container lifecycle management Parametrized Functions Dynamic configuration Sandboxes Sandboxes Running commands Networking and security File access Snapshots Pricing and resources VM Sandboxes BetaSidecars AlphaEndpoints Endpoints Shared Endpoints Dedicated Endpoints Endpoint integrations Images Defining Images Using existing container images Named images Fast pull from registry GPUs and other resources GPU acceleration Using CUDA on Modal Configuring CPU, memory, and disk Scaling out Scaling out Input concurrency Batch processing Job queues Dynamic batching Multi-node clusters BetaDeployment Apps, Functions, and entrypoints Managing deployments Continuous deployment Scheduled Functions HTTP Applications Servers Web Functions Streaming endpoints Web Function URLs Request timeouts Proxy Tokens Networking Tunnels Proxies BetaCluster networking Data sharing and storage Passing local data Volumes Storing model weights Cloud bucket mounts Dicts Queues Dataset ingestion Secrets and environment variables Secrets Environment variables Performance Cold start performance Memory Snapshots High-performance LLM inference Reliability and robustness Failures and retries Preemption Timeouts GPU health Troubleshooting Security and privacy Security and privacy Data residency Customer-supplied encryption keys AlphaAudit logs Integrations Using OIDC to authenticate with external services Connecting Modal to your Datadog account Connecting Modal to your OpenTelemetry provider Okta SSO Microsoft Entra SSO Custom SAML SSO SCIM Integration BetaSlack notifications BetaWorkspace & account settings Workspaces Environments Modal user account setup Service users Role-Based Access Control (RBAC) Billing Network egress billing Budgets Other topics Feature maturity Notebooks Restricted Functions JavaScript/Go SDKs BetaModal 1.0 migration guide File and project structure Developing and debugging Jupyter notebooks Asynchronous API usage Global variables Region selection GPU Metrics

Dedicated Endpoints

Dedicated Endpoints run any model in the Modal Library, or custom weights, on isolated, autoscaling containers. Use them when you need control over autoscaling and regions, or dedicated capacity.

Create a Dedicated Endpoint 

Create an Endpoint from the CLI:

modal endpoint create --model Qwen/Qwen3.5-4B

Modal resolves the model, selects a compatible serving recipe, and starts provisioning. The command prints the Endpoint ID and a dashboard link where you can watch it come online. Dedicated Endpoints can also be created from the Endpoints tab in the dashboard.

If you omit --name, Modal derives a name from the model.

View the generated source 

A Dedicated Endpoint is a Modal App built with the same primitives available in the Modal SDK, including @app.server(). Open the Source view to inspect its generated serve.py. You can copy and adapt that code into your own Modal App when you need full control of the serving stack.

Serve custom weights 

Custom weights use the serving recipe for a compatible Modal Library model. Pass that model with --model, then provide weights from Hugging Face or a Modal Volume.

From Hugging Face:

modal endpoint create \
  --name my-fine-tune \
  --model Qwen/Qwen3.6-27B \
  --custom-hf-repo aisingapore/Qwen-SEA-LION-v4.5-27B-IT \
  --custom-hf-revision da42f2c0984d716fb2032e4176d81adfac98c630

Use --custom-hf-token for gated or private repositories.

From a Modal Volume containing a config.json file:

modal endpoint create \
  --name my-volume-model \
  --model Qwen/Qwen3.5-4B \
  --custom-volume-name my-volume \
  --custom-volume-path /checkpoints/1234

Configure capacity and placement 

By default, Dedicated Endpoints scale up under load and down to zero when idle. Configure minimum, maximum, and buffer containers from the dashboard.

The routing region controls where requests enter Modal. Compute placement controls where containers run. Set them independently:

modal endpoint create \
  --model Qwen/Qwen3.5-4B \
  --routing-region us-east \
  --compute-region us-west

Use --colocate-compute instead to run compute in the routing region. Pinning compute to a region incurs a region selection multiplier.

Metrics 

The Activity view shows request volume over time. Use Responses to inspect individual requests and Containers to inspect the containers serving them.

For text-generation models, Metrics separates Inference metrics—latency, throughput, running and queued requests, cache usage, and speculative decoding—from Server metrics such as autoscaling and CPU, memory, network, and GPU utilization.

Benchmarks 

For text-generation models, the Benchmark view can run a repeatable real-time or agentic workload against the live Endpoint. Benchmarks generate traffic, trigger autoscaling, and incur the usual compute cost.

Treat results as point-in-time measurements: fleet size, placement, and cold starts can all affect them.

Manage a Dedicated Endpoint 

List Endpoints in an Environment:

modal endpoint list --env prod

Stop an Endpoint when you no longer need it:

modal endpoint stop my-endpoint --env prod

Stopping an Endpoint is permanent. It tears down the serving application and deletes its managed model cache; the Endpoint cannot be restarted.

Pricing 

Dedicated Endpoints bill for the GPU, CPU, memory, and other resources used by their containers at standard Modal compute rates. An Endpoint scaled to zero does not incur compute charges.