Web endpoints
Modal gives you a few ways to expose functions as web endpoints. You can either turn any Modal function into a web endpoint with a single line of code, or you can serve an entire ASGI-compatible app (including existing apps written with frameworks such as FastAPI or Flask).
→ All Modal web endpoints have a limit of 100 requests per second (rps). Get in touch if you need higher limits.
Note that if you wish to invoke a Modal function from another Python application, you could deploy and invoke the function using the native client library, without having to set up a web endpoint.
@web_endpoint
The easiest way to create a web endpoint out of an existing function is to use
the @modal.web_endpoint
decorator.
from modal import Stub, web_endpoint
stub = Stub()
@stub.function()
@web_endpoint()
def f():
return "Hello world!"
This decorator wraps the Modal function in a FastAPI application.
Developing with modal serve
You can run this code as an ephemeral app, by running the command
modal serve server_script.py
Where server_script.py
is the file name of your code. This will create an
ephemeral app for the duration of your script (until you hit Ctrl-C to stop it).
It creates a temporary URL that you can use like any other REST endpoint. This
URL is on the public internet.
The serve
command (which you can also call programmatically using
stub.serve()
, will live update an app when any of the supporting files change.
Live updating is particularly useful when working with apps containing web endpoints, as any changes made to web endpoint handlers will show up almost immediately, without requiring a manual restart of the app.
Deploying a web server
You can also deploy your app and create a persistent web endpoint in the cloud
by running modal deploy
:
Passing arguments to web endpoints
When using @web_endpoint
, you can use
query parameters just like in FastAPI
which will be passed to your function as arguments. For instance
from modal import Stub, web_endpoint
stub = Stub()
@stub.function()
@web_endpoint()
def square(x: int):
return {"square": x**2}
If you hit this with an urlencoded query string with the “x” param present, it will send that to the function:
% curl 'https://modal-labs--web-endpoint-get-py-square-erikbern-dev.modal.run?x=42'
{"square":1764}
If you want to use a POST
request, you can use the method
argument to
@web_endpoint
to set the HTTP verb. To accept any valid JSON, you can
use Dict
as your type annotation
and FastAPI will handle the rest.
from typing import Dict
from modal import Stub, web_endpoint
stub = Stub()
@stub.function()
@web_endpoint(method="POST")
def square(item: Dict):
return {"square": item['x']**2}
This now creates an endpoint that lets us hit it using JSON:
% curl 'https://modal-labs--web-endpoint-post-py-square-erikbern-dev.modal.run' -X POST -H 'Content-Type: application/json' -d '{"x": 42}'
{"square":1764}
This is often the easiest way to get started, but note that FastAPI recommends that you use typed Pydantic models in order to get automatic validation and documentation. FastAPI also lets you pass data to web endpoints in other ways, for instance as form data and file uploads.
More configuration
In addition to the keyword arguments supported by a regular stub.function
, web
endpoints take an optional method
argument to set the HTTP method of the REST
endpoint (see reference for a full list of
supported arguments).
How do web endpoints run in the cloud?
Note that web endpoints, like everything else on Modal, only run when they need to. When you hit the web endpoint the first time, it will boot up the container, which might take a few seconds. Modal keeps the container alive for a short period in case there are subsequent requests. If there are a lot of requests, Modal might create more containers running in parallel.
Under the hood, Modal wraps your function in a FastAPI application, and so functions you write need to follow the same request and response semantics as FastAPI. This also means you can use all of FastAPI’s powerful features, such as Pydantic models for automatic validation, typed query and path parameters, and response types.
For long running web endpoints (taking more than 150s to complete), Modal by default uses chains of HTTP redirects to keep each request reasonably short lived. For more information see Web endpoint timeouts.
More complex example
Here’s everything together, combining Modal’s abilities to run functions in user-defined containers with the expressivity of FastAPI:
from pydantic import BaseModel
from fastapi.responses import HTMLResponse
from modal import Image, Stub, web_endpoint
image = Image.debian_slim().pip_install("boto3")
stub = Stub(image=image)
class Item(BaseModel):
name: str
qty: int = 42
@stub.function()
@web_endpoint(method="POST")
def f(item: Item):
import boto3
# do things with boto3...
return HTMLResponse(f"<html>Hello, {item.name}!</html>")
This endpoint definition would be called like so:
curl -d '{"name": "Erik", "qty": 10}' \
-H "Content-Type: application/json" \
-X POST https://ecorp--web-demo-f-dev.modal.run
Or in Python with the requests
library:
import requests
data = {"name": "Erik", "qty": 10}
requests.post("https://ecorp--web-demo-f-dev.modal.run", json=data, timeout=10.0)
Serving ASGI and WSGI apps
You can also serve any app written in an ASGI or WSGI compatible web application framework on Modal.
ASGI provides support for async web applications. WSGI provides support for synchronous web applications.
ASGI
For ASGI apps, you can create a function decorated with
@modal.asgi_app
that returns a reference to
your web app:
from fastapi import FastAPI, Request
from fastapi.responses import HTMLResponse
from modal import Image, Stub, asgi_app
web_app = FastAPI()
stub = Stub()
image = Image.debian_slim().pip_install("boto3")
@web_app.post("/foo")
async def foo(request: Request):
body = await request.json()
return body
@web_app.get("/bar")
async def bar(arg="world"):
return HTMLResponse(f"<h1>Hello Fast {arg}!</h1>")
@stub.function(image=image)
@asgi_app()
def fastapi_app():
return web_app
Now, as before, when you deploy this script as a modal app, you get a URL for your app that you can use:
WSGI
You can serve WSGI apps using the
@modal.wsgi_app
decorator:
from modal import Image, Stub, wsgi_app
stub = Stub()
image = Image.debian_slim().pip_install("flask")
@stub.function(image=image)
@wsgi_app()
def flask_app():
from flask import Flask, request
web_app = Flask(__name__)
@web_app.get("/")
def home():
return "Hello Flask World!"
@web_app.post("/echo")
def echo():
return request.json
return web_app
See Flask’s docs for more information on using Flask as a WSGI app.
Cold start performance
Consult the guide page on cold start performance for more information on when functions incur cold start penalties, and how to mitigate the impact of them.
Authentication
Modal doesn’t have an first class way to add authentication to web endpoints
yet, however token-based authentication is easy to implement in whichever
framework you’re using. For example, if you’re using @modal.web_endpoint
or
@modal.asgi_app
with FastAPI, you can add authentication like this:
from fastapi import Depends, HTTPException, status, Request
from fastapi.security import HTTPBearer, HTTPAuthorizationCredentials
from modal import Secret, Stub, web_endpoint
auth_scheme = HTTPBearer()
stub = Stub("auth-example")
@stub.function(secret=Secret.from_name("my-web-auth-token"))
@web_endpoint()
async def f(request: Request, token: HTTPAuthorizationCredentials = Depends(auth_scheme)):
import os
print(os.environ["AUTH_TOKEN"])
if token.credentials != os.environ["AUTH_TOKEN"]:
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="Incorrect bearer token",
headers={"WWW-Authenticate": "Bearer"},
)
# Function body
return "hi"
This assumes you have a Modal secret named
my-web-auth-token
created, with contents {AUTH_TOKEN: secret-random-token}
.
Now, your endpoint will return a 401 status code except when you hit it with the
correct Authorization
header set (note that you have to prefix the token with
Bearer
):
curl --header "Authorization: Bearer secret-random-token" https://modal-labs--auth-example-f.modal.run
Custom domains
Custom domains are available on our Organization and Enterprise plans.
You can use your own domain names with Modal web endpoints. If your plan supports custom domains, visit the Domains tab in your workspace settings to add a domain name to your workspace.
You can use three kinds of domains with Modal:
- Apex: root domain names like
example.com
- Subdomains: single subdomain entries such as
my-app.example.com
,api.example.com
, etc. - Wildcard domains: either a wild card used in a subdomain (
*.example.com
) or on a sub-subdomain (*.modal.example.com
), etc.
You’ll be asked to update your domain DNS records with your domain name registrar and then validate the configuration in Modal. Once the records have been properly updated and propagated, your custom domain will be ready to use.
You can assign any Modal web endpoint to an Apex domain or subdomain. Provided
you have one of these domain types registered, you can assign them in your code
using the argument custom_domains
, for example:
from modal import Stub, web_endpoint
stub = Stub("custom-domains-example")
@stub.function()
@web_endpoint(custom_domains=["api.example.com"])
def hello(message:str):
return {
"message": f"hello {message}"
}
You will then be able to access that web endpoint with Modal serve
or
deploy
.
$ curl -s https://api.example.com?message=world | jq
{
"message": "hello world"
}
Note that Modal automatically generates and renews TLS certificates for your custom domains. Since we do this when your domain is first accessed, there may be an additional 1-2s latency on the first request. Additional requests use a cached certificate.
You can also register multiple domain names and associate them with the same web endpoint.
from modal import Stub, web_endpoint
stub = Stub("custom-domains-example-2")
@stub.function()
@web_endpoint(custom_domains=["api.example.com", "api.example.net"])
def hello(message:str):
return {
"message": f"hello {message}"
}
For Wildcard domains, Modal will automatically resolve arbitrary custom
endpoints (and issue TLS certificates). For example, if you add the wildcard
domain *.example.com
, then you can create any custom domains under
example.com
:
import random
from modal import Stub, web_endpoint
stub = Stub("custom-domains-example-2")
random_domain_name = random.choice(range(10))
@stub.function()
@web_endpoint(custom_domains=[f"{random_domain_name}.example.com"])
def hello(message:str):
return {
"message": f"hello {message}"
}
Custom domains can also be used with
ASGI or
WSGI apps using the same
custom_domains
argument.