Tunnels (beta)
Modal allows you to expose live TCP ports on a Modal container. This is done by creating a tunnel that forwards the port to the public Internet.
import modal
app = modal.App()
@app.function()
def start_app():
# Inside this `with` block, port 8000 on the container can be accessed by
# the address at `tunnel.url`, which is randomly assigned.
with modal.forward(8000) as tunnel:
print(f"tunnel.url = {tunnel.url}")
print(f"tunnel.tls_socket = {tunnel.tls_socket}")
# ... start some web server at port 8000, using any framework
Tunnels are direct connections and terminate TLS automatically. Within a few milliseconds of container startup, this function prints a message such as:
tunnel.url = https://wtqcahqwhd4tu0.r5.modal.host
tunnel.tls_socket = ('wtqcahqwhd4tu0.r5.modal.host', 443)
Refer to the Sandbox
guide for more
details on how to expose ports from a Modal Sandbox
.
Build with tunnels
Tunnels are the fastest way to get a low-latency, direct connection to a running container. You can use them to run live browser applications with interactive terminals, Jupyter notebooks, VS Code servers, and more.
As a quick example, here is how you would expose a Jupyter notebook:
import os
import secrets
import subprocess
import modal
app = modal.App()
app.image = modal.Image.debian_slim().pip_install("jupyterlab")
@app.function()
def run_jupyter():
token = secrets.token_urlsafe(13)
with modal.forward(8888) as tunnel:
url = tunnel.url + "/?token=" + token
print(f"Starting Jupyter at {url}")
subprocess.run(
[
"jupyter",
"lab",
"--no-browser",
"--allow-root",
"--ip=0.0.0.0",
"--port=8888",
"--LabApp.allow_origin='*'",
"--LabApp.allow_remote_access=1",
],
env={**os.environ, "JUPYTER_TOKEN": token, "SHELL": "/bin/bash"},
stderr=subprocess.DEVNULL,
)
When you run the function, it starts Jupyter and gives you the public URL. It’s as simple as that.
All Modal features are supported. If you
need GPUs, pass gpu=
to the
@app.function()
decorator. If you
need more CPUs, RAM, or to attach
volumes or
network file systems, those
also just work.
Programmable startup
The tunnel API is completely on-demand, so you can start them as the result of a web request.
For example, you could make something like Jupyter Hub without leaving Modal, giving your users their own Jupyter notebooks when they visit a URL:
import modal
image = modal.Image.debian_slim().pip_install("fastapi[standard]")
app = modal.App(image=image)
@app.function(timeout=900) # 15 minutes
def run_jupyter(q):
... # as before, but return the URL on app.q
@app.function()
@modal.web_endpoint(method="POST")
def jupyter_hub():
from fastapi import HTTPException
from fastapi.responses import RedirectResponse
... # do some validation on the secret or bearer token
if is_valid:
with modal.Queue.ephemeral() as q:
run_jupyter.spawn(q)
url = q.get()
return RedirectResponse(url, status_code=303)
else:
raise HTTPException(401, "Not authenticated")
This gives every user who sends a POST request to the web endpoint their own Jupyter notebook server, on a fully isolated Modal container.
You could do the same with VS Code and get some basic version of an instant, serverless IDE!
Advanced: Unencrypted TCP tunnels
By default, tunnels are only exposed to the Internet at a secure random URL, and connections have automatic TLS (the “S” in HTTPS). However, sometimes you might need to expose a protocol like an SSH server that goes directly over TCP. In this case, we have support for unencrypted tunnels:
with modal.forward(8000, unencrypted=True) as tunnel:
print(f"tunnel.tcp_socket = {tunnel.tcp_socket}")
Might produce an output like:
tunnel.tcp_socket = ('r3.modal.host', 23447)
You can then connect over TCP, for example with nc r3.modal.host 23447
. Unlike
encrypted TLS sockets, these cannot be given a non-guessable, cryptographically
random URL due to how the TCP protocol works, so they are assigned a random port
number instead.
Pricing
Modal only charges for containers based on the resources you use. There is no additional charge for having an active tunnel.
For example, if you start a Jupyter notebook on port 8888 and access it via tunnel, you can use it for an hour for development (with 0.01 CPUs) and then actually run an intensive job with 16 CPUs for one minute. The amount you would be billed for in that hour is 0.01 + 16 * (1/60) = 0.28 CPUs, even though you had access to 16 CPUs without needing to restart your notebook.
Security
Tunnels are run on Modal’s private global network of Internet relays. On startup, your container will connect to the nearest tunnel so you get the minimum latency, very similar in performance to a direct connection with the machine.
This makes them ideal for live debugging sessions, using web-based terminals like ttyd.
The generated URLs are cryptographically random, but they are also public on the Internet, so anyone can access your application if they are given the URL.
We do not currently do any detection of requests above L4, so if you are running a web server, we will not add special proxy HTTP headers or translate HTTP/2. You’re just getting the TLS-encrypted TCP stream directly!