modal.CloudBucketMount
class CloudBucketMount(object)
Mounts a cloud bucket to your container. Currently supports AWS S3 buckets.
S3 buckets are mounted using AWS S3 Mountpoint. S3 mounts are optimized for reading large files sequentially. It does not support every file operation; consult the AWS S3 Mountpoint documentation for more information.
AWS S3 Usage
import subprocess
app = modal.App() # Note: "app" was called "stub" up until April 2024
secret = modal.Secret.from_dict({
"AWS_ACCESS_KEY_ID": "...",
"AWS_SECRET_ACCESS_KEY": "...",
})
@app.function(
volumes={
"/my-mount": modal.CloudBucketMount(
bucket_name="s3-bucket-name",
secret=secret,
read_only=True
)
}
)
def f():
subprocess.run(["ls", "/my-mount"], check=True)
Cloudflare R2 Usage
Cloudflare R2 is S3-compatible so its setup looks very similar to S3.
But additionally the bucket_endpoint_url
argument must be passed.
import subprocess
app = modal.App() # Note: "app" was called "stub" up until April 2024
secret = modal.Secret.from_dict({
"AWS_ACCESS_KEY_ID": "...",
"AWS_SECRET_ACCESS_KEY": "...",
})
@app.function(
volumes={
"/my-mount": modal.CloudBucketMount(
bucket_name="my-r2-bucket",
bucket_endpoint_url="https://<ACCOUNT ID>.r2.cloudflarestorage.com",
secret=secret,
read_only=True
)
}
)
def f():
subprocess.run(["ls", "/my-mount"], check=True)
Google GCS Usage
Google Cloud Storage (GCS) is partially S3-compatible.
Currently only read_only=True
is supported for GCS buckets. GCS Buckets also require a secret with Google-specific
key names (see below) populated with a HMAC key.
import subprocess
app = modal.App() # Note: "app" was called "stub" up until April 2024
gcp_hmac_secret = modal.Secret.from_dict({
"GOOGLE_ACCESS_KEY_ID": "GOOG1ERM12345...",
"GOOGLE_ACCESS_KEY_SECRET": "HTJ123abcdef...",
})
@app.function(
volumes={
"/my-mount": modal.CloudBucketMount(
bucket_name="my-gcs-bucket",
bucket_endpoint_url="https://storage.googleapis.com",
secret=gcp_hmac_secret,
read_only=True, # writing to bucket currently unsupported
)
}
)
def f():
subprocess.run(["ls", "/my-mount"], check=True)
def __init__(self, bucket_name: str, bucket_endpoint_url: Optional[str] = None, secret: Optional[modal.secret._Secret] = None, read_only: bool = False, requester_pays: bool = False) -> None