modal.Volume

class Volume(modal.object.Object)

A writeable volume that can be used to share files between one or more Modal functions.

The contents of a volume is exposed as a filesystem. You can use it to share data between different functions, or to persist durable state across several instances of the same function.

Unlike a networked filesystem, you need to explicitly reload the volume to see changes made since it was mounted. Similarly, you need to explicitly commit any changes you make to the volume for the changes to become visible outside the current container.

Concurrent modification is supported, but concurrent modifications of the same files should be avoided! Last write wins in case of concurrent modification of the same file - any data the last writer didn’t have when committing changes will be lost!

As a result, volumes are typically not a good fit for use cases where you need to make concurrent modifications to the same file (nor is distributed file locking supported).

Volumes can only be committed and reloaded if there are no open files for the volume - attempting to reload or commit with open files will result in an error.

Usage

import modal

stub = modal.Stub()
stub.volume = modal.Volume.new()

@stub.function(volumes={"/root/foo": stub.volume})
def f():
    with open("/root/foo/bar.txt", "w") as f:
        f.write("hello")
    stub.app.volume.commit()  # Persist changes

@stub.function(volumes={"/root/foo": stub.volume})
def g():
    stub.app.volume.reload()  # Fetch latest changes
    with open("/root/foo/bar.txt", "r") as f:
        print(f.read())
def __init__(self, *args, **kwargs):

from_id

@classmethod
def from_id(cls: Type[O], object_id: str, client: Optional[_Client] = None) -> O:

Retrieve an object from its unique ID (accessed through obj.object_id).

persist

def persist(
    self, label: str, namespace=api_pb2.DEPLOYMENT_NAMESPACE_WORKSPACE, environment_name: Optional[str] = None
):

Object.persist is deprecated for generic objects. See NetworkFileSystem.persisted or Dict.persisted.

from_name

@classmethod
def from_name(
    cls: Type[O],
    app_name: str,
    tag: Optional[str] = None,
    namespace=api_pb2.DEPLOYMENT_NAMESPACE_WORKSPACE,
    environment_name: Optional[str] = None,
) -> O:

Retrieve an object with a given name and tag.

Useful for referencing secrets, as well as calling a function from a different app. Use this when attaching the object to a stub or function.

Examples

# Retrieve a secret
stub.my_secret = Secret.from_name("my-secret")

# Retrieve a function from a different app
stub.other_function = Function.from_name("other-app", "function")

# Retrieve a persisted Volume, Queue, or Dict
stub.my_volume = Volume.from_name("my-volume")
stub.my_queue = Queue.from_name("my-queue")
stub.my_dict = Dict.from_name("my-dict")

lookup

@classmethod
def lookup(
    cls: Type[O],
    app_name: str,
    tag: Optional[str] = None,
    namespace=api_pb2.DEPLOYMENT_NAMESPACE_WORKSPACE,
    client: Optional[_Client] = None,
    environment_name: Optional[str] = None,
) -> O:

Lookup an object with a given name and tag.

This is a general-purpose method for objects like functions, network file systems, and secrets. It gives a reference to the object in a running app.

Examples

# Lookup a secret
my_secret = Secret.lookup("my-secret")

# Lookup a function from a different app
other_function = Function.lookup("other-app", "function")

# Lookup a persisted Volume, Queue, or Dict
my_volume = Volume.lookup("my-volume")
my_queue = Queue.lookup("my-queue")
my_dict = Dict.lookup("my-dict")

new

@staticmethod
def new() -> "_Volume":

Construct a new volume, which is empty by default.

persisted

@staticmethod
def persisted(
    label: str,
    namespace=api_pb2.DEPLOYMENT_NAMESPACE_WORKSPACE,
    environment_name: Optional[str] = None,
) -> "_Volume":

Deploy a Modal app containing this object. This object can then be imported from other apps using the returned reference, or by calling modal.Volume.from_name(label) (or the equivalent method on respective class).

Example Usage

import modal

volume = modal.Volume.persisted("my-volume")

stub = modal.Stub()

# Volume refers to the same object, even across instances of `stub`.
@stub.function(volumes={"/vol": volume})
def f():
    pass

commit

@live_method
def commit(self):

Commit changes to the volume and fetch any other changes made to the volume by other containers.

Committing always triggers a reload after saving changes.

If successful, the changes made are now persisted in durable storage and available to other containers accessing the volume.

Committing will fail if there are open files for the volume.

reload

@live_method
def reload(self):

Make latest committed state of volume available in the running container.

Uncommitted changes to the volume, such as new or modified files, will be preserved during reload. Uncommitted changes will shadow any changes made by other writers - e.g. if you have an uncommitted modified a file that was also updated by another writer you will not see the other change.

Reloading will fail if there are open files for the volume.

iterdir

@live_method_gen
def iterdir(self, path: str) -> AsyncIterator[api_pb2.VolumeListFilesEntry]:

Iterate over all files in a directory in the volume.

  • Passing a directory path lists all files in the directory (names are relative to the directory)
  • Passing a file path returns a list containing only that file’s listing description
  • Passing a glob path (including at least one * or ** sequence) returns all files matching that glob path (using absolute paths)

listdir

@live_method
def listdir(self, path: str) -> List[api_pb2.VolumeListFilesEntry]:

List all files under a path prefix in the modal.Volume.

  • Passing a directory path lists all files in the directory
  • Passing a file path returns a list containing only that file’s listing description
  • Passing a glob path (including at least one * or ** sequence) returns all files matching that glob path (using absolute paths)

read_file

@live_method_gen
def read_file(self, path: Union[str, bytes]) -> AsyncIterator[bytes]:

Read a file from the modal.Volume.

Example:

vol = modal.Volume.lookup("my-modal-volume")
data = b""
for chunk in vol.read_file("1mb.csv"):
    data += chunk
print(len(data))  # == 1024 * 1024