Volumes
Modal Volumes provide a high-performance distributed file system for your Modal applications. They are designed for write-once, read-many I/O workloads, like creating machine learning model weights and distributing them for inference.
This page is a high-level guide to using Modal Volumes.
For reference documentation on the modal.Volume
object, see this page.
For reference documentation on the modal volume
CLI command, see this page.
Volumes v2ย
A new generation of the file system, Volumes v2, is now available as a beta preview.
๐ฑ Instructions that are specific to v2 Volumes will be annotated with ๐ฑ below.
Read more about Volumes v2 below.
Creating a Volumeย
The easiest way to create a Volume and use it as a part of your App is to use
the modal volume create
CLI command. This will create the Volume and output
some sample code:
% modal volume create my-volume
Created volume 'my-volume' in environment 'main'.
๐ฑ To create a v2 Volume, pass
--version=2
in the command above.
Using a Volume on Modalย
To attach an existing Volume to a Modal Function, use Volume.from_name
:
vol = modal.Volume.from_name("my-volume")
@app.function(volumes={"/data": vol})
def run():
with open("/data/xyz.txt", "w") as f:
f.write("hello")
vol.commit() # Needed to make sure all changes are persisted before exit
You can also browse and manipulate Volumes from an ad hoc Modal Shell:
% modal shell --volume my-volume --volume another-volume
Volumes will be mounted under /mnt
.
Downloading a file from a Volumeย
While thereโs no file size limit for individual files in a volume, the frontend only supports downloading files up to 16โฏMB. For larger files, please use the CLI:
% modal volume get my-volume xyz.txt xyz-local.txt
Creating Volumes lazily from codeย
You can also create Volumes lazily from code using:
vol = modal.Volume.from_name("my-volume", create_if_missing=True)
๐ฑ To create a v2 Volume, pass
version=2
to the call tofrom_name()
in the code above.
This will create the Volume if it doesnโt exist.
Using a Volume from outside of Modalย
Volumes can also be used outside Modal via the Python SDK or our CLI.
Using a Volume from local codeย
You can interact with Volumes from anywhere you like using the modal
Python client library.
vol = modal.Volume.from_name("my-volume")
with vol.batch_upload() as batch:
batch.put_file("local-path.txt", "/remote-path.txt")
batch.put_directory("/local/directory/", "/remote/directory")
batch.put_file(io.BytesIO(b"some data"), "/foobar")
For more details, see the reference documentation.
Using a Volume via the command lineย
You can also interact with Volumes using the command line interface. You can run modal volume
to get a full list of its subcommands:
% modal volume
Usage: modal volume [OPTIONS] COMMAND [ARGS]...
Read and edit modal.Volume volumes.
Note: users of modal.NetworkFileSystem should use the modal nfs command instead.
โญโ Options โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฎ
โ --help Show this message and exit. โ
โฐโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฏ
โญโ File operations โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฎ
โ cp Copy within a modal.Volume. Copy source file to destination file or multiple source files to destination directory. โ
โ get Download files from a modal.Volume object. โ
โ ls List files and directories in a modal.Volume volume. โ
โ put Upload a file or directory to a modal.Volume. โ
โ rm Delete a file or directory from a modal.Volume. โ
โฐโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฏ
โญโ Management โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฎ
โ create Create a named, persistent modal.Volume. โ
โ delete Delete a named, persistent modal.Volume. โ
โ list List the details of all modal.Volume volumes in an Environment. โ
โฐโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฏ
For more details, see the reference documentation.
Volume commits and reloadsย
Unlike a normal filesystem, you need to explicitly reload the Volume to see
changes made since it was first mounted. This reload is handled by invoking the .reload()
method on a Volume object.
Similarly, any Volume changes made within a container need to be committed for
those the changes to become visible outside the current container. This is handled
periodically by background commits and directly by invoking
the .commit()
method on a modal.Volume
object.
At container creation time the latest state of an attached Volume is mounted. If
the Volume is then subsequently modified by a commit operation in another
running container, that Volume modification wonโt become available until the
original container does a .reload()
.
Consider this example which demonstrates the effect of a reload:
import pathlib
import modal
app = modal.App()
volume = modal.Volume.from_name("my-volume")
p = pathlib.Path("/root/foo/bar.txt")
@app.function(volumes={"/root/foo": volume})
def f():
p.write_text("hello")
print(f"Created {p=}")
volume.commit() # Persist changes
print(f"Committed {p=}")
@app.function(volumes={"/root/foo": volume})
def g(reload: bool = False):
if reload:
volume.reload() # Fetch latest changes
if p.exists():
print(f"{p=} contains '{p.read_text()}'")
else:
print(f"{p=} does not exist!")
@app.local_entrypoint()
def main():
g.remote() # 1. container for `g` starts
f.remote() # 2. container for `f` starts, commits file
g.remote(reload=False) # 3. reuses container for `g`, no reload
g.remote(reload=True) # 4. reuses container, but reloads to see file.
The output for this example is this:
p=PosixPath('/root/foo/bar.txt') does not exist!
Created p=PosixPath('/root/foo/bar.txt')
Committed p=PosixPath('/root/foo/bar.txt')
p=PosixPath('/root/foo/bar.txt') does not exist!
p=PosixPath('/root/foo/bar.txt') contains hello
This code runs two containers, one for f
and one for g
. Only the last
function invocation reads the file created and committed by f
because it was
configured to reload.
Background commitsย
Modal Volumes run background commits:
every few seconds while your Function executes,
the contents of attached Volumes will be committed
without your application code calling .commit
.
A final snapshot and commit is also automatically performed on container shutdown.
Being able to persist changes to Volumes without changing your application code is especially useful when training or fine-tuning models using frameworks.
Model servingย
A single ML model can be served by simply baking it into a modal.Image
at
build time using run_function
. But
if you have dozens of models to serve, or otherwise need to decouple image
builds from model storage and serving, use a modal.Volume
.
Volumes can be used to save a large number of ML models and later serve any one of them at runtime with great performance. This snippet below shows the basic structure of the solution.
import modal
app = modal.App()
volume = modal.Volume.from_name("model-store")
model_store_path = "/vol/models"
@app.function(volumes={model_store_path: volume}, gpu="any")
def run_training():
model = train(...)
save(model_store_path, model)
volume.commit() # Persist changes
@app.function(volumes={model_store_path: volume})
def inference(model_id: str, request):
try:
model = load_model(model_store_path, model_id)
except NotFound:
volume.reload() # Fetch latest changes
model = load_model(model_store_path, model_id)
return model.run(request)
For more details, see our guide to storing model weights on Modal.
Model checkpointingย
Checkpoints are snapshots of an ML model and can be configured by the callback functions of ML frameworks. You can use saved checkpoints to restart a training job from the last saved checkpoint. This is particularly helpful in managing preemption.
For more, see our example code for long-running training.
Hugging Face transformers
ย
To periodically checkpoint into a modal.Volume
, just set the Trainer
โs output_dir
to a directory in the Volume.
import pathlib
volume = modal.Volume.from_name("my-volume")
VOL_MOUNT_PATH = pathlib.Path("/vol")
@app.function(
gpu="A10G",
timeout=2 * 60 * 60, # run for at most two hours
volumes={VOL_MOUNT_PATH: volume},
)
def finetune():
from transformers import Seq2SeqTrainer
...
training_args = Seq2SeqTrainingArguments(
output_dir=str(VOL_MOUNT_PATH / "model"),
# ... more args here
)
trainer = Seq2SeqTrainer(
model=model,
args=training_args,
train_dataset=tokenized_xsum_train,
eval_dataset=tokenized_xsum_test,
)
Volume performanceย
Volumes work best when they contain less than 50,000 files and directories. The latency to attach or modify a Volume scales linearly with the number of files in the Volume, and past a few tens of thousands of files the linear component starts to dominate the fixed overhead.
There is currently a hard limit of 500,000 inodes (files, directories and
symbolic links) per Volume. If you reach this limit, any further attempts to
create new files or directories will error with ENOSPC
(No space left on device).
Filesystem consistencyย
Concurrent modificationย
Concurrent modification from multiple containers is supported, but concurrent modifications of the same files should be avoided. Last write wins in case of concurrent modification of the same file โ any data the last writer didnโt have when committing changes will be lost!
The number of commits you can run concurrently is limited. If you run too many concurrent commits each commit will take longer due to contention. If you are committing small changes, avoid doing more than 5 concurrent commits (the number of concurrent commits you can make is proportional to the size of the changes being committed).
As a result, Volumes are typically not a good fit for use cases where you need to make concurrent modifications to the same file (nor is distributed file locking supported).
While a reload is in progress the Volume will appear empty to the container that initiated the reload. That means you cannot read from or write to a Volume in a container where a reload is ongoing (note that this only applies to the container where the reload was issued, other containers remain unaffected).
Busy Volume errorsย
You can only reload a Volume when there no open files on the Volume. If you have
open files on the Volume the .reload()
operation will fail with โvolume busyโ. The following is a simple example of how
a โvolume busyโ error can occur:
volume = modal.Volume.from_name("my-volume")
@app.function(volumes={"/vol": volume})
def reload_with_open_files():
f = open("/vol/data.txt", "r")
volume.reload() # Cannot reload when files in the Volume are open.
Canโt find file on Volume errorsย
When accessing files in your Volume, donโt forget to pre-pend where your Volume is mounted in the container.
In the example below, where the Volume has been mounted at /data
, โhelloโ is
being written to /data/xyz.txt
.
import modal
app = modal.App()
vol = modal.Volume.from_name("my-volume")
@app.function(volumes={"/data": vol})
def run():
with open("/data/xyz.txt", "w") as f:
f.write("hello")
vol.commit()
If you instead write to /xyz.txt
, the file will be saved to the local disk of the Modal Function.
When you dump the contents of the Volume, you will not see the xyz.txt
file.
Volumes v2 overviewย
Volumes v2 generally behave just like Volumes v1, and most of the existing APIs and CLI commands that you are used to will work the same between versions. Because the file system implementation is completely different, there will be some significant performance characteristics that can differ from version 1 Volumes. Below is an outline of the key differences you should be aware of.
Volumes v2 is still in betaย
This new file system version is still in beta, and we cannot guarantee that no data will be lost. We donโt recommend using Volumes v2 for any mission-critical data at this time. You can still reap the benefits of v2 for data that isnโt precious, or that is easy to rebuild, such as log files, regularly updated training data and model weights, caches, and more.
Volumes v2 are HIPAA compliantย
If you delete the volume, the data is be guaranteed to be lost according to HIPAA requirements.
Volumes v2 is more scaleableย
Volumes v2 support more files, higher throughput, and more irregular access patterns. Commits and reloads are also faster.
Additionally, Volumes v2 supports hard-linking of files, where multiple paths can point to the same inode.
In v2, you can store as many files as you wantย
There is no limit on the number of files in Volumes v2.
By contrast, in Volumes v1, there is a limit on the number of files of 500,000, and we recommend keeping the count to 50,000 or less.
In v2, you can write concurrently from hundreds of containersย
The file system should not experience any performance degradation as more containers write to distinct files simultaneously.
By contrast, in Volumes v1, we recommend no more than five writers access the Volume at once.
Note, however, that concurrent access to a particular file in a Volume still has last-write-wins semantics in many circumstances. These semantics are unacceptable for most applications, so any particular file should only be written to by a single container at a time.
In v2, random accesses have improved performanceย
In v1, writes to locations inside a file would sometimes incur substantial overhead, like a rewrite of the entire file.
In v2, this overhead is removed, and only changes are written.
Volumes v2 has a few limits in placeย
While we work out performance trade-offs and listen to user feedback, we have put some artificial limits in place.
- Files must be less than one 1 TiB.
- At most 32,768 files can be stored in a single directory. Directory depth is unbounded, so the total file count is unbounded.
- Traversing the filesystem can be slower in v2 than in v1, due to demand loading of the filesystem tree.
Upgrading v1 Volumesย
Currently, there is no automated tool for upgrading v1 Volumes to v2. We are planning to implement an automated migration path but for now v1 Volumes need to be manually migrated by creating a new v2 Volume and either copying files over from the v1 Volume or writing new files.
To reuse the name of an existing v1 Volume for a new v2 Volume, first stop all apps that are utilizing the v1 Volume before deleting it. If this is not feasible, e.g. due to wanting to avoid downtime, use a new name for the v2 Volume.
Warning: When deleting an existing Volume, any deployed apps or running functions utilizing that Volume will cease to function, even if a new Volume is created with the same name. This is because Volumes are identified with opaque unique IDs that are resolved at application deployment or start time. A newly created Volume with the same name as a deleted Volume will have a new Volume ID and any deployed or running apps will still be referring to the old ID until these apps are re-deployed or restarted.
In order to create a new volume and copy data over from the old volume, you can
use a tool like cp
if you intend to copy all the data in one go, or rsync
if you want to incrementally copy the data across a longer time span:
$ modal volume create --version=2 2files2furious
$ modal shell --volume files-and-furious --volume 2files2furious
Welcome to Modal's debug shell!
We've provided a number of utilities for you, like `curl` and `ps`.
# Option 1: use `cp`
root / โ cp -rp /mnt/files-and-furious/. /mnt/2files2furious/.
root / โ sync /mnt/2files2furious # Ensure changes are persisted before exiting
# Option 2: use `rsync`
root / โ apt install -y rsync
root / โ rsync -a /mnt/files-and-furious/. /mnt/2files2furious/.
root / โ sync /mnt/2files2furious # Ensure changes are persisted before exiting
Further examplesย
- Character LoRA fine-tuning with model storage on a Volume
- Protein folding with model weights and output files stored on Volumes
- Dataset visualization with Datasette using a SQLite database on a Volume