Modal logo
Introduction Custom container images Defining Images Using existing container images Named images Fast pull from registry GPUs and other resources GPU acceleration Using CUDA on Modal Configuring CPU, memory, and disk Scaling out Scaling out Input concurrency Batch processing Job queues Dynamic batching Multi-node clusters BetaDeployment Apps, Functions, and entrypoints Managing deployments Invoking deployed functions Continuous deployment Running untrusted code in Functions Modal Endpoints Endpoints Endpoint metrics Benchmark an endpoint Endpoint integrations Modal Sandboxes Sandboxes Running commands Networking and security File access Snapshots Pricing and resources VM Sandboxes BetaModal Notebooks Secrets and environment variables Secrets Environment variables Scheduling and cron jobs HTTP Applications Servers Web Functions Streaming endpoints Web Function URLs Request timeouts Proxy Tokens Networking Tunnels Proxies BetaCluster networking Data sharing and storage Passing local data Volumes Storing model weights Cloud bucket mounts Dicts Queues Dataset ingestion Performance Cold start performance Memory Snapshots High-performance LLM inference Reliability and robustness Failures and retries Preemption Timeouts GPU health Troubleshooting Security and privacy Security and privacy Customer-supplied encryption keys AlphaAudit logs Integrations Using OIDC to authenticate with external services Connecting Modal to your Datadog account Connecting Modal to your OpenTelemetry provider Okta SSO Custom SAML SSO SCIM Integration BetaSlack notifications BetaWorkspace & account settings Workspaces Environments Modal user account setup Service users Role-Based Access Control (RBAC) Billing Budgets Other topics Feature maturity JavaScript/Go SDKs BetaModal 1.0 migration guide File and project structure Developing and debugging Jupyter notebooks Asynchronous API usage Global variables Region selection Container lifecycle hooks Parametrized functions Dynamic function configuration S3 Gateway endpoints GPU Metrics

Migration Guide: Sandbox Filesystem APIs

We have introduced new APIs for interacting with the contents of a Sandbox’s filesystem. The new APIs are backed by a new implementation that is more reliable and performant.

We strongly encourage migrating all usage to the new APIs. The legacy APIs are deprecated, and support will be discontinued on an accelerated timeline.

Overview 

The legacy APIs are exposed via a stateful, descriptor-oriented API. Calling sb.open(path) returns a file-like object with methods for reading or writing chunks of data. Additionally, basic operations like creating directories, listing directory contents, and removing files are exposed via methods on the Sandbox instance itself.

The new APIs are exposed via methods in the filesystem namespace on a Sandbox instance. Reads and writes are now path-oriented, operating on an entire file with a single method call. Basic operations continue to be supported with more verbose method names.

Python SDK 

The new APIs were added between versions 1.4.0 and 1.5.0, with performance optimizations made in 1.5.3. The legacy API will be removed in 1.6.0.

Replace all sb.open() calls and interactions on the returned FileIO object with calls to one of the following methods:

  • sb.filesystem.copy_from_local(local_path, remote_path)
  • sb.filesystem.copy_to_local(remote_path, local_path)
  • sb.filesystem.read_bytes(remote_path)
  • sb.filesystem.read_text(remote_path)
  • sb.filesystem.write_bytes(data, remote_path)
  • sb.filesystem.write_text(data, remote_path)

Examples:

Write data into the Sandbox:

# Deprecated usage
with sb.open("test.txt", "w") as f:
    f.write("Hello World!\n")

# Supported usage
sb.filesystem.write_text("Hello world!\n", "/test.txt")

Consider using objects from Python’s io module to migrate code that makes deeper use of the FileIO interface:

# Deprecated usage
with sb.open(path) as f:
    for line in f.readlines():
        ...

# Supported usage
data = io.StringIO(sb.filesystem.read_text(path))
for line in data.readlines():
    ...

Notes:

  • Unlike the stateful legacy FileIO methods, each call to one of the new methods performs a complete operation.
  • Writing to a path in a nonexistent directory will create that directory and any parents.
  • Writing to a path with an existing file will overwrite its data. There is no equivalent method for writing to a file that was opened in append mode. Consider a read-append-write operation for small files, or write chunks as separate files and combine via sb.exec().
  • There is also no exclusive (x) mode; check existence with sb.filesystem.stat(path) and catch modal.exception.SandboxFilesystemNotFoundError

For migrating basic file operations, use the following name substitutions:

  • sb.mkdir(path) -> sb.filesystem.make_directory(path)
  • sb.rm(path) -> sb.filesystem.remove(path)
  • sb.ls(path) -> sb.filesystem.list_files(path)

Notes:

  • The new sb.filesystem.make_directory(path) has create_parents=True by default and is idempotent; sb.mkdir has parents=False by default and errors if the target exists.
  • The new sb.filesystem.remove(dir_path) will remove dir_path if the directory is empty; sb.rm(dir_path) would error in this case.
  • The new sb.filesystem.remove(path, recursive=True) will raise InvalidError if path is on a mount type that does not support recursive removals.
  • Both recursive and create_parents must be passed as keyword arguments in the new APIs.

For listing files, note that the new API returns a list of FileInfo dataclasses rather than a list of file names:

# Deprecated usage
for file_name in sb.ls(path):
    print(file_name)

# Supported usage
for file_info in sb.filesystem.list_files(path):
    print(file_info.name)

For migrating file watching, substitute sb.watch(path) -> sb.filesystem.watch(path). The filter, recursive, and timeout parameters have the same semantics, but they must be passed as keyword arguments. If code that processes the events references the FileWatchEventType enum, update its import path to modal.types, as modal.file_io will be removed.

General notes:

The legacy APIs accept relative remote path arguments with undefined semantics. The new APIs consistently require remote_path values to be absolute, rejecting relative paths with modal.exception.InvalidError.

The legacy APIs raise most remote exceptions with built-in types (FileNotFoundError, NotADirectoryError, etc.), falling back to modal.exception.FilesystemExecutionError on undefined error codes. For remote errors, the new APIs raise Modal exceptions with a fine-grained type system inheriting from modal.exception.SandboxFilesystemError. These exceptions do not inherit the builtin types, so exception handling code must be migrated. Local exceptions still use built-in types. See method docstrings for details on specific error cases.

See more details in the reference documentation: https://modal.com/docs/sdk/py/latest/Sandbox

JS SDK 

The new APIs were added in version 0.7.6; legacy APIs were removed in 0.8.0.

Legacy filesystem operations were limited to sandbox.open() and the SandboxFile object it returned. Replace all such calls with one of the following methods:

  • sb.filesystem.copyFromLocal(localPath, remotePath)
  • sb.filesystem.copyToLocal(remotePath, localPath)
  • sb.filesystem.readBytes(remotePath)
  • sb.filesystem.readText(remotePath)
  • sb.filesystem.writeBytes(data, remotePath)
  • sb.filesystem.writeText(data, remotePath)

Note that the new APIs require remotePath to be absolute, rejecting relative paths with InvalidError.

Examples:

Write data into the Sandbox:

// Deprecated usage
const handle = await sb.open("/tmp/test.txt", "w");
await handle.write(new TextEncoder().encode("Hello World!\n"));
await handle.close();

// Supported usage
await sb.filesystem.writeText("Hello World!\n", "/tmp/test.txt");

Read data out of the Sandbox:

// Deprecated usage
const handle = await sb.open("/tmp/test.txt", "r");
const contents = new TextDecoder().decode(await handle.read());
await handle.close();

// Supported usage
const contents = await sb.filesystem.readText("/tmp/test.txt");

Notes:

  • Each call to the new methods performs a complete operation, so there are no handles to close() and no flush() step.
  • Writing to a path in a nonexistent directory will create that directory and any parents.
  • Writing to a path with an existing file will overwrite its data. There is no equivalent method for writing to a file in append mode. Consider a read-append-write operation for small files, or write chunks as separate files and combinine via sb.exec().
  • The new readText and writeText methods handle UTF-8 encoding, replacing manual TextDecoder and TextEncoder use.
  • The writeBytes method accepts Uint8Array, ArrayBuffer, or Buffer; SandboxFile.write required a Uint8Array.

Exception handling:

The legacy API threw SandboxFilesystemError for every remote filesystem error. The new APIs throw fine-grained subclasses of SandboxFilesystemError, or InvalidError in some cases; all of them are exported from the package root. Unlike in the Python SDK, code that matches on instanceof SandboxFilesystemError continues to work, but code that matches on specific error messages must be migrated.

The remaining methods in the filesystem namespace — listFiles, makeDirectory, remove, stat, and watch — have no counterpart in the legacy JS API, so there is nothing to migrate for those. Note that watch requires version 0.9.0 or later.

See more details in the reference documentation: https://modal.com/docs/sdk/js/latest/Sandbox