Modal logo
IntroductionCustom container images Defining ImagesUsing existing container imagesFast pull from registryGPUs and other resources GPU accelerationUsing CUDA on ModalReserving CPU and memoryScaling out Scaling outInput concurrencyBatch processingJob queuesDynamic batching (beta)Multi-node clusters (beta)Deployment Apps, Functions, and entrypointsManaging deploymentsInvoking deployed functionsContinuous deploymentRunning untrusted code in FunctionsModal Sandboxes SandboxesRunning commandsNetworking and securityFile accessSnapshotsModal NotebooksSecrets and environment variables SecretsEnvironment variablesScheduling and cron jobsWeb endpoints Web endpointsStreaming endpointsWeb endpoint URLsRequest timeoutsProxy Auth TokensNetworking TunnelsProxies (beta)Cluster networkingData sharing and storage Passing local dataVolumesStoring model weightsCloud bucket mountsDictsQueuesDataset ingestionPerformance Cold start performanceMemory SnapshotHigh-performance LLM inferenceGeographic latencyReliability and robustness Failures and retriesPreemptionTimeoutsGPU healthTroubleshootingSecurity and privacyIntegrations Using OIDC to authenticate with external servicesConnecting Modal to your Datadog accountConnecting Modal to your OpenTelemetry providerOkta SSOCustom SAML SSOSlack notifications (beta)Workspace & account settings WorkspacesEnvironmentsModal user account setupService usersBillingOther topics JavaScript/Go SDKsModal 1.0 migration guideFile and project structureDeveloping and debuggingDeveloping Modal code with LLMsJupyter notebooksAsynchronous API usageGlobal variablesRegion selectionContainer lifecycle hooksParametrized functionsS3 Gateway endpointsGPU Metrics

Developing Modal code with LLMs and agents

Modal is designed for fast iteration on infrastructure by both humans and machines.

Modal was built to provide human backend engineers the rapid feedback loops frontend engineers take for granted. This also means that Modal works well with code generation agents, especially those that can run CLI commands like modal run in an implement, test and debug loop, like OpenCode, Amp, Claude Code, Cursor’s agent mode, Gemini CLI, etc.

There are of course also many concepts and design patterns that are unique to Modal, so below we gather rules and guidelines that we have found useful when developing Modal code with LLMs. You can paste/import this into your AGENTS.md, CLAUDE.md, .cursor/rules/modal.mdc, etc. or use it as a starting point for your own rules or prompts. We also provide an llms.txt.

If you’re instead looking for information about how to run code generation agents on Modal, see the guide page for Modal Sandboxes or this example code.