S3 Volumes

Give a sandbox storage that survives its lifecycle: create a Volume → mount it in a sandbox → read/write → destroy the sandbox, the data stays → remount it next time. The backend is any S3-compatible object store (AWS S3, Tencent Cloud COS, Cloudflare R2, MinIO, …), and the default CubeSandbox install ships a bundled MinIO as that backend.

The default install (one-click / Helm defaults) is already set up. The installer starts a MinIO and wires the S3 Volume plugin, credentials, and CubeMaster/Cubelet config for you. You don't need to edit volume-s3.conf or install the plugin — jump straight to Usage.
Only if you want to point the backend at external S3 (AWS S3, Tencent Cloud COS, R2, a self-hosted MinIO, …) do you need Connecting external S3.

What it gives you

Capability	Notes
Survives sandbox teardown	Volume data stays after the sandbox is killed; remount it next time
Shared across sandboxes	One Volume can be mounted by several sandboxes at once
Read-only mounts	Each mount can be independently read-only — ideal for sharing models/datasets
Any S3-compatible backend	Bundled MinIO by default; swap to AWS S3, Tencent Cloud COS, Cloudflare R2, …
Credentials stay on the host	Secrets live only on CubeMaster/Cubelet; the sandbox sees just a filesystem

Usage

1. Install the SDK

bash

pip install 'cubesandbox>=0.6.0'

Use cubesandbox, not the official e2b SDK

The official e2b Python SDK hardcodes requests to the e2b.cloud backend and cannot target CubeSandbox. Use cubesandbox.

2. Set environment variables

On a dev machine that can reach CubeAPI:

bash

export CUBE_API_URL=http://<cubeapi-host>:3000
export CUBE_TEMPLATE_ID=<your-template-id>

# Required for remote sandbox I/O on mounted volumes (data plane via CubeProxy)
export CUBE_PROXY_NODE_IP=<cubeproxy-or-cubelet-node-ip>

# When cluster auth is enabled:
# export CUBE_API_KEY=<your-key>

3. Create a Volume and mount it

python

from cubesandbox import Sandbox, Volume

vol = Volume.create("my-data", driver="s3")   # in the default install, omitting driver also uses S3
print("volume_id:", vol.volume_id)

with Sandbox.create(volume_mounts={"/workspace": vol}) as sb:
    sb.files.write("/workspace/hello.txt", "from S3 volume")
    print(sb.files.read("/workspace/hello.txt"))   # → from S3 volume

# The sandbox is gone but the data stays; remount it next time, the file is still there
with Sandbox.create(volume_mounts={"/workspace": vol}) as sb:
    print(sb.files.read("/workspace/hello.txt"))

# Actually delete the Volume (backend data wiped — irreversible)
Volume.destroy(vol.volume_id)

volume_mounts is a {in-sandbox path: Volume} mapping; mount several at once.
Destroying a sandbox does not delete Volume data; only Volume.destroy() wipes it.

python

from cubesandbox import VolumeMount

vol = Volume.create("shared-dataset", driver="s3")

with Sandbox.create(volume_mounts={"/data": VolumeMount(vol, read_only=True)}) as sb:
    sb.files.read("/data/model.bin")

Connecting external S3

The default install uses bundled MinIO. To switch to external S3 (AWS S3, Tencent Cloud COS, R2, a self-hosted MinIO, …), change the "source" for your deploy method — the installer generates volume-s3.conf for you, so do not edit that file by hand (it gets rewritten).

one-click deploy

In .one-click.env (or env.example), turn off the bundled MinIO and fill in CUBE_S3_*, then re-run install.sh — it writes volume-s3.conf:

bash

CUBE_SANDBOX_MINIO_ENABLED=0
CUBE_S3_ENDPOINT=https://s3.example.com
CUBE_S3_ACCESS_KEY_ID=...
CUBE_S3_SECRET_ACCESS_KEY=...
CUBE_S3_BUCKET=cube-volumes
# CUBE_S3_REGION=us-east-1
# CUBE_S3_S3FS_EXTRA_OPTS=-ouse_path_request_style

Compute nodes don't run MinIO; copy the CUBE_S3_* backfilled on the control node:

bash

# Run on the control node; paste the output into the compute node's .env
grep '^CUBE_S3_' /usr/local/services/cubetoolbox/.one-click.env

If missing, the installer warns and continues; the S3 volume plugin stays unavailable. See one-click README · Bundled MinIO vs the S3 volume plugin.

Helm deploy

In values, turn off chart MinIO and fill in volumeS3.* (or point volumeS3.existingSecret at an existing Secret); after helm upgrade the chart renders the volume-s3.conf Secret:

yaml

minio:
  enabled: false
volumeS3:
  endpoint: https://s3.example.com
  accessKeyId: ...
  secretAccessKey: ...
  bucket: cube-volumes
  # region: us-east-1
  # extraOpts: -ouse_path_request_style

See the volumeS3 section of the chart values.yaml.

Manual deploy from scratch (not one-click / not Helm)

For manually deploying the plugin and hand-writing volume-s3.conf, see examples/volume/s3/README.md; for a Tencent Cloud COS walkthrough (bucket, access keys) — using the dedicated cos driver plugin, not this one — see examples/volume/cos/README.md.

Common backends

Provider	`ENDPOINT`	`REGION`
AWS S3	`https://s3.<region>.amazonaws.com`	the bucket's region
Tencent Cloud COS	`https://cos.<region>.myqcloud.com`	the bucket's region (e.g. `ap-guangzhou`)
Cloudflare R2	`https://<account-id>.r2.cloudflarestorage.com`	`auto`
MinIO	`http://<minio-host>:9000`	any value

FAQ

How is S3 Volume different from Host Mount?

Host Mount bind-mounts a pre-existing directory on the node into a sandbox — node-local, great for sharing data that already lives there. S3 Volume is a user-scoped persistent volume (e2b Volume API): created/destroyed on demand by the plugin, managed across sandbox lifecycles, backed by object storage. They complement each other: Host Mount for "a directory already on the node", S3 Volume for "one independent persistent volume per user/task".

What happens if I omit `driver`? Why does it default to S3?

When Volume.create(...) omits driver, CubeMaster picks the first entry in the volume_plugins list. The default install now lists s3 first, so omitting driver routes to S3. Writing driver="s3" explicitly is still recommended — it's clearer in multi-backend clusters and avoids behavior silently changing if someone reorders cos ahead of s3.

Does data survive sandbox teardown? What's the Volume lifecycle?

Yes. Mount/unmount is driven by a reference count — you don't manage mount points yourself:

Operation	Behavior
`Volume.create`	Creates a `volumes/<id>/` prefix in the backend
Sandbox mount (refCount 0→1)	Cubelet mounts the prefix with s3fs on the host, exposes it to the microVM
Sandbox mount (refCount >0)	Reuses the existing mount, no second mount
Sandbox teardown (refCount >1)	Decrements only, no unmount
Sandbox teardown (refCount →0)	`fusermount -u` unmounts; backend data is retained
`Volume.destroy`	Deletes the whole `volumes/<id>/` prefix — irreversible

What happens when I delete a Volume? Why do I sometimes get a 409?

Volume.destroy wipes the whole volumes/<id>/ prefix in the backend — irreversible. While any sandbox still holds the Volume, the API rejects the call with 409 — destroy those sandboxes first. Note that a stale refcount after a node crash can let the cluster-wide count reach 0 while a node still holds the mount; confirm before deleting.

Does the bucket have to exist already?

No. Create runs head-bucket first; it auto-creates the bucket if missing and reuses an existing one — no s3:CreateBucket permission needed when the bucket already exists.

Troubleshooting

Symptom	Check
`unknown driver: s3`	CubeMaster `volume_plugins` has no `s3` entry, or not restarted
`no plugin registered for driver "s3"`	Cubelet has no same-name plugin, or not restarted
Attach fails `s3fs mount failed`	`ls /dev/fuse` on the host; credentials/`ENDPOINT` correct
`InvalidAccessKeyId` / `SignatureDoesNotMatch`	Wrong key, missing bucket permission, or `REGION` doesn't match the endpoint's SigV4 region
Bucket name has dots	s3fs uses virtual-hosted-style addressing by default, which breaks TLS for dotted names. Use a dot-free name, or set `S3FS_EXTRA_OPTS=-ouse_path_request_style` in the conf (MinIO usually needs it too)
SDK write fails	`CUBE_PROXY_NODE_IP` unset; or CubeAPI/template not ready
`Volume.create` not using s3	Omitting `driver` picks the first `volume_plugins` entry — in the default install that's `s3`. If your cluster reordered `cos` ahead of `s3`, write `driver="s3"` explicitly

More: Volume Plugin Development, S3 plugin full walkthrough.

Kubernetes Deployment

Framework & Tool Integrations

Case Studies

S3 Volumes

What it gives you

Usage

1. Install the SDK

2. Set environment variables

3. Create a Volume and mount it

Connecting external S3

one-click deploy

Helm deploy

Manual deploy from scratch (not one-click / not Helm)

Common backends

FAQ

How is S3 Volume different from Host Mount?

What happens if I omit `driver`? Why does it default to S3?

Does data survive sandbox teardown? What's the Volume lifecycle?

What happens when I delete a Volume? Why do I sometimes get a 409?

Does the bucket have to exist already?

Troubleshooting

S3 Volumes ​

What it gives you ​

Usage ​

1. Install the SDK ​

2. Set environment variables ​

3. Create a Volume and mount it ​

Read-only mount / sharing across sandboxes ​

Connecting external S3 ​

one-click deploy ​

Helm deploy ​

Manual deploy from scratch (not one-click / not Helm) ​

Common backends ​

FAQ ​

How is S3 Volume different from Host Mount? ​

What happens if I omit driver? Why does it default to S3? ​

Does data survive sandbox teardown? What's the Volume lifecycle? ​

What happens when I delete a Volume? Why do I sometimes get a 409? ​

Does the bucket have to exist already? ​

Troubleshooting ​

S3 Volumes

What it gives you

Usage

1. Install the SDK

2. Set environment variables

3. Create a Volume and mount it

Read-only mount / sharing across sandboxes

Connecting external S3

one-click deploy

Helm deploy

Manual deploy from scratch (not one-click / not Helm)

Common backends

FAQ

How is S3 Volume different from Host Mount?

What happens if I omit `driver`? Why does it default to S3?

Does data survive sandbox teardown? What's the Volume lifecycle?

What happens when I delete a Volume? Why do I sometimes get a 409?

Does the bucket have to exist already?

Troubleshooting