CubeSandbox

English

简体中文

English

简体中文

Appearance

Agent platform integration: manual freeze / resume ​

This guide is for agent orchestrators (not end-user SDK demos) that expose sandbox_freeze / sandbox_connect_by_id style tools on top of CubeSandbox.

Mental model: three independent S3 layers ​

LayerWhat it isWhat it preserves
Volume (cube-volume-s3 + MinIO)Per-user mount at e.g. /home/user/persistentFiles across kill + recreate
s3lvol (ONE_CLICK_ENABLE_S3LVOL)Cluster snapshot backendCross-node pause/snapshot
Template --backend s3Template registration flagCross-node pause/snapshot (xfs templates pause on-node too; S3 is for cross-node — see Cross-node snapshots)

A Volume mount does not replace instance freeze. Freeze keeps the same sandbox ID (memory + disk state of that instance). Volume keeps files when the instance is destroyed and a new one is created.

Critical: pause() does not stop the idle timer ​

With the default on_timeout="kill", a sandbox keeps its creation-time idle deadline even after pause(). A paused sandbox can still be destroyed when that deadline expires.

Symptom: POST /pause succeeds, CubeMaster logs show action=pause, but connect / GET /sandboxes/{id} returns 404 an hour later.

Mitigations (pick one or combine):

  1. Extend retention with POST /sandboxes/{id}/timeout and {"timeout": <seconds>} (e.g. {"timeout": 86400} for 24h). Apply it before pause on a running sandbox, or immediately after pause returns — the idle sweeper runs every few seconds and still evaluates the pre-pause deadline in the gap between the two calls.
  2. On create, use lifecycle.on_timeout="pause" plus a long timeout, or timeout=NEVER_TIMEOUT when policy allows.
  3. Use POST /sandboxes/{id}/connect before talking to envd when resuming a paused sandbox (see below).

Freeze (agent tool) ​

text
1. POST /sandboxes/{id}/timeout  {"timeout": 86400}   # while still running
2. POST /sandboxes/{id}/pause          # wait until state=paused
3. Drop in-memory handles; keep sandboxId in your user binding store

Before step 3, confirm the pause landed: GET /sandboxes/{id} should show state="paused". This applies whether you call the REST API directly or through a supported SDK (including E2B-compat sandbox.pause()).

Resume (connect_by_id) ​

text
1. GET  /sandboxes/{id}                # optional: check state
2. POST /sandboxes/{id}/connect        # auto-resumes paused → running
3. POST /sandboxes/{id}/timeout {"timeout": 300}   # optional: set interactive idle window
4. Connect envd / data plane (commands, files, VNC)

Step 3 is required when you need a shorter idle window after resume: connect(timeout=…) only extends the existing deadline (e.g. after a 24h freeze, connect(timeout=300) still leaves ~24h). Call POST /timeout after connect to replace it with the interactive window you want.

Calling envd or the data plane directly on a paused instance often fails before the control plane has resumed it — commonly a same-node 504 against a stale proxy backend, 503 + Retry-After while pause is in flight, or 410 Gone after the sandbox was killed. Always go through POST /sandboxes/{id}/connect first (see Lifecycle).

Volume permissions (s3fs) ​

The S3 volume plugin does not set s3fs uid/gid/umask; guest-visible ownership depends on s3fs defaults. Use the supported deploy knobs — the installer generates volume-s3.conf for you and rewrites it on upgrade, so do not hand-edit that file (see S3 Volume):

bash
# one-click (.one-click.env)
# install.sh only adds -ouse_path_request_style when this var is empty — keep it for bundled MinIO.
CUBE_S3_S3FS_EXTRA_OPTS='-ouse_path_request_style -ouid=1000 -ogid=1000 -oumask=022'
yaml
# Helm (values.yaml) — only when minio.enabled=false and volumeS3.endpoint / existingSecret is set
volumeS3:
  extraOpts: "-ouse_path_request_style -ouid=1000 -ogid=1000 -oumask=022"

With the chart-managed MinIO, volumeS3.extraOpts is ignored (the chart hardcodes S3FS options). Point an external MinIO backend at the same -ouse_path_request_style token.

For manual plugin deploy only, set S3FS_EXTRA_OPTS in volume-s3.conf. As a fallback, run post-mount chown/chmod in your agent bootstrap or template entrypoint. See also Host mount permissions for the related ownership troubleshooting pattern.

Operational checklist ​

  • [ ] s3lvol healthy on all compute nodes before tpl create-from-image --backend s3 (on Kubernetes, enabling s3lvol recreates the Big Pod and interrupts sandboxes on that node when using Pod network — cubeNode.hostNetwork: false; see Kubernetes Upgrade)
  • [ ] S3 templates READY on all nodes (cubemastercli tpl redo --template-id <id> --failed-only --node <node-ip> if one node FAILED)
  • [ ] Separate buckets: user volumes vs s3lvol snapshot storage
  • [ ] Agent binding TTL ≤ paused retention timeout (avoid binding to destroyed IDs)

See also ​