--- url: https://cubesandbox.com/guide/tutorials/bring-your-own-image.md --- # Custom Template Images This tutorial shows how to add `envd` to **your own application or container image** for use with the CubeSandbox SDK and E2B SDK. For the general workflow to create templates from OCI images and configure application ports and readiness probes, see [Create Templates from OCI Image](./template-from-image.md). *** ## 1. When does my image need `envd`? `envd` is the data-plane service that the CubeSandbox SDK and E2B SDK use for sandbox operations such as running commands, reading and writing files, and opening PTY sessions: | Capability | `envd` interface inside the sandbox | Without `envd` | | --- | --- | --- | | `envd` health check (can be used as the template probe) | `GET :49983/health` → 204 | This probe endpoint is unavailable | | `Sandbox.commands.run()` | Process API on `:49983` | Command APIs are unavailable | | `Sandbox.files.read/write()` | Files API on `:49983` | File APIs are unavailable | | Create-time environment variable initialization | `POST :49983/init` | Sandbox creation with environment variables fails | For interactive development and code-execution sandboxes, keeping `envd` is recommended so you can use the SDK to run commands, work with files, and troubleshoot the sandbox. An image that only serves its own application and does not use these capabilities can omit `envd`; configure its template probe to use the application's own HTTP health endpoint. ## 2. Quick start: build on top of `cubesandbox-base` `cubesandbox-base` is a plain `ubuntu:22.04` with `envd` preinstalled at `/usr/bin/envd` and a generic entrypoint that runs `envd` in the background while honoring any `CMD` you supply. Three steps get you to a working template: **write a Dockerfile → build and push → create the template**. > Prefer to read a runnable end-to-end example? See > [`examples/cubesandbox-base-nginx`](https://github.com/TencentCloud/CubeSandbox/tree/master/examples/cubesandbox-base-nginx) > in the repo — a minimal demo that stacks nginx on top of > `cubesandbox-base`. ### 2.1 Write a Dockerfile ```dockerfile FROM ghcr.io/tencentcloud/cubesandbox-base:2026.16 # Install your own tooling RUN apt-get update \ && apt-get install -y --no-install-recommends python3 python3-pip \ && rm -rf /var/lib/apt/lists/* RUN pip install --no-cache-dir pandas matplotlib numpy # Optional: if your app needs to be the foreground process, set CMD here. # envd stays alive in the background. # CMD ["python3", "/srv/app.py"] ``` ### 2.2 Build and push ```bash docker build -t my-registry.example.com/my-team/my-sandbox:v1 . docker push my-registry.example.com/my-team/my-sandbox:v1 ``` The registry must be reachable from your Cube cluster. ::: tip Plain HTTP registry Prefix the image with `http://`, for example `http://my-registry.example.com/my-team/my-sandbox:v1`. ::: ### 2.3 Create a Cube template Expose `49983` (envd) plus whatever ports your own application listens on: ```bash cubemastercli tpl create-from-image \ --image my-registry.example.com/my-team/my-sandbox:v1 \ --writable-layer-size 1G \ --expose-port 49983 \ --expose-port \ --probe 49983 \ --probe-path /health ``` Once you have a `template_id`, you can use the CubeSandbox SDK or E2B SDK to create sandboxes from it. See [Create Templates from OCI Image](./template-from-image.md) for an example. For practical examples, see [Local and Remote Image Build Examples](./template-build-practice.md). ## 3. Inject `envd` into an Existing Image If an existing image does not contain `envd`, either copy it from `cubesandbox-base` while building a custom image or let `cubemastercli` inject it during `create-from-image`. ### Copy It in the Dockerfile When you want to bring your own custom image, copy `envd` and the entrypoint **out of** `cubesandbox-base` with a `COPY --from=` stage: ```dockerfile FROM e2bdev/code-interpreter:latest USER root # Pull envd and the generic entrypoint from cubesandbox-base. COPY --from=ghcr.io/tencentcloud/cubesandbox-base:2026.16 \ /usr/bin/envd /usr/bin/envd COPY --from=ghcr.io/tencentcloud/cubesandbox-base:2026.16 \ /usr/local/bin/cube-entrypoint.sh /usr/local/bin/cube-entrypoint.sh # The upstream image already has its own entrypoint/CMD. Either wrap it # with cube-entrypoint.sh (preferred), or start envd manually from your # own script — see section 4 for the manual pattern. ENTRYPOINT ["/usr/local/bin/cube-entrypoint.sh"] CMD ["/bin/sh", "-c", "sudo --preserve-env=E2B_LOCAL /root/.jupyter/start-up.sh"] ``` A second example, starting from a slim Python image: ```dockerfile FROM python:3.11-slim COPY --from=ghcr.io/tencentcloud/cubesandbox-base:2026.16 \ /usr/bin/envd /usr/bin/envd COPY --from=ghcr.io/tencentcloud/cubesandbox-base:2026.16 \ /usr/local/bin/cube-entrypoint.sh /usr/local/bin/cube-entrypoint.sh RUN apt-get update \ && apt-get install -y --no-install-recommends curl \ && rm -rf /var/lib/apt/lists/* RUN pip install --no-cache-dir fastapi uvicorn COPY app.py /srv/app.py EXPOSE 49983 8000 ENTRYPOINT ["/usr/local/bin/cube-entrypoint.sh"] CMD ["uvicorn", "app:app", "--app-dir", "/srv", "--host", "0.0.0.0", "--port", "8000"] ``` Section 5 runs `curl` inside the container. If your base image does not include `curl`, install it as part of the image build before running that check. Build, push and template creation are identical to sections 2.2 / 2.3. ### Inject It During Template Creation If you do not want to modify the Dockerfile, use `--enable-inject-envd` to upload and inject `envd` while creating the template: ```bash cubemastercli tpl create-from-image \ --image \ --writable-layer-size 1G \ --expose-port 49983 \ --probe 49983 \ --probe-path /health \ --enable-inject-envd ``` | Option | Description | | --- | --- | | `--enable-inject-envd` | Upload an `envd` binary from `cubemastercli` and write it into the template rootfs. | | `--envd-path` | A local path on the machine running `cubemastercli`; used only with `--enable-inject-envd`. If omitted, the CLI uses its build-time embedded default `envd` when available. | `--envd-path` refers to the machine running the CLI, not the CubeMaster host. The CLI uploads the binary in the multipart `create-from-image` request. CubeMaster validates it, writes it to `/usr/local/bin/envd` in the template rootfs, and includes its SHA-256 in the rootfs artifact fingerprint so artifacts built with different `envd` binaries are not reused interchangeably. The uploaded file must be a non-empty ELF binary no larger than 16 MiB and compatible with the target rootfs operating system and CPU architecture. For example, a Linux x86\_64 image requires a Linux x86\_64 `envd` binary. If `cubemastercli` was built without an embedded default `envd`, `--envd-path` is required. To build the CLI with a default binary, prepare `envd` and run: ```bash make cubemastercli ENVD_LOCAL_PATH=/path/to/envd ``` For the `cubebox` instance type, CubeMaster also preserves the injection annotation and automatically wraps the main container command when creating a sandbox: it starts `/usr/local/bin/envd` in the background, executes the image's original command, and adds port `49983` to the exposed ports. The original image entrypoint therefore does not need to be changed when using this method. The command wrapper is not applied to non-`cubebox` instance types. ## 4. The entrypoint contract `cube-entrypoint.sh` implements a simple "envd-in-the-background, your app in the foreground" pattern: 1. It always starts `envd -port "${ENVD_PORT:-49983}"` in the background so that `/health` is reachable within about a second of container startup. 2. If the container was started **with** a user `CMD`, the script `exec`s that command. `envd` keeps running in the background; the user process owns `stdout`/`stderr` and receives `SIGTERM` on stop. 3. If the container was started **without** a `CMD`, the script simply waits on `envd`, keeping it as the foreground process. Environment variables: | Variable | Default | Purpose | | ------------------ | ------------------- | ---------------------------------------------------- | | `ENVD_PORT` | `49983` | Port `envd` listens on. | | `ENVD_EXTRA_ARGS` | *(empty)* | Extra flags passed after `-port`. `-isnotfc` is appended automatically if not already present, to skip Firecracker MMDS lookup. | | `ENVD_LOG_FILE` | `/var/log/envd.log` | File that captures envd stdout/stderr. Use `-` to inherit the container stdio. | | `ENVD_BIN` | `/usr/bin/envd` | Override if you install envd elsewhere. | ### Starting envd manually If you already have a non-trivial entrypoint of your own and don't want to delegate to `cube-entrypoint.sh`, just add one line before handing control to your main process: ```bash #!/bin/bash # your-entrypoint.sh # Start envd in the background. # -isnotfc is REQUIRED: it tells envd to skip the Firecracker MMDS lookup # at 169.254.169.254. CubeSandbox does not use Firecracker, so the MMDS # service does not exist. Without this flag envd will attempt to access # the non-existent MMDS, which may cause various problems such as network # timeouts, /init delays, or env_vars injection failures. /usr/bin/envd -port 49983 -isnotfc >/var/log/envd.log 2>&1 & # ... your usual startup sequence ... exec "$@" ``` ## 5. Verifying the image locally (optional) Before creating a template, check that the image stays running with its default startup command and that envd responds. Run the following steps in the same terminal; the host needs Docker, and the image needs `curl` and `/usr/bin/envd`. **1. Start the image.** ```bash IMG=my-registry.example.com/my-team/my-sandbox:v1 cid=$(docker create "$IMG") && docker start "$cid" ``` If `docker create` reports an error, resolve it before continuing. If `docker start` reports an error, use the container ID in `$cid` to inspect the failure in step 3. If both commands succeed, continue to step 2: a successful `docker start` does not guarantee that the container stays running. **2. Check the container and envd.** ```bash docker inspect --format '{{json .State}}' "$cid" ``` The state must show `"Status":"running"` and `"Running":true`. If it shows `exited`, go to step 3, even if `ExitCode` is `0`: the container must stay running to serve sandbox requests. ```bash docker exec "$cid" curl -sS --noproxy '*' --connect-timeout 1 --max-time 3 \ -o /dev/null -w 'envd /health => %{http_code}\n' \ http://127.0.0.1:49983/health # Expected: envd /health => 204 docker exec "$cid" /usr/bin/envd -version # => 2026.16 ``` The health request must complete successfully and print `204`; any other HTTP code, including `200` or `500`, is a failed check. If envd is still starting, wait a few seconds and retry the health request. If it still fails, go to step 3. The version command must also succeed; compare its output with the envd version you installed (`2026.16` for the base image used above). Run the state check once more after both probes: ```bash docker inspect --format '{{json .State}}' "$cid" ``` The state must still show `"Status":"running"` and `"Running":true`. If the container has exited, go to step 3 even if both probes succeeded. A running container, a successful `204` response, and the expected version confirm basic local startup and envd readiness. If the final state check also passes, skip to step 4 to remove the test container. Then create a template and verify the SDK operations your application uses. Local checks do not exercise cluster image pulling, sandbox networking, or envd `/init`. **3. If a check fails, inspect the state and logs before removing the container.** ```bash docker inspect --format '{{json .State}}' "$cid" docker logs --tail 100 "$cid" logdir=$(mktemp -d) docker cp "$cid":/var/log/envd.log "$logdir/envd.log" && tail -n 100 "$logdir/envd.log" ``` Use `ExitCode`, `OOMKilled`, and `Error` in the state output together with the startup logs to find the cause. `docker cp` can retrieve the envd log even when the container has stopped. If that file does not exist, check the startup output and the log path configured by your entrypoint. After collecting the diagnostics, remove the container in step 4. Fix the image, then repeat from step 1. **4. Clean up after verification or troubleshooting.** ```bash docker rm -f "$cid" ``` If you copied logs, they remain in `$logdir` for inspection and can be deleted when no longer needed. ## 6. Troubleshooting | Symptom | Likely cause | Fix | | --------------------------------------------- | --------------------------------------------------------------------- | ----------------------------------------------------------------------------------- | | Template creation fails the readiness probe | envd did not start / started on the wrong port | Ensure `ENTRYPOINT` invokes `cube-entrypoint.sh` **or** your own script runs `envd -port 49983 &` before `exec`. | | `curl :49983/health` returns `000` | Nothing is listening; entrypoint replaced | Check docker inspect --format '{{json .Config.Entrypoint}}'; keep `cube-entrypoint.sh` as the wrapper. | | envd exits immediately | Version mismatch between binary and kernel/init expectations | Verify with `docker exec ... /usr/bin/envd -version`; re-copy from the pinned base tag. | | envd `/init` slow or `create_time env_vars` fail | `-isnotfc` flag missing; envd attempts invalid MMDS access at `169.254.169.254` | Use `cube-entrypoint.sh` (appends `-isnotfc` automatically), or add `-isnotfc` to the command line when starting envd manually. | | Port 49983 conflicts with your own service | Your app also listens on 49983 | Move your app to a different port and expose both with `--expose-port`. | | `sudo: command not found` in your CMD | You started `FROM` a `-slim` / `-alpine` image without sudo | Either `apt-get install -y sudo`, or drop `sudo` from your entrypoint — `cube-entrypoint.sh` doesn't require it. | | Template creation times out in `PULLING` | Registry unreachable from Cube nodes | Push to a registry the cluster can reach, or supply `--registry-username` / `--registry-password`. | ## 7. Advanced — rebuild the base image yourself The base image is produced by a single GitHub Actions workflow in this repository: [`.github/workflows/build-envd-base-image.yml`](https://github.com/TencentCloud/CubeSandbox/blob/master/.github/workflows/build-envd-base-image.yml). It checks out `e2b-dev/infra` at the chosen tag (default `2026.16`), compiles `envd` with Go 1.25.4 in-place on native `linux/amd64` and `linux/arm64` runners, builds `docker/Dockerfile.cube-base`, runs a `:49983/health` smoke test on each architecture, then publishes a multi-arch manifest list to `ghcr.io/tencentcloud/cubesandbox-base`.