PVM Deployment

When to use this guide: Your cloud server does not expose /dev/kvm (nested virtualization is blocked by the cloud provider). If your machine already has KVM support, refer to Quick Start or Self-Build Deployment instead.

Production Use

If you plan to use Cube Sandbox in a production environment, please refer to the Network Hardening guide to secure your deployment before exposing services to untrusted networks.

PVM enables you to deploy Cube Sandbox on an ordinary cloud server, with all sandbox instances running inside PVM-backed Micro-VMs.

Compared to a standard deployment, PVM adds only two extra steps:

1. Install the PVM host kernel and reboot
2. Pass CUBE_PVM_ENABLE=1 when running the Cube Sandbox installer

Production-ready

Tencent Cloud has deployed PVM instances at scale in production environments, with reliability validated in production. The improvements have been open-sourced in the OpenCloudOS kernel.

What is PVM? (Technical background)

PVM (Pagetable-based Virtual Machine) is a page-table-based nested virtualization framework built on top of KVM. Unlike conventional nested virtualization, PVM does not require the host hypervisor to expose hardware virtualization extensions (Intel VT-x / AMD-V) to the guest. Instead, it uses a minimal shared memory region between the guest and the guest hypervisor, combined with an efficient shadow page table design, to handle privilege-level transitions and memory virtualization — all transparently to the host hypervisor.

PVM was originally proposed in the paper PVM: Efficient Shadow Paging for Deploying Secure Containers in Cloud-native Environment. Tencent Cloud has since made extensive improvements in features, performance, and bug fixes, and has open-sourced the work in the OpenCloudOS kernel for the community. Over the years, we have deployed a large number of PVM instances in Tencent Cloud production environments, and their reliability has been validated in production.

Prerequisites

• x86_64 Linux server (cloud server or physical machine)
• Root access
• No /dev/kvm required — PVM provides KVM capability after the kernel switch
• All other requirements are the same as Quick Start (≥ 8 GB RAM, XFS-capable storage for /data/cubelet, etc.)

Run all commands as root

Every command in this guide must be run as root. Switch to the root user first:

sudo su root

Then run all subsequent commands directly in the root shell.

Step 0: Provision a Cloud Server

Purchase an x86_64 cloud server from any cloud provider — no special requirements apply.

OpenCloudOS 9 (RPM-based) is the recommended OS. The Cube Sandbox PVM host kernel is built on the OpenCloudOS kernel, so choosing OpenCloudOS 9 gives the best compatibility with the fewest distribution-specific differences to handle. Other mainstream distributions — Ubuntu, Debian, CentOS, etc. — are equally supported; just follow the corresponding section below.

Recommended specifications

• CPU: ≥ 4 cores
• RAM: ≥ 8 GB
• System disk: ≥ 50 GB (attaching a dedicated data disk for /data/cubelet is recommended)

Step 1: Install the PVM Host Kernel

Go to the CubeSandbox GitHub Releases page, open the latest release that includes PVM kernel assets, then right-click each asset → Copy link address and paste the URL into the wget commands below.

Choose the package format that matches your Linux distribution:

RPM-based (OpenCloudOS, RHEL, CentOS, TencentOS, Fedora)

Go to the Releases page, find kernel-*opencloudos9.cubesandbox.pvm.host*.x86_64.rpm, right-click and copy the download link:

# Replace the URLs below with the actual download links copied from the Releases page
wget "<kernel rpm download URL>"

# --oldpackage skips the version check if a newer kernel is already installed
rpm -ivh --oldpackage kernel-*.rpm

Set the PVM kernel as the default boot entry:

# List installed kernels and find the index of the PVM kernel
grubby --info=ALL | grep -E "^kernel|^index"

# Replace <index> with the number shown for the PVM kernel
grubby --set-default-index=<index>

# Verify
grubby --default-kernel

Configure the required kernel boot parameters:

bash <(curl -fsSL \
  https://raw.githubusercontent.com/TencentCloud/CubeSandbox/master/deploy/pvm/grub/host_grub_config.sh)

DEB-based (Ubuntu, Debian)

Go to the Releases page, find linux-image-*opencloudos9.cubesandbox.pvm.host*_amd64.deb, right-click and copy the download link:

# Replace the URLs below with the actual download links copied from the Releases page
wget "<linux-image deb download URL>"

dpkg -i linux-image-*opencloudos9.cubesandbox.pvm.host*.deb

Set the PVM kernel as the default boot entry:

# List installed kernels and note the PVM kernel version string
ls /boot/vmlinuz-*

# Point GRUB to the PVM kernel (the version string is read automatically)
KVER="$(ls /boot/vmlinuz-*opencloudos9.cubesandbox.pvm.host* | sed 's|/boot/vmlinuz-||' | tail -1)"
sed -i "s|^GRUB_DEFAULT=.*|GRUB_DEFAULT=\"Advanced options for Ubuntu>Ubuntu, with Linux ${KVER}\"|" \
  /etc/default/grub

Configure the required kernel boot parameters (the script also calls update-grub, applying the default-entry change above):

bash <(curl -fsSL \
  https://raw.githubusercontent.com/TencentCloud/CubeSandbox/master/deploy/pvm/grub/host_grub_config.sh)

Reboot and Verify

reboot

After rebooting, confirm you are running the PVM kernel and that the KVM module loads successfully:

# Confirm the kernel version
uname -r
# Expected output contains: opencloudos9.cubesandbox.pvm.host

# Load the PVM KVM module
modprobe kvm_pvm

# Confirm the module is loaded
lsmod | grep kvm
# Expected output includes kvm_pvm

Configure kvm_pvm to load automatically on boot:

echo 'kvm_pvm' > /etc/modules-load.d/kvm-pvm.conf

Already have Cube Sandbox installed?

If you have a running Cube Sandbox installation using the standard kernel, you do not need to reinstall it. Simply reboot into the PVM kernel and proceed to Step 2.

Step 2: Install Cube Sandbox with PVM Enabled

Why CUBE_PVM_ENABLE=1?

The release bundle ships two guest kernels: a standard one (vmlinux) and a PVM-optimized one (vmlinux-pvm). Setting CUBE_PVM_ENABLE=1 tells the installer to use the PVM guest kernel as the active runtime kernel. Without this flag, the standard guest kernel is used and PVM has no effect.

Option A: Online Install (Recommended)

The online install script downloads the release bundle to a temporary directory before running install.sh. Because the temporary directory contains no .env file, the CUBE_PVM_ENABLE=1 environment variable takes effect directly:

# Using GitHub (default)
curl -sL https://github.com/tencentcloud/CubeSandbox/raw/master/deploy/one-click/online-install.sh \
  | CUBE_PVM_ENABLE=1 bash

# Using the CN mirror for faster downloads in mainland China
curl -sL https://cnb.cool/CubeSandbox/CubeSandbox/-/git/raw/master/deploy/one-click/online-install.sh \
  | CUBE_PVM_ENABLE=1 MIRROR=cn bash

To explicitly set the node IP (recommended on machines with multiple network interfaces):

curl -sL https://github.com/tencentcloud/CubeSandbox/raw/master/deploy/one-click/online-install.sh \
  | CUBE_PVM_ENABLE=1 bash -s -- --node-ip=<your-server-ip>

Option B: Manual Download

Download cube-sandbox-one-click-<sha>.tar.gz from GitHub Releases, extract it, and pass the environment variable inline:

tar -xzf cube-sandbox-one-click-<sha>.tar.gz
cd cube-sandbox-one-click-<sha>

# The extracted directory contains no .env file, so the env variable takes effect
CUBE_PVM_ENABLE=1 ./install.sh

env file override pitfall

If you followed the developer guide and ran cp env.example .env, a .env file now exists in the directory with the default value CUBE_PVM_ENABLE=0. The installer sources this file, which overrides any same-named variable exported from the parent shell, silently disabling PVM.

To fix this, update the .env file before running the installer:

sed -i 's/^CUBE_PVM_ENABLE=0/CUBE_PVM_ENABLE=1/' .env
grep CUBE_PVM_ENABLE .env   # Verify: expected CUBE_PVM_ENABLE=1

./install.sh

Confirm a Successful Install

The install log should contain:

[one-click] CUBE_PVM_ENABLE=1, installed PVM guest kernel as .../cube-kernel-scf/vmlinux

Step 3: Verify the PVM Environment

# Confirm PVM is enabled in the runtime configuration
grep CUBE_PVM_ENABLE /usr/local/services/cubetoolbox/.one-click.env
# Expected: CUBE_PVM_ENABLE=1

# Confirm the KVM device is available
ls -la /dev/kvm

# Confirm the PVM KVM module is loaded
lsmod | grep kvm_pvm

Step 4: Create a Template and Get Started

With PVM up and running, the rest of the process is identical to a standard deployment. Follow Step 3: Create a Template in the Quick Start guide to create your first template and run a sandbox.

Troubleshooting

Q1: Install log shows using ordinary guest kernel instead of the PVM guest kernel

A: This is almost always caused by a .env file containing CUBE_PVM_ENABLE=0 overriding the environment variable. See the warning in Option B above.

---

Q2: lsmod | grep kvm_pvm returns no output, or /dev/kvm does not exist

A: Run uname -r to confirm you rebooted into the PVM kernel (the version string should contain opencloudos9.cubesandbox.pvm.host). Once confirmed, run modprobe kvm_pvm manually. If it still fails, verify the kernel packages are installed correctly:

# RPM-based
rpm -qa | grep cube.pvm

# DEB-based
dpkg -l | grep cube.pvm