Use the Proxmox backend

Run the same VM on a remote Proxmox node. The interface is identical to the local backend — only TARGET changes — and your host needs only Docker (no libvirt/qemu).

Validated end-to-end against a real PVE 8 node (boot → SSH → guest Docker + Claude Code). Attribute names track bpg/proxmox; if your provider version differs, expect to adjust a name or two.

You need three things from your Proxmox node: an API token, the names of the node/datastores/bridge ztd should use, and snippets enabled on a datastore. This guide walks through each.

1. Mint an API token

ztd authenticates with a Proxmox API token, not your login password. A token is a USER@REALM!TOKENID=UUID string tied to a user and a set of privileges.

Create a dedicated user + role (recommended)

Run these on the Proxmox node (SSH in as root, or use the node shell). They make a ztd user, a role with exactly the privileges the backend needs, grant it at the root path /, and mint the token:

# A role scoped to what ztd does: create/configure VMs, allocate disk, download
# the cloud image, upload the cloud-init snippet, and read the guest agent for the
# IP. Privileges are comma-separated. Three are easy to miss and each fails late:
#   Sys.AccessNetwork  — PVE 8 gates the image URL download behind it (anti-SSRF)
#   Sys.Modify         — the URL-metadata query needs it (else the download 403s)
#   VM.GuestAgent.Audit — reading the guest agent for the VM's IP (else ip/ssh fail)
#   VM.Audit           — listing existing VMs to auto-pick a free VMID (below)
pveum role add ZTDProvision -privs \
"VM.Allocate,VM.Clone,VM.Config.CDROM,VM.Config.Cloudinit,VM.Config.CPU,\
VM.Config.Disk,VM.Config.HWType,VM.Config.Memory,VM.Config.Network,\
VM.Config.Options,VM.Audit,VM.PowerMgmt,VM.GuestAgent.Audit,Datastore.Allocate,\
Datastore.AllocateSpace,Datastore.AllocateTemplate,Datastore.Audit,\
Sys.Audit,Sys.Console,Sys.Modify,Sys.AccessNetwork,SDN.Use"

pveum user add ztd@pve
pveum acl modify / -user ztd@pve -role ZTDProvision

# Mint the token. Privilege separation OFF so the token inherits the user's role.
pveum user token add ztd@pve terraform --privsep 0

The last command prints a table — copy the value (the UUID) now, it is shown only once. Your full token is:

ztd@pve!terraform=<that-uuid>

Quick path (root token)

For a first smoke test you can skip the dedicated user and mint a token for root@pam instead — it inherits root’s full privileges:

pveum user token add root@pam terraform --privsep 0
# token: root@pam!terraform=<uuid>

This works but is over-privileged; move to the dedicated user once it boots.

GUI alternative: Datacenter → Permissions → API Tokens → Add, uncheck Privilege Separation, then grant the user a role under Permissions.

2. Drop the token in .ztd/secrets/

The endpoint and token are secrets, so they live in a gitignored env file — never in .ztd/ztd.toml. Copy the example and fill it in:

mkdir -p .ztd/secrets
cp examples/proxmox.env.example .ztd/secrets/proxmox.env
$EDITOR .ztd/secrets/proxmox.env
# .ztd/secrets/proxmox.env
export TF_VAR_proxmox_endpoint="https://proxmox.lan:8006/"
export TF_VAR_proxmox_api_token="ztd@pve!terraform=xxxxxxxx-...."

The ztd wrapper sources this file automatically and compose.yaml forwards both values into the toolbox. (Setting raw PROXMOX_VE_* env vars in your shell does not work — they never reach the container, and main.tf would override the endpoint with a placeholder. Use the file.)

2b. Make sure SSH to the node works

ztd uploads the cloud-init as a Proxmox snippet, and the provider does that over SSH to the node (the PVE API can’t upload snippets). The API token has no SSH identity, so this is a separate credential from step 1: your own SSH key must be authorized for a Linux user on the node.

You can SSH as root (simplest, proxmox_ssh_username defaults to it), but to avoid root logins create a dedicated, unprivileged Linux user and point ztd at it. This is independent of the ztd@pve API user — it’s a real account on the node (PAM), used only for the snippet upload:

# On the PVE node, as root. Pick any name; we use ztd-ssh.
useradd -m -s /bin/bash ztd-ssh

# Authorize YOUR public key (the one in your ssh-agent on the host):
install -d -m 700 -o ztd-ssh -g ztd-ssh /home/ztd-ssh/.ssh
$EDITOR /home/ztd-ssh/.ssh/authorized_keys     # paste ~/.ssh/id_ed25519.pub
chown ztd-ssh:ztd-ssh /home/ztd-ssh/.ssh/authorized_keys
chmod 600 /home/ztd-ssh/.ssh/authorized_keys

# Let ztd-ssh write the snippet. bpg uploads it by piping into `tee` over SSH —
# and in stream mode it does NOT use sudo (despite its docs), so a sudoers rule
# won't help: the snippets directory itself must be writable by ztd-ssh. A plain
# chown gets reset when PVE recreates the directory, so use an inheriting ACL,
# which survives up/down cycles (requires the `acl` package):
apt-get install -y acl
# default ACL on the parent → a recreated snippets dir inherits the grant:
setfacl -m d:u:ztd-ssh:rwx /var/lib/vz
# the current dir + a default ACL so the files tee creates inside are writable:
setfacl -m u:ztd-ssh:rwx -m d:u:ztd-ssh:rwx /var/lib/vz/snippets

(tee opens the file for writing, so the file — not just the dir — must be writable; the default ACL handles that. Non-default snippet datastore? Apply the same to its path, e.g. /mnt/pve/<store>/snippets.)

Then tell ztd to use the user, in .ztd/ztd.toml:

[proxmox]
ssh_username = "ztd-ssh"

Either way:

  • Verify the login from your host: ssh <user>@<node> must succeed with no password (key only). For root, sshd needs PermitRootLogin prohibit-password.

  • Run an ssh-agent with that key loaded (ssh-add -l lists it). The ztd wrapper forwards your agent into the toolbox so the upload can authenticate. This is required, not optional — the provider is configured with ssh { agent = true }, so with no agent up fails at snippet upload every time, and TARGET=proxmox ./ztd check fails accordingly. An agent is per-login session, so if ssh-add -l says “no identities” you need it again:

    eval "$(ssh-agent -s)"
ssh-add ~/.ssh/id_ed25519

  • Confirm ztd-ssh can write the snippet dir: sudo -u ztd-ssh sh -c 'echo ok > /var/lib/vz/snippets/.probe && rm /var/lib/vz/snippets/.probe'

Root SSH avoids all of this. If proxmox_ssh_username = "root", bpg writes the snippet as root and no ACL is needed — at the cost of a root login. The ACL path above is the price of a non-root SSH user.

3. Find your node, datastore, and bridge names

These are non-secret, so they go in .ztd/ztd.toml. Discover them on the node:

pvesh get /nodes                # the node NAME (it's the hostname, often NOT "pve")
pvesm status                    # datastore names + content types
ip -br link show type bridge    # bridges → e.g. "vmbr0"

Get proxmox_node exactly right. It’s the node’s hostname, not the literal string pve. A wrong name doesn’t error cleanly — the cluster proxies storage and snippet calls to a member that doesn’t exist and hangs until a multi-minute HTTP 595 timeout. ./ztd check (step 6) verifies it for you.

Then set them in .ztd/ztd.toml (uncomment the [proxmox] block):

[proxmox]
insecure          = true        # if the node uses a self-signed cert
node              = "pve"        # ← replace with YOUR node name from above
datastore         = "local-lvm" # VM disk (must allow "images")
image_datastore   = "local"     # downloaded cloud image (must allow "import")
snippet_datastore = "local"     # cloud-init (must allow "snippets")
bridge            = "vmbr0"

4. Enable the right content types on the datastores

In pvesm status each datastore lists its allowed content types. The three datastores above need, respectively, images, import, and snippets. Enable any that are missing under Datacenter → Storage → (datastore) → Edit → Content, or from the shell:

# Example: allow snippets + import on the `local` datastore.
pvesm set local --content iso,vztmpl,backup,snippets,import

(The default local datastore usually needs snippets and import added; they are off by default.)

5. Get your code into the guest

There’s no host filesystem to live-mount over a WAN, and the guest never clones from a forge (it holds no forge credentials). Instead, after up, sync your working tree up over the ephemeral key:

TARGET=proxmox ./ztd mount     # sshfs-syncs your tree (incl. .git) into the guest

Pull the agent’s commits back with TARGET=proxmox ./ztd fetch and push from the host — see Publish the agent’s results.

6. Run the preflight

Before the first up, validate the whole setup against your node. The Proxmox backend has two credentials that fail late and cryptically if wrong — the API token and SSH to the node — so check exercises both for real:

TARGET=proxmox ./ztd check

It runs on your host (not in the toolbox) and verifies, against the live node:

  • .ztd/secrets/proxmox.env is present and the endpoint/token are filled in;
  • the API token is accepted (/version over the endpoint, honouring proxmox_insecure);
  • proxmox_node resolves (a node-scoped call — catches the wrong-name → 595 trap from step 3);
  • the token can download images (a URL-metadata probe — catches a role missing Sys.AccessNetwork);
  • an ssh-agent key is loaded and ssh <proxmox_ssh_username>@<node> logs in — the snippet-upload leg from step 2b;
  • the snippet and image datastores carry the snippets and import content types from step 4.

Fix anything it flags and re-run until every line is [ ok ]. Each [MISS] prints the exact command to fix it. (./ztd check with no TARGET still checks the local kvm backend — the two are independent.)

7. Bring it up

TARGET=proxmox ./ztd init   # first run only — pulls the proxmox provider
TARGET=proxmox ./ztd up
TARGET=proxmox ./ztd ssh
TARGET=proxmox ./ztd down

Every command takes the same TARGET=proxmox prefix. Omit it and you’re back on the local kvm backend.

A few things to expect on up:

  • It waits for the guest’s IP. ztd resolves the address from the guest agent via the API (bpg’s own ipv4_addresses field stays empty), so up polls for a few minutes while cloud-init installs and starts qemu-guest-agent, then prints IP: <addr>. ssh/ip use the same source.
  • First boot can kernel-panic, and up self-heals. Debian cloud images occasionally panic at init (Attempted to kill init) on a fresh boot under node I/O pressure — it’s a race, not a misconfig. up detects the missing agent IP, resets the VM via the API (this is why the role needs VM.PowerMgmt), and retries up to 3×; a quiescent reboot always comes up clean.

Tearing down with an offline storage

If any datastore on the node is permanently offline (e.g. a dead NFS/PBS mount), qm destroy — and therefore TARGET=proxmox ./ztd down — fails with storage '<name>' is not online, even though the VM’s own disks were removed. qm destroy scans every storage to purge references. Disable the dead storage so PVE stops scanning it, then re-run down:

pvesm set <name> --disable    # or `pvesm remove <name>` to delete its config
TARGET=proxmox ./ztd down

Troubleshooting

Most of these are caught up front by TARGET=proxmox ./ztd check (step 6) — run it first; it names the exact fix. The notes below are for when an error still slips through to up.

  • 401 authentication failure — the token string is wrong, or privilege separation was left on and the token has no role. Re-check the USER@REALM!ID=UUID form and that you minted with --privsep 0 (or assigned the token its own ACL).
  • certificate signed by unknown authority — self-signed PVE cert; set proxmox_insecure = true.
  • datastore '…' does not support content type 'snippets'/'import' — step 4.
  • HTTP 400 … invalid filename or wrong extension (on proxmox_download_file) — your PVE version disagrees with the image’s filename extension. The module names it .qcow2 for PVE ≥ 8.4; PVE < 8.4 wants .img instead. Flip file_name in modules/vm-proxmox/main.tf if you’re on the older line.
  • Error initiating file download … HTTP 403 … Permission check failed (on proxmox_download_file) — the image URL download is blocked. Two causes, both in step 1: the role lacks Sys.AccessNetwork/Sys.Modify, or proxmox_node names a node that doesn’t exist (the metadata query is routed to a bogus node path and the permission check fails). Check pvesh get /nodes and the role privileges.
  • HTTP 595 … Connection timed out / error listing files from datastore — proxmox_node is wrong. The cluster proxies node-scoped calls (storage, snippet upload) to the named member, and a non-existent name hangs until timeout. Set proxmox_node to a name from pvesh get /nodes. (./ztd check now catches this as “node ‘…’ not reachable”.)
  • 403 Permission check failed (general) — the role is missing a privilege; the ZTDProvision set in step 1 covers a normal up, widen it if your node differs.
  • tee: /var/lib/vz/snippets/… Permission denied (on the cloud-init file) — the non-root SSH user can’t write the snippets dir. bpg does not use sudo for the stream upload, so a sudoers rule won’t help and a plain chown gets reset when PVE recreates the dir. Apply the inheriting ACL from step 2b (setfacl … d:u:ztd-ssh:rwx), or switch proxmox_ssh_username to root.
  • ip/ssh fail with no agent-reported IP — ztd reads the IP from the guest agent over the API. Either the agent isn’t up yet (cloud-init still installing it — wait), or the token’s role lacks VM.GuestAgent.Audit (step 1). Check with qm guest cmd <vmid> network-get-interfaces on the node.
  • Guest kernel-panics on boot (Attempted to kill init) — an early-boot race, not a misconfig. up auto-resets past it (needs VM.PowerMgmt); to clear one by hand, qm reset <vmid> on the node — a quiescent reboot comes up clean.
  • snippet upload fails / SSH errors / hangs at the cloud-init file — the node-SSH leg (step 2b), not the API token. Confirm ssh <user>@<node> works and ssh-add -l shows your key. With no agent, add private_key to the provider ssh {} block instead (a PEM key file readable in the toolbox), or load an agent and retry.

VM IDs

Proxmox’s default “next free ID” starts at 100 — the same range you hand-number your own VMs — so ztd instead picks a primary 132124044, falls back to 90084068 if that’s taken, and finally a random ID between them, keeping its disposable VMs out of your way. Auto-selection lists existing VMs (needs VM.Audit, already in the role above); it doesn’t see LXC containers, so if one squats on an anchor, apply errors — set an explicit ID with proxmox_vm_id (or TF_VAR_proxmox_vm_id). IDs must be 100–999999999 and are unique cluster-wide.

Getting your code into a remote guest

The virtiofs live-mount works locally but not over a WAN, so on Proxmox cloud-init just creates the (empty) work dir. Sync your working tree up with ./ztd mount (sshfs, first mount seeds it, incl. .git) and pull the agent’s commits back with ./ztd fetch — both operator-initiated over the ephemeral key. The guest never clones from or pushes to a forge; the host publishes results. See Repo sync.

Backward How-to guides Forward a port from the VM Forward