For SaladCloud users

Keep your training progress when a Salad node goes away

You deployed the SpotWarp recipe. Connect the SpotWarp client on your own machine, and your container's /workspace is backed up continuously and restored onto the replacement node automatically, with your job restarted for you.

For Salad users: start with a 14-day evaluation, no license key needed. When you subscribe, code SALAD gives you 50% off your first 3 months.

Set it up in a few minutes Pricing with the SALAD discount

Why SpotWarp

No keys inside the container

The container holds no storage credentials at all. Your own machine pulls the workspace over SSH, and your SaladCloud API key never leaves that machine.

Your checkpoints, on your disk

Checkpoints and model weights land on your own disk, not in a third-party bucket. No storage account, no sync code, no storage bill.

Automatic restore and restart

When the node is lost, SpotWarp restores /workspace onto the replacement node in the same container group and runs your resume command.

Seconds with a warm standby

With a standby replica running, our live test on October 21, 2026 restored the workspace and restarted the job in 17.1 seconds (one run, 5 MB checkpoint).

Install the client on your machine

Linux, macOS or Windows

Linux / macOS

curl -fsSL https://gpu-action.com/install.sh | bash

Windows PowerShell

irm https://gpu-action.com/install.ps1 | iex

Or with pip

pip install --upgrade spotwarp

Connect it to your Salad project

one-time setup

terminal

spotwarp init

It asks for your SaladCloud API key, organization and project, and the container group to protect. Press Enter at the license prompt to use the 14-day evaluation. Everything is saved on your machine only.

Start protecting your job

runs in the background

terminal

spotwarp start -d --container <container-group-name> \
  --resume-cmd "python3 /workspace/train.py --resume"

Check it any time with spotwarp status. Backups land in ./backups/<container-group-name>/ on your machine.

Before you start

Add your SSH public key to your SaladCloud account. SpotWarp reaches your container over SSH with that key.
Have your training save checkpoints under /workspace and resume from the latest one. SpotWarp moves and restores them for you.
Keep the machine running SpotWarp switched on while your job runs.

Good to know

The warm standby is a second running replica, billed by SaladCloud like any other. Start with --no-standby to keep one replica; recovery then waits for SaladCloud to allocate a new node.
On Linux and macOS, backups copy only what changed. On Windows they copy the whole workspace each cycle.
The client is open source: github.com/enplabs/spotwarp. Full documentation: gpu-action.com/docs.