The real cost of an eviction was never the downtime — it's the hours of progress that vanish with it. SpotWarp is built for SaladCloud. It runs continuous automatic backups of your container workspace to your own machine, and when a node is reclaimed it restores that workspace onto a replacement SaladCloud node and restarts your job, so recovery is hands-off too.
curl -fsSL https://gpu-action.com/install.sh | bash
Built for SaladCloud Container Engine
SkyPilot (UC Berkeley) is a respected open-source tool for running jobs across many clouds. SpotWarp is a focused failover client built for SaladCloud. Here is how they differ.
| Dimension |
SkyPilot (Open Source)
|
SpotWarp
Built for SaladCloud
|
|---|---|---|
| Supported Platforms |
Many Clouds
Multi-Cloud and Kubernetes
AWS, GCP, Azure, Kubernetes, RunPod, Lambda and more. SaladCloud is not in its documented list.
|
SaladCloud
Built for SaladCloud
Watches and recovers your container group through the SaladCloud public API.
|
| Setup |
Per-Cloud Credentials
Connect Each Cloud
Install, then connect each cloud with its own CLI or credential file and check it with
sky check. |
Two Commands
One Salad API Key
pip install spotwarp, then spotwarp init with your SaladCloud API key, organization and project. |
| Where Checkpoints Live |
Bucket or Volume
Storage You Provide
A cloud bucket mounted with
file_mounts, or a Kubernetes volume. The instance's local disk is not kept across a recovery. |
No Bucket
Your Own Machine
/workspace is copied to your machine over SSH in the background. No bucket, no storage bill. |
| Keys |
Cloud Credentials
Configured per Cloud
Uses your cloud provider credentials, set up for each cloud you use.
|
No Keys in the Container
Key Stays on Your Machine
Your SaladCloud API key never leaves your machine, and the container holds no credentials at all.
|
| Job Definition |
YAML
SkyPilot Task File
Jobs are described in a SkyPilot YAML file and launched with
sky jobs launch. |
No Job File
Your Container, Unchanged
Your container runs as usual.
--resume-cmd restarts your own training command after a restore. |
| Recovery |
New Cluster
Relaunch Anywhere
Provisions a new cluster, possibly in another region or cloud, and restarts the job.
|
Warm Standby
Same Container Group
Uses the replacement node in the same Salad container group (a warm standby by default), restores
/workspace and runs your resume command. 17.1 s in our test with the standby running. |
| Price |
Free
Open Source
Free and open source.
|
Subscription
$49 / month or $490 / year
Open-source client. Salad users get 50% off the first 3 months with code SALAD.
|
SkyPilot is an excellent open-source tool when you run across many clouds and already keep checkpoints in cloud storage.
If you run on SaladCloud and want your checkpoints on your own machine, restored automatically without buckets or job files, SpotWarp is built for exactly that.
Your workspace is synced to your local machine in the background the whole time your instance runs — not just triggered after an eviction. This is the part that actually prevents loss; everything else just makes recovering from it fast.
SpotWarp checks your SaladCloud container group every 5 seconds and, by default, keeps a second replica running as a warm standby. When your active node is lost, it restores your workspace onto the standby and restarts your job. The standby is billed by SaladCloud like any other replica.
Your SaladCloud API key NEVER leaves your host machine. The spotwarp daemon executes locally on your server. Only subscription validation is verified centrally.
We believe developers should never trust closed binaries with production access. The client daemon is open source and auditable on GitHub.
Audit every single line of Python code before running on your infrastructure.
The container image used by our SaladCloud recipe (PyTorch 2.1 / CUDA 12.1). The failover daemon itself runs on your own machine.
docker pull ghcr.io/enplabs/spotwarp:pytorch2.1
No tiers, no per-GPU counting. One license protects everything running on that machine. You pay SaladCloud directly for your containers. This is just the Guard subscription.
Backed by a 14-day money-back guarantee on both plans. Cancel anytime.
Flexible monthly billing. Cancel anytime.
Save 17% ($98/yr) with annual billing
Everything you need to know about SpotWarp pricing, security, and failover protection.
What We've Actually Tested
SpotWarp is new, so instead of quoting customers we don't have yet, here is the real test record from SaladCloud: what we deployed, what we forced to fail, and what recovered. It is one run, so treat it as a measurement, not a guarantee.
Forced node loss on live SaladCloud
Ran the real daemon (the code released as v3.5.1) against a live SaladCloud container group, then forced the active node to be reallocated through the SaladCloud API. With the warm standby already running, the daemon restored the workspace onto it and restarted the job 17.1 seconds after the node was killed.
Backup and restore integrity
A 5 MB checkpoint written on the first node reached the local machine 16 seconds after the daemon started. After the restore, the file on the replacement node had the same SHA-256 hash as the original, and the resume command ran on the new node.
Allocation is SaladCloud's time
In the same test, SaladCloud took about 8 minutes to give us the first running node (allocation, image download and container start), and the standby needed about 4 minutes after the daemon started before it was ready. The 17.1 seconds applies once a standby is running. Without one, a replacement has to be allocated first, which takes minutes.
A bug our own testing caught
In v3.5.0 the warm-standby step failed silently because of a missing helper, so the daemon never added the second replica and recovery had to wait for a brand-new node. We found it by running the real daemon end to end and fixed it in v3.5.1. We mention it because this is the kind of thing you only find by actually deploying.
You don't have to take our word
The client daemon is open source, the package is public on PyPI, and every release binary ships with a SHA256 checksum. Read the failover code before you run it. That is the point of shipping it open.
And what we haven't done yet
--no-standby; without it, recovery waits for SaladCloud to allocate a new node.scpand copy the whole workspace each cycle. Linux and macOS usersyncand copy only what changed.We would rather you read this before subscribing than discover it afterwards. The 14-day money-back guarantee covers you either way.