NVIDIA releases open-source Snapshot for fast GPU workload restore
First seen Nov 10 Updated Nov 10, 16:12 UTC Heat 21 high evidence
NVIDIA released Snapshot, an open-source checkpoint and restore system for GPU workloads on Kubernetes. It captures initialized CPU and GPU memory state. NVIDIA says restoring from it brings inference replicas online much faster than cold starts.
- What to do
- Read the NVIDIA Snapshot repository and follow the Dynamo user guide to restore workers.
- Who should care
- Platform and inference engineers running GPU serving on Kubernetes, especially with NVIDIA Dynamo.
Why it matters: Teams running GPU inference on Kubernetes can try restoring replicas from checkpoints instead of waiting minutes for cold starts.
Key facts
- Restore time
- 3.6 to 42.7 seconds
- Speedup
- 8x to 36x over cold starts
- Checkpoints up to 1 TB across eight GPUs
- Checkpoint creation
- 15 seconds to 5.5 minutes
- Benchmarked on B200 GPUs with VAST NFS storage
- GPU state restore takes 18 to 29 seconds on large restores
Timeline
Official
-
Start and Restore GPU Workloads in Seconds with NVIDIA Snapshotdeveloper.nvidia.com ยท NVIDIA
Evidence
High evidence: the claim comes from an official source (the company or project itself), or at least two independent sites reported it.
Details written Nov 10, 16:11 UTC from the sources listed above.