Most of an ML project's test suite runs fine on CPU. The part that does not, CUDA kernels, mixed-precision paths, multi-GPU code, model loading on real hardware, tends to break quietly until someone runs it on a GPU by hand. A GPU CI runner closes that gap. Instead of buying a GPU box for CI, you can keep the CI runner on a small VPS and send only the GPU tests to a rented GPU instance over SSH, paying per minute and destroying the instance when the test window ends.
If you are still deciding which parts of your stack need a GPU at all, read GPU instance vs VPS for AI first.
The GPU CI runner pattern
git push -> CI runner on a VPS -> rsync code to GPU instance -> run GPU tests over SSH -> results back to CI- The CI runner lives on a VPS. It is always on, cheap, and holds the runner token, secrets and caches. The self-hosted GitHub Actions runner on a VPS guide shows how to set one up.
- The GPU instance is disposable. It holds a copy of the code for the duration of the run and nothing else of value.
- The two talk over SSH with a dedicated key. GPU instances on Cloudj accept SSH keys only.
This article assumes you deploy and destroy the GPU instance from the Cloudj console, for example at the start of a release week or a nightly test window, and give its host and port to the CI as secrets.
Pick a small card
CI tests check correctness, not speed. A small, inexpensive card usually does the job:
| Card | VRAM | Fits CI that |
|---|---|---|
| RTX 3060 | 12 GB | Runs CUDA unit tests, small model forward and backward passes, mixed-precision checks |
| RTX 4070 | 12 GB | Same memory on the newer Ada architecture, useful if you need to test Ada-specific code paths |
If you test multi-GPU code, pick a 2x listing; the GPU count selector offers 1x, 2x, 4x and 8x in one machine. Live "from" prices are on each model page and on GPU cloud.
Choose the region near your artefacts
For CI, distance to end users is irrelevant. What matters is the distance to where your code, wheels, datasets and model weights come from:
- if your CI VPS and package caches are in Europe, search with
DE, NL; - if they are in the US, use
US; - if your test fixtures download from a model hub, any well-connected country works.
Open the RTX 3060 deploy page, enter the codes, and check download speed in Mbps on each listing, since every CI run pulls code and dependencies. If nothing is listed, add neighbouring codes or clear the Country field; stock changes during the day. Location is shown at country level only.
Make the environment reproducible
Every run should start from a known state. Two options:
- Template plus setup script. Deploy with "Ubuntu 22.04 + CUDA (SSH)" or "PyTorch + JupyterLab" and let the CI job install the project's requirements on each run.
- Custom image. Build a public Docker image with your CUDA, Python and dependencies pinned, and deploy that. It makes runs start faster and identical every time. Custom images need a higher account level and must be public; private registries are not supported. See custom Docker images for GPU instances.
The CI script
A minimal script the runner executes, with GPU_HOST and GPU_PORT taken from the instance's Connection tab and stored as CI secrets:
#!/usr/bin/env bash
set -euo pipefail
SSH="ssh -p ${GPU_PORT} -o StrictHostKeyChecking=accept-new"
TARGET="root@${GPU_HOST}"
rsync -az --delete --exclude .git -e "$SSH" ./ "$TARGET:/workspace/repo/"
$SSH "$TARGET" 'cd /workspace/repo && nvidia-smi && pip install -q -r requirements.txt && pytest -m gpu -q --junitxml=gpu-results.xml'
rsync -az -e "$SSH" "$TARGET:/workspace/repo/gpu-results.xml" ./And a GitHub Actions job that runs it on the self-hosted runner:
gpu-tests:
runs-on: self-hosted
steps:
- uses: actions/checkout@v4
- run: ./ci/run-gpu-tests.sh
env:
GPU_HOST: ${{ secrets.GPU_HOST }}
GPU_PORT: ${{ secrets.GPU_PORT }}Mark GPU-only tests with a pytest marker so the CPU suite keeps running on ordinary runners.
Keep the bill small
GPU instances are billed per minute from your balance while running. The price shown at deploy is locked, and a deposit is held when you deploy.
- Batch the GPU work. Run GPU tests on merges to the main branch or nightly, not on every commit.
- Destroy after the window. Destroy deletes the instance and its disk and stops charges. A CI instance holds nothing you cannot recreate.
- Avoid Stop for CI. Stop keeps the disk but releases the GPU, so the next start may wait in "Waiting for GPU" and your pipeline would hang. Destroy and deploy fresh instead. The stop vs destroy guide explains the difference.
Handle a missing GPU gracefully
Every listing is an independent host with no hardware SLA, and each has an "available until" date. Make the CI job fail clearly, not hang, when the GPU is unreachable: add a short SSH connect timeout and a readable error message. If the host goes offline, billing pauses but the instance is not moved, so deploy a new one and update the secrets.
FAQ
Do I need a GPU for my whole test suite?
No. Mark GPU-specific tests and send only those to the GPU instance.
Which card should I use for CI?
A 12 GB card such as the RTX 3060 or RTX 4070 covers most correctness tests.
Can I use a private Docker registry?
No. Custom images must be public, and they need a higher account level.
Why not stop the instance between runs?
Stopping releases the GPU, so the next start may wait. Destroy and redeploy for predictable CI.
Set up your GPU test window
Open the RTX 3060 deploy page, pick a country near your artefacts and deploy a small instance for your next test window. Compare entry cards on GPU cloud.