# SSH GPU server access: connect to a cloud GPU and check it

URL: https://cloudj.net/blog/ssh-into-gpu-instance
Published: 2026-11-12 · Updated: 2026-11-12

> SSH GPU server guide: keys, the host and port from the Connection tab, the short key delay, nvidia-smi checks, tmux for long jobs and port forwarding.

Every Cloudj GPU instance is reached the same way: over SSH, with a key. There is no password login, so getting SSH right is the first skill you need. This SSH GPU server guide covers creating and adding a key, finding the host and the port, which is not always 22, what to do in the first few minutes after deploy, how to confirm the GPU with `nvidia-smi`, and the two habits that save most beginners: `tmux` and port forwarding.

## Step 1: create an SSH key

If you already use SSH keys for servers, you can reuse one. Otherwise, create one on your computer. On macOS, Linux and Windows 10 or 11 (PowerShell includes OpenSSH):

```bash
ssh-keygen -t ed25519 -C "my-laptop"
```

Press Enter to accept the default location and set a passphrase. This creates a private key, which never leaves your computer, and a public key ending in `.pub`. [Setting up SSH keys for a VPS](https://cloudj.net/blog/ssh-key-vps-setup) explains keys in more detail; the same key works for GPU instances.

Print the public key so you can copy it:

```bash
cat ~/.ssh/id_ed25519.pub
```

## Step 2: add the key and select it at deploy

Add the public key to your Cloudj account in the console. When you deploy a GPU, you must select at least one SSH key; deploy will not proceed without one. If you work in a team, add each person's key and select all of them.

Choosing the machine itself, including its country, is covered in [choosing a GPU server country in the console](https://cloudj.net/blog/filter-gpu-by-country-console).

## Step 3: find the host and port

Once the instance is running, open its **Connection** tab. It shows:

- the **host**,
- the **port**, which is often not 22,
- the full SSH command, ready to copy.

The command looks like this:

```bash
ssh -p PORT root@HOST
```

Always copy the port from the Connection tab rather than assuming 22.

### The first few minutes

A newly added key can take up to 3 minutes to be accepted after the instance starts. If you see `Permission denied (publickey)` right after deploy, wait a moment and try again before you change anything.

## Step 4: make connecting easier

Add an entry to `~/.ssh/config` on your computer:

```text
Host gpu1
    HostName HOST
    Port PORT
    User root
    IdentityFile ~/.ssh/id_ed25519
    ServerAliveInterval 30
```

Now `ssh gpu1` connects. `ServerAliveInterval` sends a small keep-alive every 30 seconds, which helps on connections that drop idle sessions. Update the entry when you deploy a new instance.

## Step 5: check the GPU

Once connected, confirm what you rented:

```bash
nvidia-smi
```

You should see the GPU model, its memory, the driver version and the highest CUDA version the driver supports. A few more useful checks:

```bash
nvidia-smi --query-gpu=name,memory.total,memory.used,utilization.gpu --format=csv
watch -n 2 nvidia-smi
df -h
```

The first prints a compact summary, the second refreshes every two seconds while a job runs, and the third shows free disk space. Disk size is fixed at deploy, so check it early.

If you use the PyTorch template, confirm PyTorch sees the GPU:

```bash
python -c "import torch; print(torch.cuda.is_available(), torch.cuda.get_device_name(0))"
```

[Running PyTorch with the Jupyter template](https://cloudj.net/blog/run-pytorch-jupyter-template) covers that template in detail.

## Step 6: keep long jobs alive with tmux

If your SSH connection drops, anything running in that session stops. Run long jobs inside `tmux`:

```bash
tmux new -s train
python train.py
```

Detach with `Ctrl+b` then `d`. Reconnect later and reattach with `tmux attach -t train`. If `tmux` is missing, install it with `apt-get install -y tmux`.

## Step 7: reach web tools with port forwarding

Jupyter, ComfyUI, model APIs and dashboards listen on a port on the GPU instance. Do not expose them publicly; forward the port over SSH instead:

```bash
ssh -p PORT -L 8188:localhost:8188 root@HOST
```

Then open `http://localhost:8188` in your browser. Change the numbers for other tools, for example `-L 8888:localhost:8888` for Jupyter or `-L 11434:localhost:11434` for an Ollama API. [Running Ollama on a GPU server in Asia](https://cloudj.net/blog/ollama-gpu-server-asia) uses this pattern.

## Moving files

```bash
rsync -avP -e "ssh -p PORT" ./data/ root@HOST:/workspace/data/
scp -P PORT root@HOST:/workspace/results.zip .
```

Note that `scp` uses a capital `-P` for the port. `rsync -P` resumes interrupted transfers, which matters for large models.

## Troubleshooting

| Symptom | Likely cause | Fix |
|---|---|---|
| `Permission denied (publickey)` just after deploy | Key not yet accepted | Wait up to 3 minutes, retry |
| `Permission denied (publickey)` later | Wrong key, or key not selected at deploy | Use `-i` with the right key; redeploy with the key selected |
| `Connection refused` or timeout | Wrong port, or instance not running | Copy the port from the Connection tab; check status on the instance page |
| `Host key verification failed` | A new instance answers on a host and port you used before | Remove the old entry with `ssh-keygen -R "[HOST]:PORT"` |
| Session drops when idle | Network closes idle connections | Add `ServerAliveInterval 30` |

The instance page also shows status, GPU utilisation and the last lines of the container logs, which help when a container does not start.

## FAQ

### Can I log in with a password?

No. GPU instances accept SSH keys only.

### Why is my port not 22?

The SSH port is assigned per instance and can be any number. Always use the one in the Connection tab.

### Can several people use the same instance?

Yes. Add each person's public key to the account and select all of them at deploy.

### How do I open Jupyter or ComfyUI?

Forward the port with `ssh -L` and open it on localhost in your browser.

## Next step

Open the [GPU deploy page](https://app.cloudj.net/deploy/gpu), deploy a small instance with your key selected, and run through the checks above. Live "from" prices for every model are on [GPU cloud](https://cloudj.net/gpu-cloud).