# GPU server Singapore for AI inference: a setup guide

URL: https://cloudj.net/blog/llm-inference-gpu-singapore
Published: 2026-11-12 · Updated: 2026-11-12

> Set up a GPU server Singapore for AI inference: search SG, MY and TH, size VRAM, install vLLM yourself and front it with a Singapore VPS running Nginx and TLS.

This is a hands-on guide for teams who need a GPU server Singapore for AI inference: an LLM API that answers users across Southeast Asia without a long round trip to another continent. We will search for a GPU in Singapore on Cloudj, size the VRAM, install an inference server ourselves, and put a Singapore VPS with Nginx and TLS in front so the public endpoint stays stable even when the GPU machine changes.

For the broader picture of Singapore GPUs, see the [GPU server Singapore](https://cloudj.net/blog/gpu-server-singapore) article.

## The architecture in one paragraph

Users call `https://api.example.com`. That domain points to a small Cloudj VPS in Singapore running Nginx with a TLS certificate. Nginx forwards requests through an SSH tunnel to the GPU instance, where vLLM serves the model on a local port. The GPU never exposes the model port to the internet. If the GPU host goes away, you deploy a new one, restart the tunnel, and users notice nothing. The [VPS front end with a GPU backend](https://cloudj.net/blog/vps-frontend-gpu-backend) article explains why this split is worth it.

## Step 1: size the VRAM

Pick the GPU by the model you plan to serve. As general guidance for a single card:

| Model size and precision | Typical VRAM need | Cards to search |
|---|---|---|
| 7B to 8B, 8-bit or 4-bit | Roughly 8 to 12 GB plus KV cache | [RTX 4090](https://cloudj.net/gpu-cloud/rtx-4090) (24 GB) |
| 13B to 14B, 4-bit or 8-bit | Roughly 10 to 18 GB plus cache | RTX 4090 (24 GB) |
| 30B to 34B, 4-bit | Roughly 20 GB plus cache | [L40S](https://cloudj.net/gpu-cloud/l40s) or [RTX A6000](https://cloudj.net/gpu-cloud/rtx-a6000) (48 GB class) |
| 70B, 4-bit | Roughly 40 GB plus cache | 48 GB class, tight; consider more VRAM |

Leave headroom. Concurrency and context length grow the KV cache, and running out of memory under load is worse than a slightly larger card. L40S and RTX A6000 are datacenter and workstation cards that need a higher account level; the console tells you if one is not available to you.

## Step 2: search Singapore and neighbours

GPU location is a country filter over a live list of host machines. Open the [RTX 4090 deploy page](https://app.cloudj.net/deploy/gpu?model=rtx-4090) and set the **Country** field to `SG`. Each result shows the country (no city), reliability, max duration, download speed, bandwidth price and a **Datacenter** badge where relevant.

If a machine is listed, check two things before deploying: the "available until" date should be well beyond your planned service period, and reliability should be high, since this is a user-facing service. If nothing fits, stock changes during the day; widen the search to nearby countries:

```text
SG, MY, TH
```

Then test latency from your users' networks before going live. For a service, also set a minimum reliability. Live "from" prices per model are on [GPU cloud](https://cloudj.net/gpu-cloud).

## Step 3: install the inference server

Deploy with the Ubuntu + CUDA template and an SSH key. None of the console templates comes with vLLM preinstalled; you install it yourself. Connect using the host and port from the **Connection** tab:

```bash
ssh -p <port> root@<host>
nvidia-smi
python3 -m venv /opt/vllm && . /opt/vllm/bin/activate
pip install vllm
```

Start the OpenAI-compatible server bound to localhost only, with an API key:

```bash
vllm serve Qwen/Qwen2.5-7B-Instruct \
  --host 127.0.0.1 --port 8000 \
  --api-key "$VLLM_API_KEY" \
  --max-model-len 8192
```

Run it under `tmux` or a process manager so it survives your SSH session. The [vLLM documentation](https://docs.vllm.ai/) covers quantized models, tuning and supported architectures. Ollama is a simpler alternative for small teams; the same front-end pattern applies. For a longer walkthrough of vLLM itself, see the [vLLM inference server](https://cloudj.net/blog/vllm-inference-server-usa) article.

## Step 4: tunnel from the VPS to the GPU

On a [Singapore VPS](https://cloudj.net/cloud-servers/singapore), create an SSH key, add its public key to the GPU instance's `~/.ssh/authorized_keys`, then open a persistent tunnel:

```bash
ssh -N -o ServerAliveInterval=30 -o ExitOnForwardFailure=yes \
  -L 127.0.0.1:8000:127.0.0.1:8000 -p <port> root@<host>
```

Wrap it in a systemd service (or `autossh`) so it restarts on failure. Now `http://127.0.0.1:8000` on the VPS reaches vLLM on the GPU, and nothing on the GPU is publicly exposed beyond SSH.

## Step 5: Nginx and TLS on the VPS

A minimal server block:

```nginx
server {
    server_name api.example.com;
    location / {
        proxy_pass http://127.0.0.1:8000;
        proxy_http_version 1.1;
        proxy_buffering off;
        proxy_read_timeout 300s;
    }
}
```

`proxy_buffering off` lets streamed tokens reach the client as they are generated. Issue a certificate with certbot, add rate limiting, and keep the API key check. The [Nginx reverse proxy with SSL](https://cloudj.net/blog/nginx-reverse-proxy-ssl-vps) guide covers the full setup.

## Step 6: health checks and failover by hand

vLLM exposes a `/health` endpoint. From the VPS, poll it every minute:

```bash
curl -fsS http://127.0.0.1:8000/health || echo "GPU backend down"
```

Alert on failure. GPU machines are independent hosts with no hardware SLA, and there is no automatic move if one goes offline. Keep a deploy script that installs vLLM and pulls the model, so that you can bring up a replacement in `SG` or a neighbour and repoint the tunnel quickly. Return a friendly error from Nginx while the backend is down.

## FAQ

### Does Cloudj provide vLLM preinstalled?

No. You install vLLM or Ollama yourself on the Ubuntu + CUDA template, or build a custom public image if your account level allows.

### Is a Singapore GPU always available?

No. Stock changes during the day. Search `SG`; if nothing fits, add `MY` and `TH` and test latency.

### Why not expose vLLM directly on the GPU?

The GPU machine can change. A VPS gives you a stable domain, TLS and access control, and keeps the model port private.

### Which GPU should I start with?

For 7B to 14B models, an RTX 4090 is a common starting point. Move to a 48 GB class card for larger models or heavy concurrency.

## Start building

Open the [RTX 4090 deploy page](https://app.cloudj.net/deploy/gpu?model=rtx-4090), set the Country to `SG` and see what is listed now. Compare live prices on [GPU cloud](https://cloudj.net/gpu-cloud) before you deploy.