# A100 vs H100 cloud: which GPU fits your training run?

URL: https://cloudj.net/blog/a100-vs-h100-cloud
Published: 2026-11-12 · Updated: 2026-11-12

> Compare A100 vs H100 cloud rentals: VRAM and bandwidth from the model pages, cost per finished run rather than per hour, the listed variants and account level.

The A100 vs H100 cloud question rarely has a universal answer. The H100 is the newer, faster card, and the A100 is older but proven. Which one you should rent depends on how much memory your job needs, how well your software uses the newer architecture, and, most of all, what each finished run costs. This article compares the variants on Cloudj using the figures printed on their model pages, then shows how to decide with a short pilot run rather than a guess.

If you are still deciding between consumer and datacenter cards in general, start with [consumer vs datacenter GPUs in the cloud](https://cloudj.net/blog/consumer-vs-datacenter-gpu-cloud).

## The four variants on the list

Cloudj lists two A100 and two H100 models. The figures below are the ones shown on each model page:

| Model | Architecture | VRAM | FP32 (model page) | Memory bandwidth (model page) | Power |
|---|---|---|---|---|---|
| [A100 PCIe](https://cloudj.net/gpu-cloud/a100-pcie) | Ampere | 40 GB | 15.6 TFLOPS | 1313 GB/s | 250 W |
| [A100 SXM4](https://cloudj.net/gpu-cloud/a100-sxm4) | Ampere | 40 GB | 15.6 TFLOPS | 1313 GB/s | 400 W |
| [H100 PCIe](https://cloudj.net/gpu-cloud/h100-pcie) | Hopper | 80 GB | 41 TFLOPS | 1760 GB/s | 350 W |
| [H100 SXM](https://cloudj.net/gpu-cloud/h100-sxm) | Hopper | 80 GB | 53.5 TFLOPS | 2869 GB/s | 700 W |

Two cautions when reading this table. First, FP32 is not what most modern training uses; mixed-precision training runs on tensor cores in BF16 or FP16 (and FP8 on Hopper), where the gap between generations is different. Second, the SXM form factor usually runs at a higher power budget, which helps sustained performance. Treat the table as a guide to memory and generation, not as a benchmark.

## Memory: the first filter

The A100s on the list are the 40 GB version; both H100s have 80 GB. That difference often decides the matter before speed does:

- **Fits in 40 GB with room to spare** (LoRA on mid-size models, fine-tuning 7B models with care, many vision models): both cards are options; compare cost per run.
- **Needs 40 to 80 GB** (full fine-tuning of 7B to 13B models, larger batches, long sequences): the H100 runs it on one card; the A100 needs memory-saving tricks or several GPUs.
- **Needs more than 80 GB on one card**: neither. Look at the 141 GB class H200 NVL in the [H200 cloud rental](https://cloudj.net/blog/h200-cloud-rental) article, or split across a multi-GPU machine.

Multi-GPU machines (2×, 4×, 8× in one host) appear only when a host lists them, for either model.

## Speed and software

Hopper adds features that newer training stacks use: FP8 support, a faster tensor core generation and higher memory bandwidth. Whether you benefit depends on your software. Recent PyTorch versions, attention kernels and training frameworks generally take advantage of Hopper; older pinned environments may not. If your stack is old and frozen, the A100's advantage of being thoroughly tested may matter more than the H100's peak speed.

Practical check, right after connecting:

```bash
nvidia-smi
python -c "import torch; print(torch.__version__, torch.cuda.get_device_name(0), torch.cuda.is_bf16_supported())"
```

## Cost per finished run, not per hour

The H100's hourly price is normally higher. But you pay for the minutes you use (billing is per minute), so the number that matters is:

```text
cost of a run ≈ running price per hour × hours to finish
              + bandwidth (GB down and up) × price per GB
```

If the H100 finishes your job in much less time, it can cost less per run despite a higher hourly rate. If your job is limited by data loading or small batches, the H100 may sit partly idle and the A100 can be the better buy. Do not guess; run a pilot:

1. Deploy one A100 and, separately, one H100 (both with the same image and code).
2. Run the same short slice of training, for example a few hundred steps, on each.
3. Measure the time per step after warm-up.
4. Multiply by the steps in your full run and by each machine's live running price.

The console shows the exact price before you deploy, and it is locked for that instance. Live "from" prices for all four variants are on their model pages and on [GPU cloud](https://cloudj.net/gpu-cloud).

## Location and availability

GPU location on Cloudj is a country filter over a live list of host machines. For training, location matters mainly for data rules and transfer time. Start broad: search without a country filter, or with the regions your rules allow. The [H100 cloud in the USA](https://cloudj.net/blog/h100-cloud-usa) and [A100 cloud in Europe](https://cloudj.net/blog/a100-cloud-europe) articles cover regional searches. Stock changes during the day; if one model is not listed, the other may be.

Before a long run, check each machine's "available until" date against your estimate. When it passes, the instance and disk are removed. Copy checkpoints off the instance regularly; machines are independent hosts with no hardware SLA.

## Account level

A100 and H100 are datacenter GPUs and need a higher account level than new accounts start with. If the console says "This GPU class is not available at your account level", contact Cloudj support from the console.

## Quick decision guide

- Needs more than 40 GB on one card: **H100**.
- Fits in 40 GB, modern stack, long run: **pilot both**, pick the lower cost per run.
- Fits in 40 GB, old frozen stack: **A100** is often the safer choice.
- Only one is listed when you need to start: use it, and keep the comparison for next time.

## FAQ

### Is the H100 always faster than the A100?

Generally, yes, especially with mixed precision on a recent stack. How much faster depends on your workload, so measure with a pilot run.

### Why are the A100s listed with 40 GB?

The A100 PCIe and SXM4 on the list are the 40 GB version. The H100s have 80 GB.

### SXM or PCIe?

SXM variants run at a higher power budget and, on the H100, list much higher memory bandwidth. Compare cost per run if both are listed.

### Can a new account rent either card?

Usually not straight away. Both need a higher account level.

## Run your pilot

Open the [A100 deploy page](https://app.cloudj.net/deploy/gpu?model=a100-pcie) and the [H100 deploy page](https://app.cloudj.net/deploy/gpu?model=h100-sxm), run the same short slice on each and compare cost per run. All live prices are on [GPU cloud](https://cloudj.net/gpu-cloud).