H100 NVL vs H100 SXM: yikuphi okushesha kakhulu?

Kukhipha imiphumela ye-LLM (decoding), i-H100 NVL ibhalwe ngememori eningi (94 GB vs 80 GB) kanye ne-bandwidth enkulu (3381 GB/s vs 2869 GB/s). I-H100 SXM ine-FP32 ephezulu ku-53.5 TFLOPS kanye nomkhawulo wamandla ongu-700 W, ngakho-ke ifanele ukuqeqeshwa okudinga amandla amakhulu wekhompyutha.

Ngingayisebenzisa imodeli ye-70B nge-16-bit ku-H100 NVL?

Cha. Izisindo ze-70B ye-16-bit zidinga cishe i-140 GB. Ku-94 GB, sebenzisa i-70B eqoqwe yaba ngama-bit angu-4 noma angu-8; kanti ku-16-bit 70B, i-H200 NVL eno-140 GB yiyona efaneleka kakhulu.

Ingabe i-Hopper idinga isofthiwe ekhethekile?

I-Hopper inekhono le-compute lingu-9.0 futhi isebenzisa izinhlobo zakamuva ze-CUDA, PyTorch, vLLM kanye ne-SGLang. Amathempulethi e-Cloudj abala inguqulo yawo encane ye-CUDA, futhi kunikezwa kuphela ama-GPU ahambisanayo kuthempulethi ngayinye.

Kusebenza kanjani ukukhokha kwe-H100 NVL?

Khokha ngehora ngenkathi useva isebenza usebenzisa ibhalansi yakho ye-Cloudj, noma khetha umjikelezo wanyanga zonke noma omude kakhulu. Ikhonsoli iqinisekisa intengo ngaphambi kokuthi u-ode, futhi useva yehora emisiwe ikhokhiswa kuphela idiski yayo.

GPU Cloud · NVIDIA · Hopper

Qasha i-NVIDIA H100 NVL GPU ene-94 GB VRAM

Renta amaseva e-cloud GPU e-H100 NVL ngehora ku-Cloudj: 94 GB VRAM, NVIDIA Hopper kanye ne-bandwidth yememori ecishe ibe ngu-3381 GB/s yokusebenza kwe-large-model inference. Yenzelwe amathimba e-LLM adinga imemori eningi kune-80 GB H100 kanye nokukhipha imiphumela (decoding) okushesha kakhulu, inikeza i-48.3 TFLOPS FP32 emandleni angu-400 W. Qala ithempulethi ye-vLLM noma ye-SGLang, khetha ukucushwa bukhoma kwe-H100 NVL bese uxhuma nge-SSH.

Qasha i-H100 NVL Wonke ama-GPU

Imininingwane ye-H100 NVL

VRAM nge-GPU ngayinye

94 GB

Architecture

NVIDIA Hopper

Compute capability

Ukubala kwe-FP32

≈ 48.3 TFLOPS

I-bandwidth yememori

≈ 3381 GB/s

Umkhawulo wamandla

400 W

Amanani abikwa umhlinzeki wengqalasizinda yemishini ebalwe ohlwini njengamanje; ukusebenza kwangempela kuyehluka ngomshini ngamunye.

Izilungiselelo zamanje nezintengo ze-H100 NVL

Isitoko esibukhoma nezintengo, kubuyekezwa ngokuqhubekayo. Intengo yokugcina iqinisekiswa ngaphambi kokuba u-ode.

Qasha manje

Ongakusebenzisa ku-cloud GPU ye-H100 NVL

Ukuhlinzeka nge-LLM ngomthamo ophezulu

I-bandwidth engu-3381 GB/s isheshisa ukukhiqizwa kwamathokheni, kanti i-94 GB igcina imodeli ye-70B eqoqwe (quantized) kanye ne-KV cache enkulu ku-GPU eyodwa.

Amapayipilayini e-Long-context kanye ne-RAG

I-14 GB eyengeziwe ngaphezu kwekhadi le-80 GB ingena ngqo kubude bokuqukethwe kanye nezicelo ezenzeka ngesikhathi esisodwa.

Uku-fine-tune amamodeli we-30B–70B

Sebenzisa i-LoRA kumamodeli esigaba se-30B nge-BF16 noma i-QLoRA kumamodeli esigaba se-70B usebenzisa i-Unsloth ku-GPU eyodwa ye-Hopper.

Ukuhlola amamodeli ngobuningi obukhulu

Sebenzisa amasudi wokulinganisa kanye nokuhlola kwe-A/B kuma-checkpoint ngokushesha, bese ucisha useva uma imiphumela isilungile.

Ingabe i-H100 NVL iyi-GPU efanele umsebenzi wakho?

Khetha i-H100 NVL uma imisebenzi yakho ye-AI noma ye-compute idinga imemori eningi kakhulu kunaleyo ehlinzekwa amakhadi ajwayelekile ezikhungo zedatha, kufika ku-94 GB ye-VRAM. I-workstation encane noma i-GPU yabathengi iyisinqumo esingcono kakhulu sokusebenza okulula nokuthuthukisa, kanti ukuhlelwa kwama-GPU amaningi noma izinga lememori elikhudlwana kuyadingeka uma amamodeli akho edlula i-94 GB.

H100 NVL uma iqhathaniswa nama-cloud GPU afanayo

GPU	VRAM nge-GPU ngayinye	Ukubala kwe-FP32	Umkhawulo wamandla
NVIDIA H100 NVL	94 GB	≈ 48.3 TFLOPS	400 W
NVIDIA H100 SXM	80 GB	≈ 53.5 TFLOPS	700 W	Imininingwane →
NVIDIA H100 PCIE	80 GB	≈ 41 TFLOPS	350 W	Imininingwane →
NVIDIA A800 PCIE	80 GB	≈ 15.6 TFLOPS	300 W	Imininingwane →
NVIDIA CMP 170HX	64 GB	≈ 12.1 TFLOPS	250 W	Imininingwane →

I-cloud GPU ye-H100 NVL: imibuzo evame ukubuzwa

Qasha i-cloud GPU ye-H100 NVL ngehora

Khetha ithempulethi, khetha izilungiselelo zamanje bese uxhuma nge-SSH noma itheminali yesiphequluli. Ukukhokhisa kuyama uma ususa i-GPU.

Qasha i-H100 NVL Wonke ama-GPU