Ishesha kangakanani i-H200 NVL kune-H100 NVL?

Kuma-specs abaliwe zombili zine-48.3 TFLOPS FP32, kodwa i-H200 NVL ine-140 GB esikhundleni se-94 GB kanye ne-3884 GB/s esikhundleni se-3381 GB/s. Imisebenzi ye-LLM eboshwe yinkumbulo (memory-bound) yiyo ehlomula kakhulu; imisebenzi ehlanzekile yokubala ayiboni umehluko omkhulu.

Ingabe imodeli ye-70B iyangena ku-16-bit kwi-H200 NVL?

Kancane kakhulu: Izisindo ze-70B ku-16-bit zithatha cishe i-140 GB, okushiya cishe kungabikho lutho lwe-KV cache. Ekusebenzeni kwangempela, sebenzisa i-70B ku-8-bit noma i-quantization yesitayela se-FP8 bese usebenzisa inkumbulo esele kwi-context.

Ubani okufanele aqashe i-H200 NVL esikhundleni sama-GPU amabili amancane?

Amaqembu amamodeli awo angena ekhadini elilodwa le-140 GB agwema ukusetha i-tensor-parallel, ukugcwala kwe-traffic phakathi kwama-GPU, kanye nokulungisa amaphutha kuma-GPU amaningi. I-GPU eyodwa enkulu ivame ukuba lula futhi isheshe ukusebenza ngokuphindaphindiwe.

Ngiqasha kanjani i-H200 NVL ku-Cloudj?

Khetha ithempulethi, khetha ukulungiselelwa kwe-H200 NVL kusuka ohlwini lwezimpahla ezikhona bukhoma, qinisekisa intengo bese udala iseva kusuka kubhalansi yakho. Xhuma nge-SSH noma nge-terminal yesiphequluli.

GPU Cloud · NVIDIA · Hopper

Qasha i-NVIDIA H200 NVL GPU ene-140 GB VRAM

Qasha amaseva e-cloud GPU e-H200 NVL ngehora ku-Cloudj: i-140 GB VRAM, NVIDIA Hopper kanye ne-memory bandwidth efinyelela ku-3884 GB/s, okuwukaphezu kwazo zonke i-GPU enenkumbulo enkulu ohlwini lwethu. Yakhelwe amaqembu asebenzisa amamodeli ekilasi le-70B nge-precision ephezulu kakhulu noma ama-context amade kakhulu, enama-48.3 TFLOPS FP32 kanye nomkhawulo wamandla we-600 W. Khetha ithempulethi ye-vLLM noma ye-SGLang, khetha ukulungiselelwa bukhoma kwe-H200 NVL bese uxhuma nge-SSH.

Qasha i-H200 NVL Wonke ama-GPU

Imininingwane ye-H200 NVL

VRAM nge-GPU ngayinye

140 GB

Architecture

NVIDIA Hopper

Compute capability

Ukubala kwe-FP32

≈ 48.3 TFLOPS

I-bandwidth yememori

≈ 3884 GB/s

Umkhawulo wamandla

600 W

Amanani abikwa umhlinzeki wengqalasizinda yemishini ebalwe ohlwini njengamanje; ukusebenza kwangempela kuyehluka ngomshini ngamunye.

Izilungiselelo zamanje nezintengo ze-H200 NVL

Isitoko esibukhoma nezintengo, kubuyekezwa ngokuqhubekayo. Intengo yokugcina iqinisekiswa ngaphambi kokuba u-ode.

Qasha manje

Ongakusebenzisa ku-cloud GPU ye-H200 NVL

Amamodeli ekilasi le-70B acishe agcwele i-precision

I-140 GB igcina imodeli ye-70B ku-8-bit ene-KV cache enkulu, noma esondelene kakhulu nesilinganiso se-16-bit, kwi-GPU eyodwa nje.

Ukuhlinzeka ngama-context amade

I-3884 GB/s igcina i-decoding ishesha kanti inkumbulo eyengeziwe igcina ama-prompt amade nezikhathi eziningi zokusebenza ezifanayo.

Ukulungisa kahle (fine-tuning) amamodeli e-70B nge-QLoRA

Sebenzisa i-70B QLoRA enama-sequence amade namaqoqo amakhulu (batches) kunezinto ezivunyelwa ikhadi le-80 GB.

Ukuhlola kwe-Mixture-of-experts

Amamodeli e-MoE anezinsimbi eziphelele ezinkulu (weights); i-140 GB ikuvumela ukuba ulayishe amaqoqo amakhulu ochwepheshe ukuze uvivinye ngaphandle kokuhlukanisa (sharding).

Ingabe i-H200 NVL iyi-GPU efanele umsebenzi wakho?

Khetha i-H200 NVL uma inkinga yakho enkulu yokuminyana ingumthamo we-VRAM ne-memory bandwidth, ikakhulukazi ekusingatheni amamodeli amakhulu ekhadini elilodwa. Uma usebenzisa i-inference elula, ukukhiqiza izithombe okujwayelekile, noma imisebenzi yokuthuthukisa engadingi i-140 GB yenkumbulo, i-workstation encane noma i-GPU yabathengi iyisinqumo esingcono kakhulu nesisebenzayo.

H200 NVL uma iqhathaniswa nama-cloud GPU afanayo

GPU	VRAM nge-GPU ngayinye	Ukubala kwe-FP32	Umkhawulo wamandla
NVIDIA H200 NVL	140 GB	≈ 48.3 TFLOPS	600 W
NVIDIA H100 NVL	94 GB	≈ 48.3 TFLOPS	400 W	Imininingwane →
NVIDIA H100 SXM	80 GB	≈ 53.5 TFLOPS	700 W	Imininingwane →
NVIDIA H100 PCIE	80 GB	≈ 41 TFLOPS	350 W	Imininingwane →
NVIDIA A800 PCIE	80 GB	≈ 15.6 TFLOPS	300 W	Imininingwane →

I-cloud GPU ye-H200 NVL: imibuzo evame ukubuzwa

Qasha i-cloud GPU ye-H200 NVL ngehora

Khetha ithempulethi, khetha izilungiselelo zamanje bese uxhuma nge-SSH noma itheminali yesiphequluli. Ukukhokhisa kuyama uma ususa i-GPU.

Qasha i-H200 NVL Wonke ama-GPU