H200 NVL yihuta kurusha H100 NVL ku kihe kigero?

Ku bimenyetso byashyizwe ahagaragara zombi zifite 48.3 TFLOPS FP32, ariko H200 NVL ifite 140 GB aho kuba 94 GB, ndetse na 3884 GB/s aho kuba 3381 GB/s. Imirimo ya LLM ishingiye ku bubiko (memory-bound) niyo yungukiramo cyane; imirimo ishingiye gusa ku mibare n'ibara (compute jobs) ntibona itandukaniro rinini.

Ese moderi ya 70B muri 16-bit ikwira kuri H200 NVL?

Ikwiramo bigoranye: uburemere bwa 70B muri 16-bit butwara hafi 140 GB, ibyo bigatuma nta mwanya usigara wa KV cache. Mu by'ukuri, koresha 70B muri 8-bit cyangwa uburyo bwa quantization ya FP8 maze ukoreshe ububiko busigaye mu kwakira amakuru y'ubutumwa (context).

Ni bande bakwiriye gukodesha H200 NVL aho gukodesha GPU ebyiri ntoya?

Amakipe afite moderi zikwira kuri karita imwe ya 140 GB yirinda gutunganya tensor-parallel, urujya n'uruza hagati ya GPU (cross-GPU traffic) ndetse no gukosora amakosa kuri GPU nyinshi (multi-GPU debugging). GPU imwe nini kenshi iroroha kandi irihuta mu gukora no kuvugurura imirimo.

Nakodesha nte H200 NVL kuri Cloudj?

Hitamo template, hitamo imiterere ya H200 NVL mu bubiko buriho, wemeze igiciro maze uhite ukora seriveri ukoresheje amafaranga ari kuri konti yawe. Wihuze ukoresheje SSH cyangwa terminal yo muri mushakisha (browser).

GPU Cloud · NVIDIA · Hopper

Kodesha GPU ya NVIDIA H200 NVL ifite 140 GB VRAM

Kodesha seriveri za GPU cloud za H200 NVL ku isaha kuri Cloudj: VRAM ya 140 GB, NVIDIA Hopper hamwe na memory bandwidth igera kuri 3884 GB/s, GPU ifite ububiko bunini kurusha izindi zose dufite. Yubakiwe amakipe akoresha moderi zo mu rwego rwa 70B ku bipimo byo hejuru (high precision) cyangwa inyandiko ndende cyane (very long contexts), ifite 48.3 TFLOPS FP32 n'ingufu ntarengwa za 600 W. Hitamo template ya vLLM cyangwa SGLang, hitamo imiterere ya H200 NVL iriho uhite wihuza ukoresheje SSH.

Kodesha H200 NVL GPU zose

Ibiranga H200 NVL

VRAM kuri buri GPU

140 GB

Ubwubatsi

NVIDIA Hopper

Ubushobozi bwo kubara

Imibare ya FP32

≈ 48.3 TFLOPS

Umuvuduko w'ububiko (Bandwidth)

≈ 3884 GB/s

Ingufu ntarengwa

600 W

Imibare yatanzwe n'uwatanze seriveri ku mashini ziriho ubu; imikorere nyakuri iratandukana bitewe na mashini.

Imiterere n'ibiciro bya H200 NVL bihari ako kanya

Ibihari n'ibiciro bihora bivugururwa. Igiciro cya nyuma cyemezwa mbere yo gutumiza.

Kodesha ubu

Ibyo ushobora gukorera kuri H200 NVL cloud GPU

Moderi zo mu rwego rwa 70B hafi y'ubushobozi bwuzuye (full precision)

140 GB yakira moderi ya 70B muri 8-bit ifite KV cache nini, cyangwa ikegera ubunini bwa 16-bit, kuri GPU imwe rukumbi.

Gutanga serivisi z'inyandiko n'ubutumwa burebure (long-context)

3884 GB/s ituma decoding yihuta mu gihe ububiko bwiyongereyeho bwakira ubutumwa burebure (long prompts) n'iseshyo ryinshi ry'icyarimwe (parallel sessions).

Gutunganya neza (fine-tuning) moderi za 70B ukoresheje QLoRA

Koresha QLoRA ya 70B ifite ibice birebire (longer sequences) n'amatsinda manini (bigger batches) kurusha ibyo ikarita ya 80 GB yakwemerera.

Igerageza rya Mixture-of-experts

Moderi za MoE zigira uburemere bwose bunini; 140 GB igufasha gupakuriramo amatsinda manini y'inzobere (expert sets) zo kugerageza nta kugabanyamo (sharding).

Ese H200 NVL niyo GPU ikwiriye imirimo yawe?

Hitamo H200 NVL niba imbogamizi yawe nyamukuru ari ubushobozi bwa VRAM na memory bandwidth, cyane cyane mu kwakira moderi nini kuri karita imwe. Niba ukora imirimo yoroshye ya inference, gukora amashusho asanzwe, cyangwa imirimo y'iterambere idasaba ububiko bwa 140 GB, GPU ntoya ya workstation cyangwa isanzwe ikoreshwa n'abakiriya niyo mahitamo yoroshye kandi akwiye.

H200 NVL ugereranyije n'izindi cloud GPU zisa nayo

GPU	VRAM kuri buri GPU	Imibare ya FP32	Ingufu ntarengwa
NVIDIA H200 NVL	140 GB	≈ 48.3 TFLOPS	600 W
NVIDIA H100 NVL	94 GB	≈ 48.3 TFLOPS	400 W	Ibisobanuro →
NVIDIA H100 SXM	80 GB	≈ 53.5 TFLOPS	700 W	Ibisobanuro →
NVIDIA H100 PCIE	80 GB	≈ 41 TFLOPS	350 W	Ibisobanuro →
NVIDIA A800 PCIE	80 GB	≈ 15.6 TFLOPS	300 W	Ibisobanuro →

H200 NVL cloud GPU: Ibibazo bikunze kubazwa

Kodesha H200 NVL cloud GPU ku isaha

Hitamo template, hitamo imiterere ihari maze wihuze ukoresheje SSH cyangwa terminal ya browser. Kwishyuza bihagarara iyo usibye iyo GPU.

Kodesha H200 NVL GPU zose