Kuki wahitamo H100 SXM aho guhitamo H100 PCIE?

H100 SXM igaragazwa kuri 53.5 TFLOPS FP32 na 2869 GB/s ugereranyije na 41 TFLOPS na 1760 GB/s ku ikarita ya PCIE, ikagira n'imbaraga za 700 W aho kuba 350 W. Ku kintu cyose gikorerwaho, ubusanzwe irangiza vuba cyane.

Ese H100 SXM cyangwa H200 NVL ni iyihe iruta indi kuri za LLM?

H200 NVL ifite 140 GB na 3884 GB/s, bityo ikaba ari yo itsinda kuri za modeli nini cyane n'inyandiko ndende (long contexts). H100 SXM yo ifite imbaraga nyinshi zo kubara za FP32 (53.5 ugereranyije na 48.3 TFLOPS) kandi ibereye imirimo yo gutoza ikwirwa muri 80 GB.

Ese nshobora gukora fine-tune kuri modeli ya 70B nkimika H100 SXM imwe rukumbi?

Ukoresheje QLoRA, yego: uburemere bwa 4-bit 70B butwara hafi 35–40 GB, bigasiga umwanya uhagije wa adapters, activations hamwe na leta ya optimizer. Gukora full 16-bit fine-tuning kuri 70B bisaba za GPU nyinshi.

Nishyura iki igihe seriveri ya H100 SXM ihagaritswe kuri Cloudj?

Ku kwishyuza ku isaha, seriveri yahagaritswe yishyurirwa gusa umwanya wa disiki yayo. Siba seriveri umaze gukoporora za checkpoint zawe maze uhagarike kwishyura burundu.

GPU Cloud · NVIDIA · Hopper

Kodesha GPU ya NVIDIA H100 SXM ifite 80 GB VRAM

Kodesha seriveri za cloud GPU za H100 SXM ku isaha kuri Cloudj: 80 GB VRAM, NVIDIA Hopper, hafi 53.5 TFLOPS FP32 na 2869 GB/s bya bandwidth ya memori. Ifite imbaraga zigera kuri 700 W, iyi ni H100 ibereye amatsinda atoza, anoza (fine-tune) akanatanga serivisi za modeli nini, yifuza module ya Hopper yihuta kurusha izindi muri uru ruhererekane mu gukora imibare. Tangiza template ya PyTorch, vLLM cyangwa Unsloth maze wihuze ukoresheje SSH mu ntambwe nkeya.

Kodesha H100 SXM GPU zose

Ibiranga H100 SXM

VRAM kuri buri GPU

80 GB

Ubwubatsi

NVIDIA Hopper

Ubushobozi bwo kubara

Imibare ya FP32

≈ 53.5 TFLOPS

Umuvuduko w'ububiko (Bandwidth)

≈ 2869 GB/s

Ingufu ntarengwa

700 W

Imibare yatanzwe n'uwatanze seriveri ku mashini ziriho ubu; imikorere nyakuri iratandukana bitewe na mashini.

Imiterere n'ibiciro bya H100 SXM bihari ako kanya

Ibihari n'ibiciro bihora bivugururwa. Igiciro cya nyuma cyemezwa mbere yo gutumiza.

Kodesha ubu

Ibyo ushobora gukorera kuri H100 SXM cloud GPU

Gutoza bisaba imbaraga zikomeye zo kubara

53.5 TFLOPS FP32 hamwe n'ingufu zigera kuri 700 W biha H100 SXM imbaraga nyinshi zo kubara kurusha ikarita iyo ari yo yose ya Hopper iri ku rutonde hano.

Inference yihuse ya za LLM

Bandwidth ya 2869 GB/s yihutisha itangwa rya za token kuri modeli za 13B–34B muri 16-bit cyangwa modeli zikoresha quantized 70B.

Gutunganya byuzuye (full fine-tuning) kwa modeli za 7B–13B

80 GB itanga umwanya uhagije wo gukora full-parameter fine-tuning kuri za LLM ntoya, bitari adapters gusa, ukoresheje igenamiterere rikwiye rya optimizer.

Gutoza za modeli z'amashusho na videwo (diffusion na video models)

Toza cyangwa unoze za modeli z'amashusho na videwo aho haba memori n'ubushobozi bwo kubara bigena igihe igikorwa gitwara.

Ese H100 SXM niyo GPU ikwiriye imirimo yawe?

Hitamo H100 SXM igihe imirimo yawe yo gutoza cyangwa inference isaba 80 GB ya memori ya datacenter ifite bandwidth yo hejuru n'ibiranga imyubakire ya Hopper. GPU ntoya ya workstation cyangwa consumer ni yo ihwitse mu bikorwa byoroheje by'iterambere cyangwa kuri modeli nto, mu gihe GPU ya datacenter nshya cyangwa ifite ubushobozi buri hejuru ari yo nziza niba seriveri imwe idashobora kwakira modeli yawe yose muri 80 GB ya VRAM.

H100 SXM ugereranyije n'izindi cloud GPU zisa nayo

GPU	VRAM kuri buri GPU	Imibare ya FP32	Ingufu ntarengwa
NVIDIA H100 SXM	80 GB	≈ 53.5 TFLOPS	700 W
NVIDIA H100 PCIE	80 GB	≈ 41 TFLOPS	350 W	Ibisobanuro →
NVIDIA A800 PCIE	80 GB	≈ 15.6 TFLOPS	300 W	Ibisobanuro →
NVIDIA H100 NVL	94 GB	≈ 48.3 TFLOPS	400 W	Ibisobanuro →
NVIDIA CMP 170HX	64 GB	≈ 12.1 TFLOPS	250 W	Ibisobanuro →

H100 SXM cloud GPU: Ibibazo bikunze kubazwa

Kodesha H100 SXM cloud GPU ku isaha

Hitamo template, hitamo imiterere ihari maze wihuze ukoresheje SSH cyangwa terminal ya browser. Kwishyuza bihagarara iyo usibye iyo GPU.

Kodesha H100 SXM GPU zose