H100 NVL cyangwa H100 SXM: iyihe yihuta cyane?

Kuri LLM decoding, H100 NVL ifite ububiko bunini (94 GB ugereranyije na 80 GB) na bandwidth yisumbuye (3381 GB/s ugereranyije na 2869 GB/s). H100 SXM yo ifite FP32 yo hejuru igera kuri 53.5 TFLOPS n'umuriro ugera kuri 700 W, bikaba biyigira iy'ingenzi mu mahugurwa y'ubushakashatsi akenera ingufu nyinshi za compute.

Ese nshobora gukoresha 70B model muri 16-bit kuri H100 NVL?

Oya. Uburemere bw'ibipimo bya 70B muri 16-bit bukenera hafi 140 GB. Kuri 94 GB, koresha 70B yahinduwe mu buryo buciriritse bwa 4 cyangwa 8 bits; ku bashaka 70B ya 16-bit, H200 NVL ifite 140 GB niyo ibakwiriye neza.

Ese Hopper ikenera software zihariye?

Hopper ifite ubushobozi bwa compute capability 9.0 kandi yemerera kubakira kuri verisiyo z'iki gihe za CUDA, PyTorch, vLLM na SGLang. Templates za Cloudj zerekana verisiyo nkeya ya CUDA ikenewe, kandi GPU zihuye nazo nizo zonyine ziboneka kuri buri template.

Kwishyura H100 NVL bikora bite?

Ishyura ku isaha mu gihe seriveri ikora ukoresheje konti yawe ya Cloudj, cyangwa uhitemo kwishyura ku kwezi cyangwa igihe kirekire. Console irakwereka igiciro mbere yo kwemeza gahunda, kandi seriveri y'isaha yahagaze yishyurirwa gusa umwanya wa disk yakoresheje.

GPU Cloud · NVIDIA · Hopper

Kodesha GPU ya NVIDIA H100 NVL ifite 94 GB VRAM

Kodesha seriveri za cloud GPU za H100 NVL ku isaha kuri Cloudj: 94 GB VRAM, NVIDIA Hopper na memory bandwidth igera kuri 3381 GB/s yo gukoresha mu kwemeza no gusohora ibisubizo by'amoko manini (large-model inference). Yagenewe amakipe akora kuri LLM akeneye ububiko bwisumbuye kurenza ubwa H100 ya 80 GB no kwihutisha decoding cyane, itanga 48.3 TFLOPS FP32 ku muriro fatizo wa 400 W. Tangiza template ya vLLM cyangwa SGLang, hitamo imiterere ya H100 NVL ihari maze uhuze ukoresheje SSH.

Kodesha H100 NVL GPU zose

Ibiranga H100 NVL

VRAM kuri buri GPU

94 GB

Ubwubatsi

NVIDIA Hopper

Ubushobozi bwo kubara

Imibare ya FP32

≈ 48.3 TFLOPS

Umuvuduko w'ububiko (Bandwidth)

≈ 3381 GB/s

Ingufu ntarengwa

400 W

Imibare yatanzwe n'uwatanze seriveri ku mashini ziriho ubu; imikorere nyakuri iratandukana bitewe na mashini.

Imiterere n'ibiciro bya H100 NVL bihari ako kanya

Ibihari n'ibiciro bihora bivugururwa. Igiciro cya nyuma cyemezwa mbere yo gutumiza.

Kodesha ubu

Ibyo ushobora gukorera kuri H100 NVL cloud GPU

Gutanga serivisi za LLM ku muvuduko wo hejuru

Bandwidth ya 3381 GB/s yongera umuvuduko wo gukora ama tokens, kandi 94 GB ifasha gukoresha quantized 70B model hamwe na KV cache nini kuri GPU imwe gusa.

Imishinga ya RAG n'inyandiko ndende (long-context)

Izi GB 14 z'inyongera kurenza ikarita ya 80 GB zikoreshwa mu kongera uburebure bw'inyandiko n'ubusabe bukorwa rimwe (concurrent requests).

Gutunganya amoko ya 30B–70B (fine-tuning)

Koresha LoRA ku moko yo mu cyiciro cya 30B muri BF16 cyangwa QLoRA ku moko yo mu cyiciro cya 70B ukoresheje Unsloth kuri GPU imwe ya Hopper.

Gusuzuma amoko ku rwego rwagutse

Kora ibizamini bya benchmark n'isuzuma rya A/B ku manota atandukanye byihuse, hanyuma uhagarike cyangwa usibe seriveri mu gihe amanota amaze kuboneka.

Ese H100 NVL niyo GPU ikwiriye imirimo yawe?

Hitamo H100 NVL mu gihe imirimo yawe ya AI cyangwa iya compute ikenera ububiko bunini kurenza ubukoreshwa mu makarita asanzwe ya datacenter, bugera kuri 94 GB za VRAM. GPU ntoya ya workstation cyangwa ikoreshwa n'abakiriya basanzwe ni yo yaba ihagije ku mirimo yoroshye yo gukora igenzura no gutangira kurema, mu gihe imiterere ya multi-GPU cyangwa urwego rw'ububiko bunini ari ngombwa niba amoko yawe arenga 94 GB.

H100 NVL ugereranyije n'izindi cloud GPU zisa nayo

GPU	VRAM kuri buri GPU	Imibare ya FP32	Ingufu ntarengwa
NVIDIA H100 NVL	94 GB	≈ 48.3 TFLOPS	400 W
NVIDIA H100 SXM	80 GB	≈ 53.5 TFLOPS	700 W	Ibisobanuro →
NVIDIA H100 PCIE	80 GB	≈ 41 TFLOPS	350 W	Ibisobanuro →
NVIDIA A800 PCIE	80 GB	≈ 15.6 TFLOPS	300 W	Ibisobanuro →
NVIDIA CMP 170HX	64 GB	≈ 12.1 TFLOPS	250 W	Ibisobanuro →

H100 NVL cloud GPU: Ibibazo bikunze kubazwa

Kodesha H100 NVL cloud GPU ku isaha

Hitamo template, hitamo imiterere ihari maze wihuze ukoresheje SSH cyangwa terminal ya browser. Kwishyuza bihagarara iyo usibye iyo GPU.

Kodesha H100 NVL GPU zose