H100 NVL da H100 SXM: wanne ya fi sauri?

Don fassarar decoding ta LLM, an bayyana cewa H100 NVL tana da ƙarin ƙwaƙwalwa (94 GB akan 80 GB) da ƙarin bandwidth (3381 GB/s akan 2869 GB/s). H100 SXM tana da mafi girman FP32 a 53.5 TFLOPS da iyakar 700 W, don haka ta fi dacewa da horarwa mai nauyin lissafi.

Zan iya gudanar da model mai 70B a yanayin 16-bit a kan H100 NVL?

A'a. Nauyin 70B na 16-bit yana buƙatar kusan 140 GB. A kan 94 GB, gudanar da 70B da aka yi wa quantized zuwa 4 ko 8 bits; don 16-bit 70B, H200 NVL mai 140 GB ita ce ta fi dacewa.

Shin Hopper yana buƙatar software na musamman?

Hopper yana da compute capability 9.0 kuma yana gudanar da sabbin manhajojin CUDA, PyTorch, vLLM da SGLang. Templates na Cloudj suna lissafa mafi ƙarancin nau'in CUDA da ake buƙata, kuma GPUs masu dacewa kawai ake bayarwa ga kowane template.

Ta yaya tsarin biyan kuɗi yake aiki don H100 NVL?

Biya a kowace sa'a yayin da sabar ke aiki daga asusunka na Cloudj, ko zaɓi tsarin wata-wata ko lokaci mafi tsawo. Console ɗin zai tabbatar da farashin kafin ka yi oda, kuma sabar ta kowace sa'a da aka tsayar ana cajin ta ne kawai don ajiyar diski ɗinta.

GPU Cloud · NVIDIA · Hopper

Yi Hayar GPU ta NVIDIA H100 NVL Mai 94 GB VRAM

Yi hayar sabobin gajimare na GPU na H100 NVL a kowace sa'a a kan Cloudj: 94 GB VRAM, NVIDIA Hopper da kusan 3381 GB/s na bandwidth na ƙwaƙwalwa don inference na manyan model. An ƙera shi don ƙungiyoyin LLM da ke buƙatar ƙarin ƙwaƙwalwa fiye da H100 mai 80 GB da kuma saurin decoding sosai, yana ba da 48.3 TFLOPS FP32 a iyakar wutar lantarki ta 400 W. Fara template na vLLM ko SGLang, zaɓi tsarin H100 NVL mai aiki, sannan ka haɗa ta SSH.

Yi hayar H100 NVL Duk GPUs

Bayanan fasaha na H100 NVL

VRAM a kowace GPU

94 GB

Ginin fasaha

NVIDIA Hopper

Ikon lissafi (Compute capability)

Lissafin FP32

≈ 48.3 TFLOPS

Gudun ƙwaƙwalwa (Bandwidth)

≈ 3381 GB/s

Ƙayyadaddun wutar lantarki

400 W

Ƙimomin da mai samar da kayan aiki ya bayar don na'urorin da aka jera a yanzu; ainihin aikin ya bambanta dangane da na'ura.

Tsare-tsare da farashin H100 NVL na yanzu

Kayan da ke akwai da farashin kai tsaye, ana sabunta su akai-akai. Ana tabbatar da farashin ƙarshe kafin ka yi oda.

Yi haya yanzu

Abubuwan da za ka iya gudanarwa a kan cloud GPU na H100 NVL

Gudanar da LLM mai saurin aiki sosai (High-throughput)

Bandwidth mai ƙarfin 3381 GB/s yana ƙara saurin samar da token, kuma 94 GB yana riƙe model 70B quantized tare da babban KV cache a kan GPU guda ɗaya.

Tsarukan RAG da dogon bayani (Long-context)

Ƙarin 14 GB a kan katin 80 GB yana tafiya kai tsaye zuwa tsawon mahallin bayani da buƙatun aiki na lokaci ɗaya.

Inganta model masu girman 30B–70B (Fine-tuning)

Gudanar da LoRA a kan model masu girman 30B a BF16 ko QLoRA a kan model masu girman 70B tare da Unsloth a kan GPU guda ɗaya na Hopper.

Gwaji da auna ingancin model a babban sikeli

Gudanar da gwaje-gwajen benchmark da kimanta A/B a cikin checkpoints cikin sauri, sannan ka goge sabar da zarar an kammala samun maki.

Shin H100 NVL ita ce GPU da ta dace da aikinku?

Zaɓi H100 NVL idan ayyukan AI ko na lissafin kwamfuta (compute) suna buƙatar ƙarin ƙwaƙwalwa fiye da yadda katinan cibiyar bayanai na yau da kullun suke bayarwa, har zuwa 94 GB na VRAM. Ƙaramar workstation ko GPU na masu amfani shine mafi kyawun zaɓi don ƙananan ayyukan inference da haɓakawa, yayin da saitin multi-GPU ko matakin ƙwaƙwalwa mafi girma ya zama dole idan model ɗinka ya zarce 94 GB.

H100 NVL idan aka kwatanta da sauran cloud GPU

GPU	VRAM a kowace GPU	Lissafin FP32	Ƙayyadaddun wutar lantarki
NVIDIA H100 NVL	94 GB	≈ 48.3 TFLOPS	400 W
NVIDIA H100 SXM	80 GB	≈ 53.5 TFLOPS	700 W	Cikakken bayani →
NVIDIA H100 PCIE	80 GB	≈ 41 TFLOPS	350 W	Cikakken bayani →
NVIDIA A800 PCIE	80 GB	≈ 15.6 TFLOPS	300 W	Cikakken bayani →
NVIDIA CMP 170HX	64 GB	≈ 12.1 TFLOPS	250 W	Cikakken bayani →

Cloud GPU na H100 NVL: tambayoyin da aka fi yi

Yi hayar cloud GPU na H100 NVL a kowace sa'a

Zaɓi samfuri, zaɓi tsari na kai tsaye, sannan ka haɗa ta SSH ko terminal na browser. Caji na tsayawa da zarar ka goge GPU ɗin.

Yi hayar H100 NVL Duk GPUs