Wane irin sauri H200 NVL ta fi H100 NVL da shi?

A bayanan da aka lissafa, duka biyun suna da 48.3 TFLOPS FP32, amma H200 NVL tana da 140 GB maimakon 94 GB da kuma 3884 GB/s maimakon 3381 GB/s. Ayyukan LLM da suka dogara da girman ƙwaƙwalwa sun fi cin moriyar wannan; ayyukan da suka dogara da lissafin kwamfuta kaɗai ba sa ganin wani babban bambanci.

Shin model mai girman 70B yana shiga a yanayin 16-bit a kan H200 NVL?

Kusan a cike: nauyin 70B a 16-bit yana ɗaukar kusan 140 GB, wanda ba ya barin kusan komai don KV cache. A aikace, yana da kyau ka gudanar da 70B a 8-bit ko tsarin FP8 quantization sannan ka yi amfani da sauran ƙwaƙwalwar don bayanan mahallin aiki (context).

Su waye ya kamata su yi hayar H200 NVL guda maimakon ƙananan GPU guda biyu?

Ƙungiyoyin da model ɗinsu zai iya shiga a kan katin 140 GB guda ɗaya suna kauce wa wahalar tsara tensor-parallel, cunkoson sadarwar tsakanin GPU daban-daban, da kuma matsalar binciken kurakuran multi-GPU. Yin amfani da babban GPU guda ɗaya ya fi sauƙi kuma ya fi sauri wajen gudanar da aiki.

Ta yaya zan yi hayar H200 NVL a Cloudj?

Zaɓi template, zaɓi tsarin H200 NVL daga jerin da ke akwai, tabbatar da farashin sannan ka buɗe sabar daga ma'ajiyar kuɗinka (balance). Ka haɗa ta hanyar SSH ko kuma terminal ɗin dake kan burawza.

GPU Cloud · NVIDIA · Hopper

Yi Hayar GPU ta NVIDIA H200 NVL Mai 140 GB VRAM

Yi hayar sabar cloud GPU ta H200 NVL a kan kowane awa a Cloudj: tana da 140 GB VRAM, fasahar NVIDIA Hopper da kuma kusan 3884 GB/s na memory bandwidth, GPU mafi girman ƙwaƙwalwa a jerinmu. An ƙera ta ne domin ƙungiyoyin da ke aiki da manyan model masu girman 70B a matakin high precision ko dogon bayani (very long contexts), tare da 48.3 TFLOPS FP32 da ƙarfin wutar lantarki 600 W. Zaɓi tsarin vLLM ko SGLang, zaɓi tsarin H200 NVL da ke akwai yanzu, sannan ka haɗa ta hanyar SSH.

Yi hayar H200 NVL Duk GPUs

Bayanan fasaha na H200 NVL

VRAM a kowace GPU

140 GB

Ginin fasaha

NVIDIA Hopper

Ikon lissafi (Compute capability)

Lissafin FP32

≈ 48.3 TFLOPS

Gudun ƙwaƙwalwa (Bandwidth)

≈ 3884 GB/s

Ƙayyadaddun wutar lantarki

600 W

Ƙimomin da mai samar da kayan aiki ya bayar don na'urorin da aka jera a yanzu; ainihin aikin ya bambanta dangane da na'ura.

Tsare-tsare da farashin H200 NVL na yanzu

Kayan da ke akwai da farashin kai tsaye, ana sabunta su akai-akai. Ana tabbatar da farashin ƙarshe kafin ka yi oda.

Yi haya yanzu

Abubuwan da za ka iya gudanarwa a kan cloud GPU na H200 NVL

Model masu girman 70B kusan a cikakken precision

Ƙwaƙwalwar 140 GB tana ɗaukar model ɗin 70B a yanayin 8-bit tare da babban KV cache, ko kusa da girman 16-bit, a kan GPU guda ɗaya.

Aiki da dogon rubutu (long-context serving)

Gudun 3884 GB/s yana tabbatar da saurin fassarar bayanai (decoding), yayin da ƙarin ƙwaƙwalwar ke ɗaukar dogayen tambayoyi (prompts) da zama da dama a lokaci guda.

Gyara da horas da model ɗin 70B tare da QLoRA

Gudanar da 70B QLoRA tare da dogayen jeri (sequences) da manyan batches fiye da yadda katin 80 GB zai iya ɗauka.

Gwajin tsarin Mixture-of-experts (MoE)

Model ɗin MoE suna da nauyi sosai; 140 GB yana ba ka damar loda rukunin masana (expert sets) mafi girma don gwaji ba tare da buƙatar sharding ba.

Shin H200 NVL ita ce GPU da ta dace da aikinku?

Zaɓi H200 NVL idan babban abin da kake buƙata shi ne ƙarfin VRAM da saurin memory bandwidth, musamman don ɗaukar manyan model a kan kati guda ɗaya. Idan kana gudanar da ayyukan da ba su da nauyi sosai, ƙirƙirar hotuna na yau da kullum, ko ayyukan gwaji da ba sa buƙatar ƙwaƙwalwar 140 GB, ƙaramar workstation ko GPU na gama-gari sun fi dacewa da aiki.

H200 NVL idan aka kwatanta da sauran cloud GPU

GPU	VRAM a kowace GPU	Lissafin FP32	Ƙayyadaddun wutar lantarki
NVIDIA H200 NVL	140 GB	≈ 48.3 TFLOPS	600 W
NVIDIA H100 NVL	94 GB	≈ 48.3 TFLOPS	400 W	Cikakken bayani →
NVIDIA H100 SXM	80 GB	≈ 53.5 TFLOPS	700 W	Cikakken bayani →
NVIDIA H100 PCIE	80 GB	≈ 41 TFLOPS	350 W	Cikakken bayani →
NVIDIA A800 PCIE	80 GB	≈ 15.6 TFLOPS	300 W	Cikakken bayani →

Cloud GPU na H200 NVL: tambayoyin da aka fi yi

Yi hayar cloud GPU na H200 NVL a kowace sa'a

Zaɓi samfuri, zaɓi tsari na kai tsaye, sannan ka haɗa ta SSH ko terminal na browser. Caji na tsayawa da zarar ka goge GPU ɗin.

Yi hayar H200 NVL Duk GPUs