Whisper GPU Vietnamese transcription on a rented cloud GPU

Call-centre recordings, podcasts, lectures, court hearings, TikTok and YouTube videos: Vietnamese audio piles up quickly, and transcribing it on a laptop CPU takes days. A Whisper GPU Vietnamese transcription setup on a rented GPU turns that into hours. This guide shows how to choose a Whisper model size for the VRAM you rent, install faster-whisper, run a batch over SSH, and why the GPU's country barely matters for this job except for one thing: how much audio you have to upload.

Why a GPU, and why batch

Whisper-family models run on CPU, but a GPU is many times faster for the larger, more accurate model sizes. Transcription is also a classic batch job: nobody waits on each file in real time, you queue thousands of files and collect the results. That shapes every decision below, from the card to the country. Batch inference in the cheapest region explains the general logic.

Model size vs VRAM

Larger Whisper models are more accurate, especially with Vietnamese tones, regional accents and noisy recordings, but they need more memory and time. Rough, general guidance for faster-whisper:

Check actual memory use with nvidia-smi while a job runs, and adjust batch size or the number of parallel workers. Community fine-tuned Vietnamese Whisper models also exist; check their licence, and convert them to the CTranslate2 format faster-whisper uses.

The RTX 3060 and RTX 4090 are consumer cards available at the starting account level. The L4 is a datacenter card that needs a higher account level. Live "from" prices are on GPU cloud.

Where to rent: upload size is the only real question

For batch transcription, the GPU's location affects one thing: how long it takes to move your audio there. If you have a few gigabytes, any country is fine. If you have terabytes of recordings stored in Vietnam, the upload path matters.

• Large archive in Vietnam: search SG, HK first. You can also try VN in the Country field, but machines in Vietnam are not guaranteed to be listed, and Singapore and Hong Kong are the usual alternatives. The cheapest GPU near Vietnam covers this search.
• Audio already in cloud storage elsewhere, or a small archive: search broadly, or US, and pick on price, reliability and max duration.

Each machine shows a bandwidth price per TB down and up, so for big uploads compare that too.

Shrink audio before upload

Most recordings are larger than they need to be for speech recognition. Converting to mono, 16 kHz, compressed audio on your own machine can cut upload size dramatically:

ffmpeg -i input.mp4 -vn -ac 1 -ar 16000 -c:a libopus -b:a 24k output.opus

Whisper resamples to 16 kHz internally anyway. Test a few files to make sure accuracy holds. The ffmpeg documentation lists more options.

Set up faster-whisper over SSH

Deploy with the Ubuntu 22.04 + CUDA template, a disk large enough for audio and transcripts, and your SSH key. Then connect:

ssh -p PORT root@HOST
nvidia-smi
pip install faster-whisper

If faster-whisper later reports a missing cuBLAS or cuDNN library, install them with pip install nvidia-cublas-cu12 nvidia-cudnn-cu12 as its README describes, or start from the PyTorch template instead.

Upload the compressed audio:

rsync -avP -e "ssh -p PORT" ./audio/ root@HOST:/workspace/audio/

Run the batch

A minimal script that writes one text file per recording:

import pathlib
from faster_whisper import WhisperModel

model = WhisperModel("large-v3", device="cuda", compute_type="float16")
out = pathlib.Path("/workspace/text"); out.mkdir(exist_ok=True)

for f in sorted(pathlib.Path("/workspace/audio").glob("*.opus")):
    target = out / (f.stem + ".txt")
    if target.exists():
        continue
    segments, info = model.transcribe(str(f), language="vi", vad_filter=True)
    target.write_text("\n".join(s.text.strip() for s in segments), encoding="utf-8")
    print("done", f.name)

Setting language="vi" skips language detection, and vad_filter=True skips long silences. Because the script skips files that already have output, you can rerun it after an interruption. Start it inside tmux so it survives a dropped SSH session:

tmux new -s asr
python transcribe.py

For more throughput on a 24 GB card, run two or three copies on different subsets of files and watch memory with nvidia-smi.

Packaging it as an image

If you transcribe regularly, build a public Docker image with faster-whisper and your script, and deploy it with the Custom image template. Custom images need a higher account level, and private registries are not supported. Using a custom Docker image on a cloud GPU explains the rules.

Collect results and clean up

rsync -avP -e "ssh -p PORT" root@HOST:/workspace/text/ ./transcripts/

Then destroy the instance to stop charges. If your archive is too large for one session, note the machine's "available until" date and copy results off regularly.

Beyond transcription

The same machine and workflow can generate Vietnamese speech or run voice pipelines. TTS and voice on a GPU server in Asia covers the reverse direction.

Privacy

Recordings often contain personal data. Choose countries you are comfortable with in the Country filter, delete audio from the machine when finished, and check your own legal obligations before uploading customer calls.

FAQ

Which Whisper model is best for Vietnamese?

large-v3 is the usual starting point for accuracy. Test community fine-tuned Vietnamese models on your own audio too.

Is an RTX 3060 enough?

For large-v3 with one worker it is usually enough. Use a 24 GB card for more parallel workers.

Does the GPU need to be near Vietnam?

Only if you upload a very large archive from Vietnam. Otherwise pick on price and reliability.

Does Cloudj preinstall Whisper?

No. Install faster-whisper yourself over SSH or bring a public Docker image.

Next step

Open the RTX 3060 deploy page, search SG, HK, and run a test batch of a hundred files before you commit your whole archive.