NVLink or PCIe: which one your job needs.
The link between the cards matters enormously for one kind of workload and not at all for three others. Knowing which you are running is worth several hundred dollars a month.
The short answer
- Running one model per card
- The link is irrelevant. Buy PCIe and spend the difference on more cards.
- Splitting one model across cards
- Tensor parallelism synchronises at every layer. NVLink is worth paying for.
- Pipeline parallelism
- One transfer per micro-batch, not per layer. PCIe is usually fine.
- Multi-node training
- We do not offer it — NVLink joins cards inside a node, nothing joins nodes.
The short version
If a model fits on one card, stop reading — none of this applies to you, and the cheapest card that holds your model is the right purchase. This guide is about what happens when it does not fit and you have to split it.
What each link is
| Link | Where you get it | Per card, each way | Topology |
|---|---|---|---|
| NVLink 4 via NVSwitch | SXM cards on an HGX board — 4- and 8-GPU nodes | 900 GB/s | All-to-all. Every card talks to every other at full rate, simultaneously. |
| NVLink 5 via NVSwitch | B200 SXM6 nodes | 1,800 GB/s | All-to-all, twice the previous generation. |
| PCIe Gen5 ×16 | All PCIe cards | 64 GB/s | Through the CPU root complex. Cards on different sockets are further apart than cards on the same one. |
| PCIe Gen4 ×16 | Some 1- and 2-GPU chassis | 32 GB/s | As above, at half the rate. |
The headline ratio is roughly 14× between NVLink 4 and PCIe Gen5. That number is real but it is not the number that matters — what matters is how often your job crosses the link.
When it decides your speed
One case, and it is the case people most often find themselves in without planning for it.
Tensor parallelism
Each layer is cut across the cards, so every card holds a slice of every weight matrix. After each layer, the partial results must be summed across all cards — an all-reduce.
- On a 70B model, that is 80 synchronisations per token
- Every one of them waits for the slowest card
- This is where 14× on the link becomes real seconds
Large-batch training
Gradient all-reduce at the end of every step, sized by the parameter count rather than the batch. A 70B model in BF16 moves 140 GB across the link per step.
- Over PCIe Gen5, roughly two seconds of pure transfer
- Over NVLink, closer to a tenth of that
- Multiplied by every step of a multi-day run
When it changes nothing
Three common workloads where paying for NVLink buys you nothing measurable. If you are in one of these, PCIe cards give you more VRAM per dollar.
One model per card. Four independent copies of a 24B model on four cards never touch the link at all. Four times the throughput, no synchronisation, no failure mode.
Pipeline parallelism. The model is split by layer groups, so a card hands off to the next once per micro-batch instead of once per layer. Transfers are rare enough that PCIe keeps up.
Diffusion and rendering. Image and video generation, Blender, Octane: each card works on its own frame or its own image. There is nothing to all-reduce.
--tensor-parallel-size 4 because the node has four cards, is slower than the same model on one of them. The synchronisation cost is real and the benefit is zero. Shard only when the model does not fit.
The same job, both ways
Two nodes we rent, both with four 80 GB cards — the same total memory, the same class of card, differing only in the link between them. Prices are ours, per month.
| 4 × NVIDIA A100 PCIe | 4 × NVIDIA A100 SXM4 | |
|---|---|---|
| Link between cards | PCIe 5.0 ×16 | NVLink 3 · 600 GB/s |
| Total VRAM | 320 GB | 320 GB |
| Memory bandwidth per card | 1,935 GB/s | 2,039 GB/s |
| Llama 3.3 70B at BF16, sharded | ~18 tok/s | ~19 tok/s |
| Price | $4,157/mo | $4,395/mo |
Our throughput model is bounded by memory bandwidth and is deliberately conservative — it does not model the all-reduce directly, so the real gap on a tensor-parallel job is wider than the table suggests, not narrower. Treat these as a floor for both machines and the ordering as the point.
Measuring it yourself
Do this on the first day. It takes two minutes and it tells you whether the machine you were sold has the topology you paid for.
# NV# means NVLink. PIX, PHB or SYS mean the traffic goes over PCIe.
$ nvidia-smi topo -m
# NVLink state and per-link throughput counters
$ nvidia-smi nvlink --status
# The honest test: an actual all-reduce across every card in the box.
# Compare busbw to the link's rated figure, not to the headline number.
$ docker run --rm --gpus all --ipc=host --shm-size=8g \
nvcr.io/nvidia/pytorch:25.02-py3 \
all_reduce_perf -b 8 -e 4G -f 2 -g 4If topo -m shows SYS between two cards on a node you bought for tensor parallelism, those two cards are talking through the CPU and across sockets — the worst case on the machine. On our SXM nodes every pair reports NV18; anything else is a fault, and it is one we want to hear about on day one.
What to buy
- Model fits on one card? Buy one card. Nothing on this page applies, and the interconnect premium is money set on fire.
- Several independent models or many independent jobs? Buy PCIe cards, as many as you need. You get more VRAM per dollar and you never cross the link.
- One model, too large for a single card, served at low concurrency? PCIe works. Expect the sharding to cost you rather than gain you, and size the node for memory rather than speed.
- One model, too large for a single card, serving real traffic or being trained? This is the case NVLink exists for. Buy SXM.
- A single job needing more than eight cards? That needs a fabric between nodes, which we do not sell. We would rather tell you here than after the invoice — see what we do not offer.
The configurator tells you which of these you are in: pick a model and it reports, for every node in the catalogue, whether it fits on a single card, has to be split, or does not fit at all. The word "split" is the moment this guide starts to matter.
See which nodes hold your model on a single card.
The configurator answers that for all 41 configurations, and shows you the price of each.
Other guides
- Sizing · 9 min
Choosing a card for image and video models
What FLUX, SDXL, SD 3.5 and Wan 2.1 need in VRAM, which card in our catalogue holds each, and why the cheapest one that fits is rarely the one to rent.
Read the guide - Sizing · 10 min
Choosing a quantisation format
What AWQ, GPTQ, GGUF and FP8 each cost in memory, speed and quality — and why the format with the smallest weights rarely gives the smallest model.
Read the guide - Sizing · 10 min
Running a mixture-of-experts model
Total parameters decide the machine, active parameters decide the speed. What DeepSeek V3 and Qwen 3 235B need in VRAM, and which node to rent for them.
Read the guide