Sizing guide · 7 min read

# NVLink or PCIe: which one your job needs.

The link between the cards matters enormously for one kind of workload and not at all for three others. Knowing which you are running is worth several hundred dollars a month.

The short answer

- **Running one model per card:** The link is irrelevant. Buy PCIe and spend the difference on more cards.
- **Splitting one model across cards:** Tensor parallelism synchronises at **every layer**. NVLink is worth paying for.
- **Pipeline parallelism:** One transfer per micro-batch, not per layer. PCIe is usually fine.
- **Multi-node training:** We do not offer it — NVLink joins cards *inside* a node, nothing joins nodes.

## The short version

If a model fits on one card, stop reading — none of this applies to you, and the cheapest card that holds your model is the right purchase. This guide is about what happens when it does not fit and you have to split it.

## What each link is

**Interconnect types, bandwidth and topology**

| Link | Where you get it | Per card, each way | Topology |
|---|---|---|---|
| NVLink 4 via NVSwitch Hopper generation | SXM cards on an HGX board — 4- and 8-GPU nodesA100 SXM4, H100 SXM5, H200 | 900 GB/s | All-to-all. Every card talks to every other at full rate, simultaneously. |
| NVLink 5 via NVSwitch Blackwell generation | B200 SXM6 nodes4- and 8-GPU | 1,800 GB/s | All-to-all, twice the previous generation. |
| PCIe Gen5 ×16 Current PCIe nodes | All PCIe cardsL4, L40S, RTX 4090/5090, A6000, A100 PCIe, H100 PCIe | 64 GB/s | Through the CPU root complex. Cards on different sockets are further apart than cards on the same one. |
| PCIe Gen4 ×16 Older platforms | Some 1- and 2-GPU chassis | 32 GB/s | As above, at half the rate. |

The headline ratio is roughly **14×** between NVLink 4 and PCIe Gen5. That number is real but it is not the number that matters — what matters is how often your job crosses the link.

## When it decides your speed

One case, and it is the case people most often find themselves in without planning for it.

### Tensor parallelism

Each layer is cut across the cards, so every card holds a slice of every weight matrix. After each layer, the partial results must be summed across all cards — an all-reduce.

- On a 70B model, that is 80 synchronisations per token
- Every one of them waits for the slowest card
- This is where 14× on the link becomes real seconds

### Large-batch training

Gradient all-reduce at the end of every step, sized by the parameter count rather than the batch. A 70B model in BF16 moves 140 GB across the link per step.

- Over PCIe Gen5, roughly two seconds of pure transfer
- Over NVLink, closer to a tenth of that
- Multiplied by every step of a multi-day run

## When it changes nothing

Three common workloads where paying for NVLink buys you nothing measurable. If you are in one of these, PCIe cards give you more VRAM per dollar.

**One model per card.** Four independent copies of a 24B model on four cards never touch the link at all. Four times the throughput, no synchronisation, no failure mode.

**Pipeline parallelism.** The model is split by layer *groups*, so a card hands off to the next once per micro-batch instead of once per layer. Transfers are rare enough that PCIe keeps up.

**Diffusion and rendering.** Image and video generation, Blender, Octane: each card works on its own frame or its own image. There is nothing to all-reduce.

**The most expensive mistake is buying tensor parallelism you did not need.** A model that fits on a single 80 GB card, served with `--tensor-parallel-size 4` because the node has four cards, is *slower* than the same model on one of them. The synchronisation cost is real and the benefit is zero. Shard only when the model does not fit.

## The same job, both ways

Two nodes we rent, both with four 80 GB cards — the same total memory, the same class of card, differing only in the link between them. Prices are ours, per month.

**A four-card SXM node compared with a four-card PCIe node**

|  | [4 × NVIDIA A100 PCIe](https://gpuserver.io/gpu/a100-80gb) | [4 × NVIDIA A100 SXM4](https://gpuserver.io/gpu/a100-sxm4) |
|---|---|---|
| Link between cards | PCIe 5.0 ×16 | NVLink 3 · 600 GB/s |
| Total VRAM | 320 GB | 320 GB |
| Memory bandwidth per card | 1,935 GB/s | 2,039 GB/s |
| Llama 3.3 70B at BF16, sharded Single-stream generation, our conservative estimate | ~18 tok/s | ~19 tok/s |
| Price | $4,157/mo | $4,395/mo |

Our throughput model is bounded by memory bandwidth and is deliberately conservative — it does not model the all-reduce directly, so the real gap on a tensor-parallel job is *wider* than the table suggests, not narrower. Treat these as a floor for both machines and the ordering as the point.

## Measuring it yourself

Do this on the first day. It takes two minutes and it tells you whether the machine you were sold has the topology you paid for.

Topology, then actual bandwidth

```
# NV# means NVLink. PIX, PHB or SYS mean the traffic goes over PCIe.
$ nvidia-smi topo -m

# NVLink state and per-link throughput counters
$ nvidia-smi nvlink --status

# The honest test: an actual all-reduce across every card in the box.
# Compare busbw to the link's rated figure, not to the headline number.
$ docker run --rm --gpus all --ipc=host --shm-size=8g \
    nvcr.io/nvidia/pytorch:25.02-py3 \
    all_reduce_perf -b 8 -e 4G -f 2 -g 4
```

If `topo -m` shows `SYS` between two cards on a node you bought for tensor parallelism, those two cards are talking through the CPU and across sockets — the worst case on the machine. On our SXM nodes every pair reports `NV18`; anything else is a fault, and it is one we want to hear about on day one.

## What to buy

1. **Model fits on one card?** Buy one card. Nothing on this page applies, and the interconnect premium is money set on fire.
2. **Several independent models or many independent jobs?** Buy PCIe cards, as many as you need. You get more VRAM per dollar and you never cross the link.
3. **One model, too large for a single card, served at low concurrency?** PCIe works. Expect the sharding to cost you rather than gain you, and size the node for memory rather than speed.
4. **One model, too large for a single card, serving real traffic or being trained?** This is the case NVLink exists for. Buy SXM.
5. **A single job needing more than eight cards?** That needs a fabric between nodes, which we do not sell. We would rather tell you here than after the invoice — see [what we do not offer](https://gpuserver.io/network#limits).

The [configurator](https://gpuserver.io/configure) tells you which of these you are in: pick a model and it reports, for every node in the catalogue, whether it fits on a single card, has to be split, or does not fit at all. The word "split" is the moment this guide starts to matter.

## See which nodes hold your model on a single card.

The configurator answers that for all 41 configurations, and shows you the price of each.

[Open the configurator](https://gpuserver.io/configure) [Read the guides](https://gpuserver.io/guides)

## Other guides

- [Sizing · 9 min Choosing a card for image and video models What FLUX, SDXL, SD 3.5 and Wan 2.1 need in VRAM, which card in our catalogue holds each, and why the cheapest one that fits is rarely the one to rent. Read the guide](https://gpuserver.io/guides/gpu-for-flux-sdxl)
- [Sizing · 10 min Choosing a quantisation format What AWQ, GPTQ, GGUF and FP8 each cost in memory, speed and quality — and why the format with the smallest weights rarely gives the smallest model. Read the guide](https://gpuserver.io/guides/awq-vs-gptq-vs-fp8)
- [Sizing · 10 min Running a mixture-of-experts model Total parameters decide the machine, active parameters decide the speed. What DeepSeek V3 and Qwen 3 235B need in VRAM, and which node to rent for them. Read the guide](https://gpuserver.io/guides/mixture-of-experts)

---

Source: https://gpuserver.io/guides/nvlink-vs-pcie/. This file is generated from the same data as the website; if a figure here differs from a page, the page is authoritative and this file is stale — the canonical source is https://gpuserver.io/.