Getting RDMA into Kubernetes Pods, Part 3: IPvlan

Sat, 14 Nov 2026 00:00:00 +0000

This series tests four ways to give a Kubernetes pod RDMA for GPU training, on two pairs of A100 nodes. Part 1 used hostNetwork and set the baseline, on the second pair with the same setup. NCCL all_reduce reached 190.54 GB/s bus bandwidth at 8 GiB on 16 GPUs. Part 2 covers SR-IOV VFs, this part covers IPvlan, and part 4 covers DRA.

With IPvlan, the pod stays on the pod network. A device plugin gives it the RDMA devices of the rails. Multus calls the IPvlan CNI plugin for each rail, and that plugin adds an IPvlan interface to the pod. NVIDIA IPAM gives each interface its own IP address. The pod is not privileged, and it got the same bandwidth as hostNetwork. NCCL reached 189.15 GB/s, and 190.75 GB/s in a second run.

Ipvlan on Under the Model

Getting RDMA into Kubernetes Pods, Part 3: IPvlan