Part 2 of a series on RDMA in Kubernetes pods for GPU training. Each pod gets its own SR-IOV virtual function on each rail, with 1 or 2 VFs per physical function.

Part 1 of a series on RDMA in Kubernetes pods for GPU training. What a hostNetwork pod needs for RDMA, the NCCL baseline on two A100 nodes, and what the extra settings cost.

An interactive walkthrough of how Dranet finds a network device, gives it to a pod, and takes it back, with links to the code of each release.

Why pods on DOCA-OFED nodes read the host’s RoCE GIDs as zero, what the driver does, and what works instead.

Where the ARP, rp_filter and accept_ra settings of a multi-rail RDMA node live on Kubernetes, and who sets them.