Which parts of Kueue’s topology-aware scheduling are moving into kube-scheduler, tested on Kubernetes v1.37.1 with the same topology and mostly the same experiments as my Kueue post.
Notes on GPUs, RDMA and Kubernetes.
RDMA Network Locality for Multi-Node Jobs with Kueue
How Kueue’s topology-aware scheduling keeps the pods of a multi-node job in one RDMA local block or network block, tested step by step on a cluster without Kueue.
Why Large RoCE Frames Disappeared on OVS-DOCA 3.5
How OVS-DOCA 3.5 kept receive buffers sized for a 1500-byte MTU, so large RoCE frames vanished. One ovs-vsctl transaction fixed it.
Why torchrun Can’t Find Node Two on the Host Network
Why torchrun loses track of itself when Kubernetes pods use the host network for RDMA, two related PyTorch issues open since 2022, and what fixed it on two A100 nodes.