kubernetes-sigs/dranet · release v1.5.0 · released 2026-10-03

Inside dranet

Follow one network device from discovery on the host, through a ResourceClaim, into a Pod network namespace, and back to the host. Each step names the Go code that does the work.

Two callers

The kubelet calls dranet over the DRA gRPC API for slow preparation work. The container runtime calls it over NRI while it builds each Pod.

Beside CNI

CNI still owns eth0 and the Pod IP. dranet adds whole devices to the Pod: NICs, SR-IOV VFs, RDMA devices, or IPvlan children.

One handoff

The PodConfigStore carries every decision from Prepare to the NRI hooks. A bbolt file keeps it across driver restarts.

Example node
1 / 15
    CONTROL PLANE WORKER NODE dranet DaemonSet pod on every node hostNetwork · privileged apply allocate watch Pods, claims ResourceSlice claim status CRI NRI gRPC · dra.sock save load lookup devices attrs HTTP scan move · configure Youkubectl apply kube-apiserver DeviceClass ResourceClaim ResourceSlice Pod kube-schedulerDRA allocator Container runtimecontainerd or CRI-O kubeletDRA manager NRI hooksattach · detach PodConfigStorememory + bbolt DRA hooksprepare · unprepare PublisherResourceSlices Inventorynetlink · sysfs · PCI Cloud providerattrs · profiles IMDScloud metadata POD NETWORK NAMESPACE HOST NETWORK NAMESPACE
    The node as dranet sees it. Highlighted boxes and arrows take part in the current step. The lower row shows the example interfaces: netdevs on the first line, RDMA devices and device files on the second.

    Where the applied config comes from

    Prepare turns several inputs into one stored DeviceConfig. On a conflict, your config wins, then the provider config, then the profile. A later stage only fills fields that are not set and adds list entries.

    1. Pick one opaque config

      From status.allocation.devices.config, keep entries for driver dra.net that match the request. Kubernetes lists DeviceClass entries before claim entries. The first non-empty entry is used, and later entries are not read. An invalid entry before it fails the claim.

      dra_hooks.go:241

    2. Merge onto the provider config

      The provider config is the base. User values win for single fields. Lists are appended and de-duplicated with the user entry kept. Routes are keyed by destination and table.

      config_merge.go:30

    3. Resolve the profile

      If the result names a profile, the profile provider (cloud or webhook) returns more config. It fills fields that are still not set and adds list entries. Without a profile provider, the inventory drops a profile that the cloud provider advertised.

      dra_hooks.go:772

    4. Validate again

      Provider and profile output is external input. The merged result is defaulted and validated before use. A failure releases the profile, so a retry cannot leak it.

      dra_hooks.go:786

    5. Add the host snapshot

      Host addresses, routes, rules, permanent neighbors, and RDMA device files are added. Then the DeviceConfig is saved for the Pod and device.

      dra_hooks.go:381

    Passthrough or IPvlan child

    The interface.type field picks one of two attach models. Passthrough is the default. The OKE provider selects IPVLAN for Ethernet RDMA NICs.

    Passthrough (default) HOST NETNS POD NETNS eth1 gone while the Pod runs whole netdev moves net1 192.168.10.5/24 copied from the host IPvlan child HOST NETNS POD NETNS rdma0 10.224.0.25/12 parent stays child of rdma0 10.208.0.25/12 from config or profile
    Passthrough moves the whole interface into the Pod. An IPvlan child leaves the parent and its address on the host, and gives the Pod a second interface on the same link. Addresses are OKE-style examples.
    PassthroughIPvlan child (type: IPVLAN)
    Host interface while the Pod runsGone from the hostStays on the host with its address
    Pod addressesHost global addresses copied, or set in config, or DHCPSet in config or by a profile, or addressing: Unnumbered
    Routes and rulesHost routes of that interface and their table rules are copiedOnly from config or a profile
    MTU and MACBoth can be set. A VF MTU cannot exceed its PF.MTU up to the parent MTU. MAC is the parent MAC and cannot be set.
    RDMA namespace modeShared or exclusiveShared only, when the parent has an RDMA device
    On Pod stopMoved back under its host name and set up. dranet does not restore the host addresses.Child deleted
    CodensAttachNetdevaddIPVlan

    RDMA modes

    The RDMA subsystem has a network namespace mode for the whole node. dranet reads it once at start (rdma system show prints it). If the read fails, dranet assumes shared mode.

    Shared (ib_core netns_mode=1)Exclusive (netns_mode=0)
    Device files (rdma_cm, uverbsN, and umadN, issmN, ucmN when present)Added to every container of the PodAdded to every container of the Pod
    NetdevMoved (passthrough) or IPvlan childMoved (passthrough). IPvlan claims fail at Prepare.
    RDMA link device (mlx5_N)Stays on the host, shared by all namespacesMoved into the Pod. On stop it returns before the netdev.
    RDMA-only device (no netdev)Device files only. Condition Ready (RDMAOnlyDeviceReady).RDMA link moved. Conditions RDMALinkReady and Ready.
    CoderunPodSandbox blocks 2 and 3 · IPvlan check in prepareDevice

    Restart and shutdown

    After a restart

    • The bbolt file reloads every stored device config. Netns paths are not stored. Synchronize gets them again from the runtime.
    • Prepare is idempotent. A device already stored for the Pod is reused, so a second profile allocation cannot leak.
    • NRI hooks must be idempotent too. The runtime can retry a Pod start many times, and each retry calls the hooks again.
    • A save goes to bbolt before memory. If the disk write fails, Prepare fails and memory does not change.

    pod_device_config.go:187 · nri_hooks.go:43

    Graceful stop

    1. Stop the kubelet plugin. No new Prepare calls arrive.
    2. Wait until each prepared Pod has had no NRI activity for 10 s. Poll every 5 s. Give up after 5 min. The Pod termination grace period (30 s by default) usually ends first.
    3. Cancel the context and stop the NRI plugin.

    This order stops a Pod from starting without its devices when dranet is upgraded while Pods are being created.

    driver.go:266

    Code map

    PackageWhat it doesStart reading at
    cmd/dranetProcess wiring: flags, health server, store, filter, providers, driver startmain app.go:101
    pkg/driverDRA hooks, NRI hooks, netns moves, L3 config (routes, rules, VRF, neighbors), ethtool, eBPF, DHCP, PodConfigStoreprepareDevice dra_hooks.go:230
    runPodSandbox nri_hooks.go:147
    pkg/inventoryDevice discovery, attributes, rescan loop, provider callsscan db.go:214
    pkg/apisNetworkConfig types, defaults, validation, mergeNetworkConfig types.go:21
    pkg/cloudproviderProvider interfaces, detection, and the GCE, AWS, Azure, Alibaba, CoreWeave, OKE, and webhook providersCloudInstance cloud.go:35
    pkg/filterCEL filter applied at publish timeFilterDevices filter.go:27
    pkg/namesDevice names from PCI addresses and interface namesNormalizePCIAddress names.go:60
    internal/nlwrapnetlink wrapper from moby that retries interrupted dumpsretryOnIntr netlink.go:75