Programmatically determine NUMA node or PCI bus, device, function number of generic PCI device

I pretty much need the same thing as Programmatically determine NUMA node or PCI bus, device, function number of Direct3D9Ex device

I’m looking for a way to programmatically determine which NUMA node a particular PCI device is connected to so that I can allocate memory for host to device transfers on that node.

So, my code opens a device file like /dev/mydev0, /dev/mydev1, etc. I can have several of those device files, and they can be on different NUMA nodes. What I need to figure out is the NUMA node, if any, that device belongs to, in application code.

I’m writing the device driver for these devices. I understand that the pointer for a struct device returned by device_create has the field

#ifdef CONFIG_NUMA;
  int numa_node;
#endif;

My idea would be to export that value through a custom ioctl call in my custom driver.

What I’m asking is if there’s a more standard way to do map the /dev entry to a NUMA node so an application can figure that out and use something like numa_bind so the memory allocations happens on the correct node.

I also understand that I can check the NUMA node of a PCI device with lspci. I don’t see how can I use that based on the /dev entry.

Thanks!

Answer

Sysfs to the rescue! Say that your device is /dev/fb0, then

$ cat  /sys/class/graphics/fb0/device/numa_node
0

Will give you the NUMA node.

Opening /sys/class/graphics/fb0/device/numa_node in text mode and reading from it will yield "0n" (0x30, 0x0A).

Advertisement

Answer