I tried to go through the numa_distance() and other related functions (From the 1st link ), But couldn’t understand. I am just trying to understand how linux calculates the NUMA distance between two nodes when this distance is said to vary based on architecture and NUMA interlink.
I referred following links:
https://github.com/jmesmon/numactl/blob/0df3f720e606a3706700e0487ba19d720f50c4b8/distance.c
https://github.com/jmesmon/numactl/blob/0df3f720e606a3706700e0487ba19d720f50c4b8/numa.h
https://github.com/jmesmon/numactl/blob/0df3f720e606a3706700e0487ba19d720f50c4b8/libnuma.c
Advertisement
Answer
Inside (recent versions of) the ACPI specification you’ll find a description of a table called “SLIT”/System Locality (Distance) Information Table. This table is just an array (like d = array[numa_node][numa_node]
) that an operating system uses to determine the relative distance between any 2 NUMA nodes; where the values in the array range from 10 to 254 (255 is used for “no connection between these NUMA domains”), where the value 10 represents how quickly something in a NUMA domain can access something in the same NUMA domain (the fastest case) and the value 254 would be 25.4 times slower.
I’d assume that firmware populates this table using hard-coded values – e.g. the motherboard manufacturer might do a few measurements and determine value/s that are good enough for all models of CPUs that the motherboard supports.
Firmware provides this table to the OS. The OS doesn’t calculate anything.