I need to be able to convert an instruction pointer (from a backtrace) into a file and line number after a program has terminated. During execution, I can store any additional information I will later require for this offline analysis, but the analysis does have to happen offline.
Right now, I along with emitting the instruction pointers, I am emitting a set of structs, which contain some of the info from dl_iterate_phdr. I include the name of module, the dlpi_addr field for the base address, and the p_memsz field of the first phdr flagged as executable.
Then, in my offline tool, I identify the module for a given address using the size and address table I saved. I subtract the dlpi_addr from the instruction address and pass that to addr2line.
This seems to work most of the time, but sometimes I think it’s off slightly (I.e., it found a neighboring symbol–I know line number info isn’t always correct). It also totally fails for the main executable. I’m pretty sure that I’m just not handling all the offsetting correctly.
What do I need to do for this to work in general? I known there are several libraries that do this (lldb, libdwarf) but they are not well documented. I’m also trying to carry the least information possible between the main program and the offline tool so I don’t want to just dump the memory map as a string, for example.
Here’s some example data:
dl_iterate_phdr callback code:
for ( ElfW(Half) i = 0; i < info->dlpi_phnum; i++ )
{
const ElfW(Phdr)& header = info->dlpi_phdr[i];
// Find the executable segment
if ( (header.p_flags & PF_X) && (header.p_type == PT_LOAD) )
{
if ( info->dlpi_addr == getauxval(AT_SYSINFO_EHDR) )
{
// vdso section.
continue;
}
printf("Module %dn", modules->m_Count);
printf("Module name: %sn", info->dlpi_name);
printf("Start address: %pn", info->dlpi_addr);
printf("Length: %jxn", (uintmax_t)header.p_memsz);
...
}
}
Output
Module 0 Module name: Start address: (nil) Length: 7276 Module 1 Module name: /lib/x86_64-linux-gnu/libdl.so.2 Start address: 0x7ff933d63000 Length: 2060 Module 2 Module name: /lib/x86_64-linux-gnu/libpthread.so.0 Start address: 0x7ff933b45000 Length: 188c4 Module 3 Module name: /usr/lib/x86_64-linux-gnu/libstdc++.so.6 Start address: 0x7ff933841000 Length: e549d Module 4 Module name: /lib/x86_64-linux-gnu/libm.so.6 Start address: 0x7ff93353b000 Length: 1042cc Module 5 Module name: /lib/x86_64-linux-gnu/libgcc_s.so.1 Start address: 0x7ff933325000 Length: 1548c Module 6 Module name: /lib/x86_64-linux-gnu/libc.so.6 Start address: 0x7ff932f60000 Length: 1b9cc0 Module 7 Module name: /lib64/ld-linux-x86-64.so.2 Start address: 0x7ff933f67000 Length: 22118 Module 8 Module name: /path/to/another.so Start address: 0x7ff932cc3000 Length: b682 Module 9 Module name: /path/to/yet/another.so Start address: 0x7ff932a1c000 Length: 164fc
cat /proc//maps
00400000-00408000 r-xp 00000000 fc:01 56098817 /path/to/main/executable 00607000-00608000 r--p 00007000 fc:01 56098817 /path/to/main/executable 00608000-00609000 rw-p 00008000 fc:01 56098817 /path/to/main/executable 010cc000-010ed000 rw-p 00000000 00:00 0 [heap] 7ff924000000-7ff924021000 rw-p 00000000 00:00 0 7ff924021000-7ff928000000 ---p 00000000 00:00 0 7ff92c000000-7ff92c021000 rw-p 00000000 00:00 0 7ff92c021000-7ff930000000 ---p 00000000 00:00 0 7ff931a1a000-7ff931a1b000 ---p 00000000 00:00 0 7ff931a1b000-7ff93221b000 rw-p 00000000 00:00 0 7ff93221b000-7ff93221c000 ---p 00000000 00:00 0 7ff93221c000-7ff932a1c000 rw-p 00000000 00:00 0 7ff932a1c000-7ff932a33000 r-xp 00000000 fc:01 56360961 /path/to/yet/another.so 7ff932a33000-7ff932c32000 ---p 00017000 fc:01 56360961 /path/to/yet/another.so 7ff932c32000-7ff932c33000 r--p 00016000 fc:01 56360961 /path/to/yet/another.so 7ff932c33000-7ff932cc3000 rw-p 00017000 fc:01 56360961 /path/to/yet/another.so 7ff932cc3000-7ff932ccf000 r-xp 00000000 fc:01 56229891 /path/to/another.so 7ff932ccf000-7ff932ece000 ---p 0000c000 fc:01 56229891 /path/to/another.so 7ff932ece000-7ff932ecf000 r--p 0000b000 fc:01 56229891 /path/to/another.so 7ff932ecf000-7ff932ed0000 rw-p 0000c000 fc:01 56229891 /path/to/another.so 7ff932ed0000-7ff932f60000 rw-p 00000000 00:00 0 7ff932f60000-7ff93311a000 r-xp 00000000 fc:01 17039726 /lib/x86_64-linux-gnu/libc-2.19.so 7ff93311a000-7ff93331a000 ---p 001ba000 fc:01 17039726 /lib/x86_64-linux-gnu/libc-2.19.so 7ff93331a000-7ff93331e000 r--p 001ba000 fc:01 17039726 /lib/x86_64-linux-gnu/libc-2.19.so 7ff93331e000-7ff933320000 rw-p 001be000 fc:01 17039726 /lib/x86_64-linux-gnu/libc-2.19.so 7ff933320000-7ff933325000 rw-p 00000000 00:00 0 7ff933325000-7ff93333b000 r-xp 00000000 fc:01 17039378 /lib/x86_64-linux-gnu/libgcc_s.so.1 7ff93333b000-7ff93353a000 ---p 00016000 fc:01 17039378 /lib/x86_64-linux-gnu/libgcc_s.so.1 7ff93353a000-7ff93353b000 rw-p 00015000 fc:01 17039378 /lib/x86_64-linux-gnu/libgcc_s.so.1 7ff93353b000-7ff933640000 r-xp 00000000 fc:01 17039591 /lib/x86_64-linux-gnu/libm-2.19.so 7ff933640000-7ff93383f000 ---p 00105000 fc:01 17039591 /lib/x86_64-linux-gnu/libm-2.19.so 7ff93383f000-7ff933840000 r--p 00104000 fc:01 17039591 /lib/x86_64-linux-gnu/libm-2.19.so 7ff933840000-7ff933841000 rw-p 00105000 fc:01 17039591 /lib/x86_64-linux-gnu/libm-2.19.so 7ff933841000-7ff933927000 r-xp 00000000 fc:01 8390197 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.19 7ff933927000-7ff933b26000 ---p 000e6000 fc:01 8390197 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.19 7ff933b26000-7ff933b2e000 r--p 000e5000 fc:01 8390197 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.19 7ff933b2e000-7ff933b30000 rw-p 000ed000 fc:01 8390197 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.19 7ff933b30000-7ff933b45000 rw-p 00000000 00:00 0 7ff933b45000-7ff933b5e000 r-xp 00000000 fc:01 17039713 /lib/x86_64-linux-gnu/libpthread-2.19.so 7ff933b5e000-7ff933d5d000 ---p 00019000 fc:01 17039713 /lib/x86_64-linux-gnu/libpthread-2.19.so 7ff933d5d000-7ff933d5e000 r--p 00018000 fc:01 17039713 /lib/x86_64-linux-gnu/libpthread-2.19.so 7ff933d5e000-7ff933d5f000 rw-p 00019000 fc:01 17039713 /lib/x86_64-linux-gnu/libpthread-2.19.so 7ff933d5f000-7ff933d63000 rw-p 00000000 00:00 0 7ff933d63000-7ff933d66000 r-xp 00000000 fc:01 17039596 /lib/x86_64-linux-gnu/libdl-2.19.so 7ff933d66000-7ff933f65000 ---p 00003000 fc:01 17039596 /lib/x86_64-linux-gnu/libdl-2.19.so 7ff933f65000-7ff933f66000 r--p 00002000 fc:01 17039596 /lib/x86_64-linux-gnu/libdl-2.19.so 7ff933f66000-7ff933f67000 rw-p 00003000 fc:01 17039596 /lib/x86_64-linux-gnu/libdl-2.19.so 7ff933f67000-7ff933f8a000 r-xp 00000000 fc:01 17039716 /lib/x86_64-linux-gnu/ld-2.19.so 7ff934047000-7ff934049000 rw-p 00000000 00:00 0 7ff93405e000-7ff93407e000 rw-p 00000000 00:00 0 7ff9340b2000-7ff9340c0000 rw-p 00000000 00:00 0 7ff9340dc000-7ff934152000 rw-p 00000000 00:00 0 7ff93415f000-7ff934166000 rw-p 00000000 00:00 0 7ff934172000-7ff934175000 rw-p 00000000 00:00 0 7ff93417b000-7ff93417c000 rw-p 00000000 00:00 0 7ff934185000-7ff934189000 rw-p 00000000 00:00 0 7ff934189000-7ff93418a000 r--p 00022000 fc:01 17039716 /lib/x86_64-linux-gnu/ld-2.19.so 7ff93418a000-7ff93418b000 rw-p 00023000 fc:01 17039716 /lib/x86_64-linux-gnu/ld-2.19.so 7ff93418b000-7ff93418c000 rw-p 00000000 00:00 0 7ffd31892000-7ffd318b6000 rw-p 00000000 00:00 0 [stack] 7ffd31913000-7ffd31915000 r--p 00000000 00:00 0 [vvar] 7ffd31915000-7ffd31917000 r-xp 00000000 00:00 0 [vdso] ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall]
readelf -l executable
Program Headers:
Type Offset VirtAddr PhysAddr
FileSiz MemSiz Flags Align
PHDR 0x0000000000000040 0x0000000000400040 0x0000000000400040
0x00000000000001f8 0x00000000000001f8 R E 8
INTERP 0x0000000000000238 0x0000000000400238 0x0000000000400238
0x000000000000001c 0x000000000000001c R 1
[Requesting program interpreter: /lib64/ld-linux-x86-64.so.2]
LOAD 0x0000000000000000 0x0000000000400000 0x0000000000400000
0x0000000000007276 0x0000000000007276 R E 200000
LOAD 0x0000000000007db8 0x0000000000607db8 0x0000000000607db8
0x0000000000000389 0x00000000000004c0 RW 200000
DYNAMIC 0x0000000000007dd8 0x0000000000607dd8 0x0000000000607dd8
0x0000000000000220 0x0000000000000220 RW 8
NOTE 0x0000000000000254 0x0000000000400254 0x0000000000400254
0x0000000000000044 0x0000000000000044 R 4
GNU_EH_FRAME 0x0000000000004a40 0x0000000000404a40 0x0000000000404a40
0x000000000000078c 0x000000000000078c R 4
GNU_STACK 0x0000000000000000 0x0000000000000000 0x0000000000000000
0x0000000000000000 0x0000000000000000 RW 10
GNU_RELRO 0x0000000000007db8 0x0000000000607db8 0x0000000000607db8
0x0000000000000248 0x0000000000000248 R 1
Example math:
- There’s a function in the main executable called “DoSomeWork” located at 0x401329. 0x401329, however, does not fall between 0x0 and 0x7276d. It does fall within 0x7276 of 0x400000, though, which is why my hack to treat the main module specially worked. (My bit about using __etext turns out to have been gibberish–the code never worked but never hurt either.) If I run addr2line on 0x401329, it works. But if I treated this module like all the others, I would fail to associate the address with any file because it doesn’t fall between (startAddress, startAddress + length).
- An example from libm that I use as a test is lgamma(float) which is at 0x7ff933561620 in this run. That cleanly falls within the range (0x7ff93353b000, 0x7ff93353b000 + 0x1042cc) which matches the memory map and the data from dl_iterate_phdr. So when I subtract the start address from the function address, I get the offset 0x26620 which works correctly when passed to addr2line for libm (it returns the file/line number of w_lgamma).
Thanks!
Advertisement
Answer
dlpi_addr is not the start address of the mapping, but the offset relative to the p_vaddr addresses in the PT_LOAD program headers at which they’re mapped. Add these values together and you’ll have absolute virtual address ranges.