I have the following code:
extern printf section .data A db 1 B db 2 C db 3 D db 4 E db 5 fmt db "%d",10,0 fmt2 db "An overflow has occured",10,0 section .text global _start _start: xor rax, rax xor rbx, rbx xor rdx, rdx xor ax, ax mov al, [B] mul ax jo over call printax mov bx, ax xor ax, ax mov al, [C] mul ax jo over call printax xor cx, cx mov cl, [C] mul cx jo over call printax xor cx, cx mov cl, [E] div cx jo over call printax xor dx, dx xor al, al mov rdi, fmt call printf jmp exit over: mov rdi, fmt2 xor al, al call printf exit: mov rax, 60 mov rdi, 0 syscall printbx: xor rsi, rsi mov si, bx mov cl, al xor al, al mov rdi, fmt call printf mov al, cl xor si, si ret printax: xor rsi, rsi mov si, ax xor al, al mov rdi, fmt call printf mov ax, si xor si, si ret
I compile it with nasm -f elf64 1.asm
and link it using ld -dynamic-linker /lib/ld-linux-x86-64.so.2 1.o -o 1 -lc
. When I execute the binary, I get
$ ./1 4 9 0 Floating point exception (core dumped)
printf()
doesn’t always fail when I call it in assembly code.
Removing printf()
calls from printax
and printbx
gives me
$ ./1 0
Update: The exception also disappears if I remove the div cx
line. Then I get the following output:
$./1 4 9 0 0 0
But it doesn’t disappear even if I add
mov cx, 1 mov ax, 1
before div cx
.
Advertisement
Answer
Take another look at the x86-64 ABI calling conventions. Specifically, on page 15:
Registers %rbp, %rbx and %r12 through %r15 “belong” to the calling function and the called function is required to preserve their values. In other words, a called function must preserve these registers’ values for its caller. Remaining registers “belong” to the called function. If a calling function wants to preserve such a register value across a function call, it must save the value in its local stack frame.
So when you call printf
, you must assume that all registers except rbp, rbx, r12..r15
are clobbered. This has two effects for your program:
You attempt to save
ax
around the call toprintf
by stashing its value in thesi
register and then putting it back later, but this doesn’t help becausesi
can be clobbered.Your
div cx
instruction divides the contents ofdx:ax
bycx
, butdx
might also have been clobbered.
The latter is the specific cause of the SIGFPE (which despite its name is also raised on an integer divide overflow), at least in my test runs. After the printf
returns, dx
contains some huge number, such that dx:ax
divided by cx
does not fit in 16 bits, which is an overflow.
(This also explains why the crash went away when you took out the printf
call – it was no longer there to clobber your registers.)
The other lesson here is that you can’t check for divide overflow on x86 by doing a jo
afterwards; this error is primarily signaled by an exception, not by flags. So you really have to check your operands before executing the instruction, or else arrange to handle the exception if it occurs (which is more complicated and beyond the scope of this answer).
A couple of other notes:
Before your final call to
printf
(just beforejmp exit
), you don’t load anything intorsi
, so the value printed is garbage.16-bit arithmetic is usually not to be preferred on x86-32 or x86-64 unless there is a really good reason for it. It is no faster and bloats your code with operand size prefixes. Better to do all your work with 32-bit arithmetic.
Since you’re using your own entry point rather than letting the C library call your
main
, this means that libc has not had the opportunity to run its own initialization code. Therefore, it is not necessarily safe to call any libc function, particularly stdio and allocation functions. It seems thatprintf
happens to work okay in this case, and maybe it is all right for debugging, but you shouldn’t plan to write your program this way for production.