Dynamic expansion of the Linux stack

I’ve noticed the Linux stack starts small and expands with page faults caused by recursion/pushes/vlas up to size getrlimit(RLIMIT_STACK,...), give or take (defaults to 8MiB on my system).

Curiously though, if I cause page faults by addressing bytes directly, within the limit, Linux will just regularly segfault without expanding the page mapping (no segfault though, if I do it after I had e.g., alloca, cause the stack expansion).

Example program:

#include <stdio.h>
#include <unistd.h>
#include <stdint.h>
#include <stdlib.h>
#define CMD "grep stack /proc/XXXXXXXXXXXXXXXX/maps"
#define CMDP "grep stack /proc/%ld/maps"
void vla(size_t Sz)
{
    char b[Sz];
    b[0]='y';
    b[1]='';
    puts(b);
}
#define OFFSET (sizeof(char)<<12)
int main(int C, char **V)
{
    char cmd[sizeof CMD]; sprintf(cmd,CMDP,(long)getpid());
    if(system(cmd)) return 1;
    for(int i=0; ; i++){
        printf("%dn", i);
        char *ptr = (char*)(((uintptr_t)&ptr)-i*OFFSET);
        if(C>1) vla(i*OFFSET); //pass an argument to the executable to turn this on
        ptr[0] = 'x';
        ptr[1] = '';
        if(system(cmd)) return 1;
        puts(ptr);
    }
}

What kernel code is doing this? How does it differentiate between natural stack growth and me poking around in the address space?

Answer

The linux kernel takes the content of the stack pointer as the limit (within reasonable boundaries). Accessing the stack below the stack pointer minus 65536 and the size for 32 unsigned longs is causing a segmentation violation. So, if you access the memory down the stack you have to make sure, that the stack pointer somehow decreases with the accesses to have the linux kernel enlarge the segment. See this snippet from /arch/x86/mm/fault.c:

if (sw_error_code & X86_PF_USER) {
    /*
     * Accessing the stack below %sp is always a bug.
     * The large cushion allows instructions like enter
     * and pusha to work. ("enter $65535, $31" pushes
     * 32 pointers and then decrements %sp by 65535.)
     */
    if (unlikely(address + 65536 + 32 * sizeof(unsigned long) < regs->sp)) {
        bad_area(regs, sw_error_code, address);
        return;
    }
}

The value of the stack pointer register is key here!