Skip to content
Advertisement

infinite loop malloc in a 32 bit kernel with 4 Gb RAM and 10 Gb Swap Partition

Joe

Let’s say I have a 32-bit kernel. 4 Gb RAM, 10 Gb Swap Partition.

I have a process which has malloc in an infinite loop. So, eventually system’s OOM would kill the process. Here are two arguments.

Argument 1: Since, it’s 32 bit Kernel with a Virtual address split of 3:1 i.e. 3Gb for user space and 1 Gb for Kernel space. The process would be allocated 3 Gb of memory and then there is no more memory to give. So, OOM would kill the process.

Argument 2: Since, it’s 32 bit Kernel with a Virtual address split of 3:1 i.e. 3Gb for user space and 1 Gb for Kernel space. The process would be allocated 4 Gb of memory and then there is no more memory to give. So, OOM would kill the process.

Argument 3: Malloc would first take 4 Gb memory of Ram, Then it would take 10 Gb of Swap partition and Then OOM would kill the process.

Which of these arguments (if any) is true ?

There are other answer’s on forum but I am not sure whether the answer to this question depends on whether its 32 bit kernel or 64 bit kernel ?

Advertisement

Answer

Why do think OOM will kill the process? In your example you will exhaust your address space earlier than you will use up all of your real memory. So in your example (32-bit) you will be able to allocate around 3 GB of address space (minus text/data/stack and other segments and also possible memory fragmentation) and then the system will just return ENOMEM.

If you’re on 64-bit system, things get more interesting. Now the result will heavily depend on whether you really are using this memory or not. If you’re not using it (just allocating), then due to overcommit (if, of course, you don’t have it disabled) you will be able to allocate some huge amounts of address space, but if you try to use it, that’s where (around 10+4 GB boundary) you will trigger OOM and that will kill the program.

update

It’s actually quite easy to check with a program like this:

#include <errno.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>

int main()
{
    static char *allocs[10 * 1024 * 1024];
    static unsigned int size;
    unsigned int i;
    unsigned int const chunk = 1024 * 1024;

    for (i = 0; i < (sizeof(allocs)/sizeof(allocs[0])); i++)
        if ((allocs[i] = malloc(chunk)) == NULL) {
            printf("failed to allocate after %u MB: %sn", i, strerror(errno));
            break;
        }

    size = i;
    if (size == (sizeof(allocs)/sizeof(allocs[0])))
         printf("allocated %u MB successfully!n", size);
    for (i = 0; i < size; i++) {
        memset(allocs[i], 1, chunk);
        if (i % 100 == 0)
            printf("memset %u MBn", i);
    }
    return 0;
}

It tries to allocate 10M of memory chunks, each 1M in size, so effectively that’s 10 TB. On 32-bit system with 4 GB of RAM it gives this result (shortened a bit):

failed to allocate after 3016 MB: Cannot allocate memory
memset 0 MB
memset 100 MB
...
memset 2900 MB
memset 3000 MB

As expected, there is just not enough address space and adding swap (I’ve done my tests with 5GB instead of 10) doesn’t help in any way.

With 64 bit it behaves a bit unexpectedly (this is without swap, just 4 GB of RAM) in that it doesn’t output anything and just gets killed by OOM killer while still doing allocations. But that’s explainable looking at OOM killer log:

[  242.827042] [ 5171]  1000  5171 156707933   612960  306023        0             0 a.out
[  242.827044] Out of memory: Kill process 5171 (a.out) score 905 or sacrifice child
[  242.827046] Killed process 5171 (a.out) total-vm:626831732kB, anon-rss:2451812kB, file-rss:28kB

So it was able to allocate around 620 GB (!) of virtual memory with 2.4 really mapped. Remember also, that we’re allocating with malloc(), so the C library has to do its own housekeeping and even if you estimate it to be around 0.5% overhead with such big allocations you get substantial numbers just for that (to be free of that overhead you can try using mmap()). At this stage we have enough memory used to be short of it, so the process gets killed while still doing allocations. If you’re to make it less greedy and change allocs to [100 * 1024] (which is just 100 GB effectively), you can easily see the effect of swap, because without it on 4 GB system you have:

allocated 102400 MB successfully!
memset 0 MB
memset 100 MB
...
memset 2800 MB
memset 2900 MB
Killed

And with 5 GB of swap added:

allocated 102400 MB successfully!
memset 0 MB
memset 100 MB
...
memset 7900 MB
memset 8000 MB
Killed

All as expected.

Advertisement