Some program that I am currently working on consumes much more memory than I think it should. So I am trying to understand how glibc malloc trimming works. I wrote the following test:
#include <malloc.h>#include <unistd.h>#define NUM_CHUNKS 1000000#define CHUNCK_SIZE 100int main(){ // disable fast bins mallopt(M_MXFAST, 0); void** array = (void**)malloc(sizeof(void*) * NUM_CHUNKS); // allocating memory for(unsigned int i = 0; i < NUM_CHUNKS; i++) { array[i] = malloc(CHUNCK_SIZE); } // releasing memory ALMOST all memory for(unsigned int i = 0; i < NUM_CHUNKS - 1 ; i++) { free(array[i]); } // when enabled memory consumption reduces //int ret = malloc_trim(0); //printf("ret=%dn", ret); malloc_stats(); sleep(100000);}Test output (without calling malloc_trim):
Arena 0:system bytes = 112054272in use bytes = 112Total (incl. mmap):system bytes = 120057856in use bytes = 8003696max mmap regions = 1max mmap bytes = 8003584Even though almost all memory was released, this test code consumes much more resident memory than expected:
[root@node0-b3]# ps aux | grep testroot 14662 1.8 0.4 129736 **118024** pts/10 S 20:19 0:00 ./testProcess smaps:
0245e000-08f3b000 rw-p 00000000 00:00 0 [heap]Size: 109428 kBRss: 109376 kBPss: 109376 kBShared_Clean: 0 kBShared_Dirty: 0 kBPrivate_Clean: 0 kBPrivate_Dirty: 109376 kBReferenced: 109376 kBAnonymous: 109376 kBAnonHugePages: 0 kBSwap: 0 kBKernelPageSize: 4 kBMMUPageSize: 4 kBLocked: 0 kBVmFlags: rd wr mr mw me ac 7f1c60720000-7f1c60ec2000 rw-p 00000000 00:00 0 Size: 7816 kBRss: 7816 kBPss: 7816 kBShared_Clean: 0 kBShared_Dirty: 0 kBPrivate_Clean: 0 kBPrivate_Dirty: 7816 kBReferenced: 7816 kBAnonymous: 7816 kBAnonHugePages: 0 kBSwap: 0 kBKernelPageSize: 4 kBMMUPageSize: 4 kBLocked: 0 kBWhen I enable the call to malloc_trim the output of the test stays almost the same:
ret=1Arena 0:system bytes = 112001024in use bytes = 112Total (incl. mmap):system bytes = 120004608in use bytes = 8003696max mmap regions = 1max mmap bytes = 8003584However, the RSS decreases significantly:
[root@node0-b3]# ps aux | grep testroot 15733 0.6 0.0 129688 **8804** pts/10 S 20:20 0:00 ./testProcess smaps (after malloc_trim):
01698000-08168000 rw-p 00000000 00:00 0 [heap]Size: 109376 kBRss: 8 kBPss: 8 kBShared_Clean: 0 kBShared_Dirty: 0 kBPrivate_Clean: 0 kBPrivate_Dirty: 8 kBReferenced: 8 kBAnonymous: 8 kBAnonHugePages: 0 kBSwap: 0 kBKernelPageSize: 4 kBMMUPageSize: 4 kBLocked: 0 kBVmFlags: rd wr mr mw me ac 7f508122a000-7f50819cc000 rw-p 00000000 00:00 0 Size: 7816 kBRss: 7816 kBPss: 7816 kBShared_Clean: 0 kBShared_Dirty: 0 kBPrivate_Clean: 0 kBPrivate_Dirty: 7816 kBReferenced: 7816 kBAnonymous: 7816 kBAnonHugePages: 0 kBSwap: 0 kBKernelPageSize: 4 kBMMUPageSize: 4 kBLocked: 0 kBAfter calling malloc_trim, the heap got shunked. I assume the 8MB mmap segment is still available because of the last piece of memory which wasn’t released.
Why heap trimming isn’t performed automatically by malloc? Is there a way to configure malloc such that trimming will be done automatically (when it can save that much of a memory)?
I am using glibc version 2.17.
Advertisement
Answer
Largely for historical reasons, memory for small allocations comes from a pool managed with the brk system call. This is a very old system call — at least as old as Version 6 Unix — and the only thing it can do is change the size of an “arena” whose position in memory is fixed. What that means is, the brk pool cannot shrink past a block that is still allocated.
Your program allocates N blocks of memory and then deallocates N-1 of them. The one block it doesn’t deallocate is the one located at the highest address. That is the worst-case scenario for brk: the size can’t be reduced at all, even though 99.99% of the pool is unused! If you change your program so that the block it doesn’t free is array[0] instead of array[NUM_CHUNKS-1], you should see both RSS and address space shrink upon the final call to free.
When you explicitly call malloc_trim, it attempts to work around this limitation using a Linux extension, madvise(MADV_DONTNEED), which releases the physical RAM, but not the address space (as you observed). I don’t know why this only happens upon an explicit call to malloc_trim.
Incidentally, the 8MB mmap segment is for your initial allocation of array.