In Linux, when memory is requested (using calloc / malloc), if a contiguous block of the requested size is not available does the kernel map multiple separate pieces of memory into one single virtual block and hand it over to the application or is it allocated on disk?
If it is allocated on disk, when a large enough block becomes free, is it automatically moved to the RAM or does it live on the disk for its life?
Advertisement
Answer
In all Virtual Memory capable OS’s, nowadays all, the kernel uses a pool of different strategies together to manage the memory.
The processor guarantees to programs that memory exists up to the nominal addressing capability of the processor, or something less depending on specs. So you’re allowed to allocate memory up to MAX legal memory amount for that OS.
The resources used are the physical memory installed in the system and disk storage.
Depending on program requests, the MM (memory manager) allocates virtual space: the OS creates a data structure that describes the requested memory (generally referred to as creating paging directories), but no real memory is allocated. The next stage is called commitment, you get into this stage when trying to access the memory previously allocated. If there is any physical memory available, the MM, called through the exception generated by the attempt to access the memory, commits pages of real memory, inserting their address in the page directory, then control flows back to user code. Now the memory exists!
And if we try to access memory not allocated? This is the case of invalid memory access that generates an exception, in Linux a SegFault.
Going back to the physical memory commitment, what if there is no more physical memory available? The MM use different algorithms to look for physical memory not accessed for a long time, no more requested or discardable. Even other things get involved such as task priority, task state (suspended tasks are good candidates for memory recollecting), etc. All this physical memory can be recollected, its contents saved in a swap file on disk (paging file), and physical storage can then be mapped into the new process.
The reverse process is made when a program tries to access its memory currently cached on disk. The MM recollects physical memory from other processes (using the same techniques described above), commits it in process space, copies data from the disk cache and then gives back control to user code.
All these processes are completely transparent to the user, and you don’t have to worry about it.
If you want to read more search for memory management, paging files, GDT (Global Descriptor Tables) and LDT (Local Descriptor Tables) here for X86 architectures, other processor use different structures and registers, but the principle is the same.