linux: Linux Memory Management - Part 1

This document is my study on:

• Operating System Concept, Tenth edition, Abraham Silberschatz, Peter Baer Galvin, Greg Gagne, Part 4 Memory Management
• Linux Device Driver v3, Chapter 15 Memory Mapping and DMA

LLD3-ch15

Address types

There are some types of addreses in the Linux system, however the kernel code is not always very clear on exactly which type of address is being used.

Physical address

Physical address is the real address of memory hardware, which is used between the processor and system’s memory. The length of physical address is also depended on system architectures.

Kernel logical address

Kernel logical address as the normal address space of the kernel. This addess is a kind of special virtual address when it is a fixed offset from its physical address. Then the virtual contiguous memory regions are by nature physically contiguous.

We can use kmalloc() to allocate a region of memory, this function return a kernel logical address. Logical addresses are usually stored in variables of type unsigned long or void *. Kernel logical address can be converted to and from physical addresses using __pa(x) and __va(x) macros (these macro are only worked in low memory region).

Kernel virtual address

Kernel virtual addres is also mapping from kernel space address to physical address, however the kernel virtual addresses are non-contiguous physically.

All logical virtual addresses are kernel virtual addresses, but many kernel virtual addresses are not kernel logical addresses.

Kernel virtual addresses (no direct physical mapping) are allocated by using vmalloc() or kmap().

Bus address

CPU only can access its registers and mani memory, the peripheral devices has its own memory, then the IOMMU remaps the memory of the device to main memory via bus addresses.

User virtual Address

• In the user-space, the User virtual address is the same as User logical address, which is generated by CPU, and used by user programs.

• Each running process has its own virtual address space.

• User virtual address length depending on the hardware architecture of the system.

• When a user-space program is compiled, the code and data of a program (mapped to process memory layout) is assigned virtual addresses. When the program is loaded into memory (running), the operating system assigns it a range of virtual addresses within its address space and setups page tables, then program can use its virtual address space to access memory.

Example:

• In the system 32 bits The user virtual address length is 32 bits (a size of a pointer in this system is 4 bytes) –> a process can have a virtual address space upto 2^32 bytes (4GB)
• In the system 64 bits The user virtual address length is 64 bits (a size of a pointer in this system is 8 bytes) –> a process can have a virtual address space upto 2^64 bytes (a large number and we don’t need to care)

    // for 32 bits architecture
    int *p;
    printf(sizeof(p)); // = 4 bytes

    // for 64 bits architecture
    int *p;
    printf(sizeof(p)); // = 8 bytes

High memory and low memory

The diffrent between kernel virtual addresses and kernel logical addreses are highlighted in 32-bits system. In 32-bits system, the virtual address space is only 2^32 = 4GB, so we need to split 4GB virtual address space into two regions: user-space address (e.g. 3GB as default configuration) and kernel-space address (e.g. 1GB). Then, with only 1GB for kernel-space address, we cannot map all our RAM (e.g. 4, 8… GB), only a limmited region (e.g. depends on the configuration, maybe 896MB, if we have more memory means less space for kernel virtual addresses) is mapped into kernel, we call this region of RAM is low memory, and the mapped virtual addresses are kernel logical addresses. That is why using using __pa(x) and __va(x) macros only work in low memory.

The remain region of RAM is not mapped to kernel (because it is beyond the address range set aside for kernel virtual addresses). Before accessing a specific high-memory page, the kernel must set up an explicit virtual mapping to make that page available in the kernel’s address. Thus, many kernel data structures must be placed in low memory; high memory tends to be reserved for user-space process pages

Linux memory

Basic hardware

CPU only can access main memory (RAM) and register built into processing core. The CPU instructions can take memory addresses as arguments. If the data is not in memory, they must be moved there before CPU can operate on them.

CPU core are accessible within one cycle of the CPU clock by built-in registers. However, completing a main memory access may take many cycles of the CPU clock –> the CPU is stalled. (intolerable: cannot be accepted!). A cache (the hardware automatically speds up memory access without any OS control) is needed.