next up previous contents index
Next: 3.1 Nodes Up: understand-html Previous: 2.3 Submitting Work   Contents   Index

3. Describing Physical Memory

Linux is available for many architectures so there needs to be an architecture independent way of describing memory. This section describes the structures used to keep account of memory banks, pages and the flags that affect VM behavior.

With large scale machines, memory may be arranged into banks that incur a different cost to use depending on processor. For example, there might be a bank of memory assigned to each CPU or a bank of memory very suitable for DMA. These banks are said to be at varying distances and exist on architectures referred to as NUMA (Non-Uniform Memory Access) architectures.

In Linux, each bank is called a node and is represented by struct pg_data_t. Every node in the system is kept on a NULL terminated list called pgdat_list. Each node is linked to the next with the field by pg_data_t$\rightarrow$node_next. For UMA architectures like PC desktops only one static pg_data_t structure called contig_page_data is used.

Each node is then divided up into a number of blocks called zones which represent ranges within memory. A zone is described by a struct zone_t and each one is one of ZONE_DMA, ZONE_NORMAL or ZONE_HIGHMEM. Each is suitable a different type of usage. ZONE_DMA is memory in the lower physical ranges which certain ISA devices require. ZONE_NORMAL is memory that can be directly mapped by the kernel in the upper 1GB of the linear address space. It is important to note that many kernel operations can only take place using ZONE_NORMAL so it is the most performance critical zone. ZONE_HIGHMEM is the rest of memory. With the x86 the zones are

ZONE_DMA First 16MiB of memory
ZONE_NORMAL 16MiB - 896MiB
ZONE_HIGHMEM 896 MiB - End


Each physical page frame is represented by a struct page and all the structs are kept in mem_map array that is stored at PAGE_OFFSET, the beginning of the virtual address space the kernel can see, 3GB on the x86. The pages are stored in this area so that the physical address for a page struct may be easily calculated. Each zone has a pointer within this array called zone_mem_map. Note that a struct page is not the actual physical page.

The high memory extensions allow the kernel to address up to 64GB in theory but in practice it can't. The struct pages to describe each page frame requires 44 bytes and this uses kernel virtual address space in ZONE_NORMAL. That means to describe 1GB of memory, approximately 45MiB of kernel memory is required. At 16GB, 720MiB of memory is consumed and almost exhausts ZONE_NORMAL making 16GB about the practical limit for physical memory on an x86.

The relationship between the structs mentioned so far is described in .

Figure 3.1: Relationship Between Nodes, Zones and Pages
\includegraphics[width=15cm]{graphs/layout.ps}



Subsections
next up previous contents index
Next: 3.1 Nodes Up: understand-html Previous: 2.3 Submitting Work   Contents   Index
Mel 2003-01-14