Interesting Page Mapping Techniques and Articles

I want to start a collection of interesting things you can do with page mapping or articles / resources about page mapping.

  • growable arenas with pre-allocated virtual memory
  • sparse memory
  • circular buffers
  • copy on write pages
  • double mapping (e.g. to have one virtual address that only allows read, while another also allows write)
5 Likes

LMDB is a key-value store that is implemented using page mapping techniques without needing a transaction log, for more info: Lightning Memory-Mapped Database - Wikipedia

And here is a zig library for it (I haven’t used it yet, but it looks promising):

2 Likes

Not directly page mapping but interesting short read about how the kernel may relocate physical pages in some situations: https://lwn.net/Articles/368869/

1 Like

merging identical pages

MADV_MERGEABLE Kernel Samepage Merging (KSM)
From https://www.man7.org/linux/man-pages/man2/madvise.2.html:

Enable Kernel Samepage Merging (KSM) for the pages in the
range specified by addr and length. The kernel regularly
scans those areas of user memory that have been marked as
mergeable, looking for pages with identical content.
These are replaced by a single write-protected page (which
is automatically copied if a process later wants to update
the content of the page).

It can consume a lot of processing power; use with care.


merging mutually exclusive pages

I watched this talk a while ago that merges physical pages, while keeping virtual pages unchanged:

2 Likes

On linux with mmap the circular buffer is waaaay easier. You can request the entire space at once then remap over the allocated virtural range so you don’t need to keep trying in a loop. Surprised you can’t do that on Windows.

You can do that on windows using MapViewOfFile3 and VirtualAlloc2.
See this example for the CreateRingBuffer function.

1 Like

That makes so much more sense then. The way he describes it is a little messed.

Not sure on Windows, but on Linux at least if you try to mmap then mmap fixed address + some size it will almost always fail because it gives you a region directly under the the previous mapping (some lib usually minux some random extra if ASLR is turned on).