pgsql: Limit the size of numa_move_pages requests - Mailing list pgsql-committers

From Tomas Vondra
Subject pgsql: Limit the size of numa_move_pages requests
Date
Msg-id E1uWXpf-004Abk-2I@gemulon.postgresql.org
Whole thread Raw
List pgsql-committers
Limit the size of numa_move_pages requests

There's a kernel bug in do_pages_stat(), affecting systems combining
64-bit kernel and 32-bit user space. The function splits the request
into chunks of 16 pointers, but forgets the pointers are 32-bit when
advancing to the next chunk. Some of the pointers get skipped, and
memory after the array is interpreted as pointers. The result is that
the produced status of memory pages is mostly bogus.

Systems combining 64-bit and 32-bit environments like this might seem
rare, but that's not the case - all 32-bit Debian packages are built in
a 32-bit chroot on a system with a 64-bit kernel.

This is a long-standing kernel bug (since 2010), affecting pretty much
all kernels, so it'll take time until all systems get a fixed kernel.
Luckily, we can work around the issue by chunking the requests the same
way do_pages_stat() does, at least on affected systems. We don't know
what kernel a 32-bit build will run on, so all 32-bit builds use chunks
of 16 elements (the largest chunk before hitting the issue).

64-bit builds are not affected by this issue, and so could work without
the chunking. But chunking has other advantages, so we apply chunking
even for 64-bit builds, with chunks of 1024 elements.

Reported-by: Christoph Berg <myon@debian.org>
Author: Christoph Berg <myon@debian.org>
Author: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Discussion: https://postgr.es/m/aEtDozLmtZddARdB@msg.df7cb.de
Context: https://marc.info/?l=linux-mm&m=175077821909222&w=2
Backpatch-through: 18

Branch
------
master

Details
-------
https://git.postgresql.org/pg/commitdiff/7fe2f67c7c9f27955df584eb79edd6ec2be7f9e4

Modified Files
--------------
src/port/pg_numa.c | 50 +++++++++++++++++++++++++++++++++++++++++++++++++-
1 file changed, 49 insertions(+), 1 deletion(-)


pgsql-committers by date:

Previous
From: Amit Kapila
Date:
Subject: pgsql: Fix typo in pg_publication.h.
Next
From: Tomas Vondra
Date:
Subject: pgsql: Limit the size of numa_move_pages requests