On Fri, Oct 24, 2025 at 8:38 AM Tomas Vondra <tomas@vondra.me> wrote:
Hmmm, I wonder if the m_w_m is high enough to confuse the trimming logic in some way. Can you try if using smaller m_w_m (maybe 128MB-256MB) makes the issue go away?
The index builds at up to 4GB of m_w_m. 5GB and above crashes.
Now that I know roughly where the limits are that de-escalates things a bit. The sort of customers deploying a month after release should be OK with just knowing to be careful about high m_w_m settings on PG18 until a fix is ready.
Hope everyone is enjoying Latvia. My obscure music collection includes a band from there I used to see in the NYC area, The Quags; https://www.youtube.com/watch?v=Bg3P4736CxM
Can you show the contents of buffer and tup? I'm especially interested in these fields: buffer->nitems buffer->maxitems buffer->nfrozen tup->nitems
I'll see if I can grab that data at the crash point.
FYI for anyone who wants to replicate this: if you have a system with 128GB+ of RAM you could probably recreate the test case. Just have to download the Planet file and run osm2pgsql with the overly tweaked settings I use. I've published all the details of how I run this regression test now.
If I'm right, I think there are two ways to fix this: (1) apply the trimming earlier, i.e. try to freeze + flush before actually merging the data (essentially, update nfrozen earlier) (2) use repalloc_huge (and palloc_huge) in GinBufferStoreTuple Or we probably should do both.
Sounds like (2) is probably mandatory and (1) is good hygiene.
