Re: MultiXact\SLRU buffers configuration - Mailing list pgsql-hackers

From Thomas Munro
Subject Re: MultiXact\SLRU buffers configuration
Date
Msg-id CA+hUKGKTESKn9-BkUT8T3vwcgZQJqsV4PSQKpE+M1qUgfSTLMg@mail.gmail.com
Whole thread Raw
In response to Re: MultiXact\SLRU buffers configuration  (Andrey Borodin <x4mmm@yandex-team.ru>)
Responses Re: MultiXact\SLRU buffers configuration  (Andrey Borodin <x4mmm@yandex-team.ru>)
List pgsql-hackers
On Sun, Apr 4, 2021 at 7:57 AM Andrey Borodin <x4mmm@yandex-team.ru> wrote:
> I was toying around with big values. For example if we set different big xact_buffers we can get something like
> FATAL:  not enough shared memory for data structure "Notify" (72768 bytes requested)
> FATAL:  not enough shared memory for data structure "Async Queue Control" (2492 bytes requested)
> FATAL:  not enough shared memory for data structure "Checkpointer Data" (393280 bytes requested)
>
> But never anything about xact_buffers. I don't think it's important, though.

I had added the hash table size in SimpleLruShmemSize(), but then SimpleLruInit() passed that same value in when allocating the struct, so the struct was oversized.  Oops.  Fixed.

> > Likewise, I removed the cap of 16 buffers for commit_ts_buffers, but
> > only if you have track_commit_timestamp enabled.
 
> Is there a reason to leave 16 pages if commit_ts is disabled? They might be useful for some artefacts of previously enabled commit_ts?

Alvaro, do you have an opinion on that?

The remaining thing that bothers me about this patch set is that there is still a linear search in the replacement algorithm, and it runs with an exclusive lock.  That creates a serious problem for large caches that still aren't large enough.  I wonder if we can do something to improve that situation in the time we have.  I considered a bunch of ideas but could only find one that fits with slru.c's simplistic locking while tracking recency.  What do you think about a hybrid of SLRU and random replacement, that retains some characteristics of both?  You could think of it as being a bit like the tournament selection of the genetic algorithm, with a tournament size of (say) 8 or 16.  Any ideas on how to evaluate this and choose the number?  See attached.
Attachment

pgsql-hackers by date:

Previous
From: Robins Tharakan
Date:
Subject: buildfarm instance bichir stuck
Next
From: Heikki Linnakangas
Date:
Subject: Re: Yet another fast GiST build