Re: SLRU optimization - configurable buffer pool and partitioning the SLRU lock - Mailing list pgsql-hackers
From | Dilip Kumar |
---|---|
Subject | Re: SLRU optimization - configurable buffer pool and partitioning the SLRU lock |
Date | |
Msg-id | CAFiTN-sVW59pjh9rarVFR0sX-G-ZrUT-vzJAHA9KnhqkH6C42A@mail.gmail.com Whole thread Raw |
In response to | Re: SLRU optimization - configurable buffer pool and partitioning the SLRU lock ("Andrey M. Borodin" <x4mmm@yandex-team.ru>) |
Responses |
Re: SLRU optimization - configurable buffer pool and partitioning the SLRU lock
Re: SLRU optimization - configurable buffer pool and partitioning the SLRU lock |
List | pgsql-hackers |
On Sun, Nov 5, 2023 at 1:37 AM Andrey M. Borodin <x4mmm@yandex-team.ru> wrote: > On 30 Oct 2023, at 09:20, Dilip Kumar <dilipbalaut@gmail.com> wrote: > > changed the logic of SlruAdjustNSlots() in 0002, such that now it > starts with the next power of 2 value of the configured slots and > keeps doubling the number of banks until we reach the number of banks > to the max SLRU_MAX_BANKS(128) and bank size is bigger than > SLRU_MIN_BANK_SIZE (8). By doing so, we will ensure we don't have too > many banks > > There was nothing wrong with having too many banks. Until bank-wise locks and counters were added in later patchsets. I agree with that, but I feel with bank-wise locks we are removing major contention from the centralized control lock and we can see that from my first email that how much benefit we can get in one of the simple test cases when we create subtransaction overflow. > Having hashtable to find SLRU page in the buffer IMV is too slow. Some comments on this approach can be found here [0]. > I'm OK with having HTAB for that if we are sure performance does not degrade significantly, but I really doubt this isthe case. > I even think SLRU buffers used HTAB in some ancient times, but I could not find commit when it was changed to linear search. The main intention of having this buffer mapping hash is to find the SLRU page faster than sequence search when banks are relatively bigger in size, but if we find the cases where having hash creates more overhead than providing gain then I am fine to remove the hash because the whole purpose of adding hash here to make the lookup faster. So far in my test I did not find the slowness. Do you or anyone else have any test case based on the previous research on whether it creates any slowness? > Maybe we could decouple locks and counters from SLRU banks? Banks were meant to be small to exploit performance of locallinear search. Lock partitions have to be bigger for sure. Yeah, that could also be an idea if we plan to drop the hash. I mean bank-wise counter is fine as we are finding a victim buffer within a bank itself, but each lock could cover more slots than one bank size or in other words, it can protect multiple banks. Let's hear more opinion on this. > > On 30 Oct 2023, at 09:20, Dilip Kumar <dilipbalaut@gmail.com> wrote: > > I have taken 0001 and 0002 from [1], done some bug fixes in 0001 > > > BTW can you please describe in more detail what kind of bugs? Yeah, actually that patch was using the same GUC (multixact_offsets_buffers) in SimpleLruInit for MultiXactOffsetCtl as well as for MultiXactMemberCtl, see the below patch snippet from the original patch. @@ -1851,13 +1851,13 @@ MultiXactShmemInit(void) MultiXactMemberCtl->PagePrecedes = MultiXactMemberPagePrecedes; SimpleLruInit(MultiXactOffsetCtl, - "MultiXactOffset", NUM_MULTIXACTOFFSET_BUFFERS, 0, + "MultiXactOffset", multixact_offsets_buffers, 0, MultiXactOffsetSLRULock, "pg_multixact/offsets", LWTRANCHE_MULTIXACTOFFSET_BUFFER, SYNC_HANDLER_MULTIXACT_OFFSET); SlruPagePrecedesUnitTests(MultiXactOffsetCtl, MULTIXACT_OFFSETS_PER_PAGE); SimpleLruInit(MultiXactMemberCtl, - "MultiXactMember", NUM_MULTIXACTMEMBER_BUFFERS, 0, + "MultiXactMember", multixact_offsets_buffers, 0, MultiXactMemberSLRULock, "pg_multixact/members", LWTRANCHE_MULTIXACTMEMBER_BUFFER, SYNC_HANDLER_MULTIXACT_MEMBER); -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com
pgsql-hackers by date: