Re: Slow standby snapshot - Mailing list pgsql-hackers

From Andrey Borodin
Subject Re: Slow standby snapshot
Date
Msg-id 5EFB07AE-ADAF-4EF8-90D8-17CC884C8466@yandex-team.ru
Whole thread Raw
In response to Re: Slow standby snapshot  (Andres Freund <andres@anarazel.de>)
Responses Re: Slow standby snapshot  (Andres Freund <andres@anarazel.de>)
List pgsql-hackers
Hi,

> 3 авг. 2021 г., в 03:01, Andres Freund <andres@anarazel.de> написал(а):
>
> Hi,
>
> On 2021-08-03 00:07:23 +0300, Michail Nikolaev wrote:
>> The main idea is simple optimistic optimization - store offset to next
>> valid entry. So, in most cases, we could just skip all the gaps.
>> Of course, it adds some additional impact for workloads without long
>> (few seconds) transactions but it is almost not detectable (because of
>> CPU caches).
>
> I'm doubtful that that's really the right direction. For workloads that
> are replay heavy we already often can see the cost of maintaining the
> known xids datastructures show up significantly - not surprising, given
> the datastructure. And for standby workloads with active primaries the
> cost of searching through the array in all backends is noticeable as
> well.  I think this needs a bigger data structure redesign.

KnownAssignedXids implements simple membership test idea. What kind of redesign would you suggest? Proposed
optimisationmakes it close to optimal, but needs eventual compression. 

Maybe use a hashtable of running transactions? It will be slightly faster when adding\removing single transactions. But
muchworse when doing KnownAssignedXidsRemove(). 

Maybe use a tree? (AVL\RB or something like that) It will be slightly better, because it does not need eventual
compressionlike exiting array. 

Best regards, Andrey Borodin.


pgsql-hackers by date:

Previous
From: vignesh C
Date:
Subject: Re: Failed transaction statistics to measure the logical replication progress
Next
From: "osumi.takamichi@fujitsu.com"
Date:
Subject: RE: Failed transaction statistics to measure the logical replication progress