Re: Improving connection scalability: GetSnapshotData() - Mailing list pgsql-hackers

From Ian Barwick
Subject Re: Improving connection scalability: GetSnapshotData()
Date
Msg-id d5f14b0e-58d2-56ab-0938-ab57ecc89d77@2ndquadrant.com
Whole thread Raw
In response to Re: Improving connection scalability: GetSnapshotData()  (Ian Barwick <ian.barwick@2ndquadrant.com>)
Responses Re: Improving connection scalability: GetSnapshotData()  (Andres Freund <andres@anarazel.de>)
List pgsql-hackers
On 2020/09/08 13:23, Ian Barwick wrote:
> On 2020/09/08 13:11, Andres Freund wrote:
>> Hi,
>>
>> On 2020-09-08 13:03:01 +0900, Ian Barwick wrote:
> (...)
>>> I wonder if it's possible to increment "xactCompletionCount"
>>> during replay along these lines:
>>>
>>>      *** a/src/backend/access/transam/xact.c
>>>      --- b/src/backend/access/transam/xact.c
>>>      *************** xact_redo_commit(xl_xact_parsed_commit *
>>>      *** 5915,5920 ****
>>>      --- 5915,5924 ----
>>>               */
>>>              if (XactCompletionApplyFeedback(parsed->xinfo))
>>>                      XLogRequestWalReceiverReply();
>>>      +
>>>      +       LWLockAcquire(ProcArrayLock, LW_EXCLUSIVE);
>>>      +       ShmemVariableCache->xactCompletionCount++;
>>>      +       LWLockRelease(ProcArrayLock);
>>>        }
>>>
>>> which seems to work (though quite possibly I've overlooked something I don't
>>> know that I don't know about and it will all break horribly somewhere,
>>> etc. etc.).
>>
>> We'd also need the same in a few more places. Probably worth looking at
>> the list where we increment it on the primary (particularly we need to
>> also increment it for aborts, and 2pc commit/aborts).
> 
> Yup.
> 
>> At first I was very confused as to why none of the existing tests have
>> found this significant issue. But after thinking about it for a minute
>> that's because they all use psql, and largely separate psql invocations
>> for each query :(. Which means that there's no cached snapshot around...
>>
>> Do you want to try to write a patch?
> 
> Sure, I'll give it a go as I have some time right now.


Attached, though bear in mind I'm not very familiar with parts of this,
particularly 2PC stuff, so consider it educated guesswork.


Regards


Ian Barwick

-- 
Ian Barwick                   https://www.2ndQuadrant.com/
  PostgreSQL Development, 24x7 Support, Training & Services

Attachment

pgsql-hackers by date:

Previous
From: Kyotaro Horiguchi
Date:
Subject: Re: shared-memory based stats collector
Next
From: Peter Eisentraut
Date:
Subject: Re: Since '2001-09-09 01:46:40'::timestamp microseconds are lost when extracting epoch