Re: [PATCH] Improve performance of NOTIFY over many databases (issueblocking on AccessExclusiveLock on object 0 of class 1262 of database 0) - Mailing list pgsql-hackers

From Martijn van Oosterhout
Subject Re: [PATCH] Improve performance of NOTIFY over many databases (issueblocking on AccessExclusiveLock on object 0 of class 1262 of database 0)
Date
Msg-id CADWG95sDbZy2_0_+8nGEAukK9yoMMoU4-_92cEOGMTMLfXLpVg@mail.gmail.com
Whole thread Raw
In response to Re: [PATCH] Improve performance of NOTIFY over many databases (issue blocking on AccessExclusiveLock on object 0 of class 1262 of database 0)  (Tom Lane <tgl@sss.pgh.pa.us>)
Responses Re: [PATCH] Improve performance of NOTIFY over many databases (issue blocking on AccessExclusiveLock on object 0 of class 1262 of database 0)  (Tom Lane <tgl@sss.pgh.pa.us>)
List pgsql-hackers
On Tue, 23 Jul 2019 at 19:21, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>
> Martijn van Oosterhout <kleptog@gmail.com> writes:
> > 2. Add a field to AsyncQueueEntry which points to the next listening
> > backend. This would allow the loops over all listening backends to
> > complete much faster, especially in the normal case where there are
> > not many listeners relative to the number of backends. The downside is
> > this requires an exclusive lock to remove listeners, but that doesn't
> > seem a big problem.
>
> I don't understand how that would work?  The sending backend doesn't
> know what the "next listening backend" is.  Having to scan the whole
> queue when a listener unlistens seems pretty awful too, especially
> if you need exclusive lock while doing so.

I mean tracking the listening backends specifically, so you can
replace the loops:

for (i=0; i < MaxBackends; i++)

with

for (i=QUEUE_FIRST_LISTENING_BACKEND; i; i = QUEUE_NEXT_LISTENING_BACKEND(i))

Such loops occur often when trying to advance the tail, when adding a
new listener,
when sending a notify, etc, all while holding a (exclusive) lock.
Seems like such an easy win
to only loop over the listening backends rather than all of them.

> > Do 2 & 3 seem like a good direction to go? I can probably work something up.
>
> I'm on board with 3, obviously.  Not following what you have in mind
> for 2.

Hope this clears it up a bit. Only waking up one at a time is a good
idea, but needs to some
careful thinking to prove it actually works.

Have a nice day,
-- 
Martijn van Oosterhout <kleptog@gmail.com> http://svana.org/kleptog/



pgsql-hackers by date:

Previous
From: Andrey Borodin
Date:
Subject: Re: Fetching timeline during recovery
Next
From: David Steele
Date:
Subject: Re: Fetching timeline during recovery