pgsql: Improve ReadRecentBuffer() scalability - Mailing list pgsql-committers

From Andres Freund
Subject pgsql: Improve ReadRecentBuffer() scalability
Date
Msg-id E1v6Xks-000hve-2C@gemulon.postgresql.org
Whole thread Raw
List pgsql-committers
Improve ReadRecentBuffer() scalability

While testing a new potential use for ReadRecentBuffer(), Andres
reported that it scales badly when called concurrently for the same
buffer by many backends.  Instead of a naive (but wrong) coding with
PinBuffer(), it used the spinlock, so that it could be careful to pin
only if the buffer was valid and holding the expected block, to avoid
breaking invariants in eg GetVictimBuffer().  Unfortunately that made it
less scalable than PinBuffer(), which uses compare-exchange instead.

We can fix that by giving PinBuffer() a new skip_if_not_valid mode that
doesn't pin invalid buffers.  It might occasionally skip when it
shouldn't due to the unlocked read of the header flags, but that's
unlikely and perfectly acceptable for an opportunistic optimisation
routine, and it can only succeed when it really should due to the
compare-exchange loop.

Note that this fixes ReadRecentBuffer()'s failure to bump the usage
count. While this could be seen as a bug, there currently aren't cases
affected by this in core, so it doesn't seem worth backpatching that portion.

Author: Thomas Munro <thomas.munro@gmail.com>
Reported-by: Andres Freund <andres@anarazel.de>
Reviewed-by: Andres Freund <andres@anarazel.de>
Reviewed-by: Matthias van de Meent <boekewurm+postgres@gmail.com>
Discussion: https://postgr.es/m/20230627020546.t6z4tntmj7wmjrfh%40awork3.anarazel.de
Discussion: https://postgr.es/m/fvfmkr5kk4nyex56ejgxj3uzi63isfxovp2biecb4bspbjrze7@az2pljabhnff

Branch
------
master

Details
-------
https://git.postgresql.org/pg/commitdiff/819dc118c0f6cd1fc08f0e807702b4bc0b0d8733

Modified Files
--------------
src/backend/storage/buffer/bufmgr.c | 64 ++++++++++++++++---------------------
1 file changed, 28 insertions(+), 36 deletions(-)


pgsql-committers by date:

Previous
From: Masahiko Sawada
Date:
Subject: pgsql: Add mem_exceeded_count column to pg_stat_replication_slots.
Next
From: Andres Freund
Date:
Subject: pgsql: bufmgr: Introduce FlushUnlockedBuffer