pgsql: Use a ResourceOwner to track buffer pins in all cases. - Mailing list pgsql-committers

Use a ResourceOwner to track buffer pins in all cases.

Historically, we've allowed auxiliary processes to take buffer pins without
tracking them in a ResourceOwner.  However, that creates problems for error
recovery.  In particular, we've seen multiple reports of assertion crashes
in the startup process when it gets an error while holding a buffer pin,
as for example if it gets ENOSPC during a write.  In a non-assert build,
the process would simply exit without releasing the pin at all.  We've
gotten away with that so far just because a failure exit of the startup
process translates to a database crash anyhow; but any similar behavior
in other aux processes could result in stuck pins and subsequent problems
in vacuum.

To improve this, institute a policy that we must *always* have a resowner
backing any attempt to pin a buffer, which we can enforce just by removing
the previous special-case code in resowner.c.  Add infrastructure to make
it easy to create a process-lifespan AuxProcessResourceOwner and clear
out its contents at appropriate times.  Replace existing ad-hoc resowner
management in bgwriter.c and other aux processes with that.  (Thus, while
the startup process gains a resowner where it had none at all before, some
other aux process types are replacing an ad-hoc resowner with this code.)
Also use the AuxProcessResourceOwner to manage buffer pins taken during
StartupXLOG and ShutdownXLOG, even when those are being run in a bootstrap
process or a standalone backend rather than a true auxiliary process.

In passing, remove some other ad-hoc resource owner creations that had
gotten cargo-culted into various other places.  As far as I can tell
that was all unnecessary, and if it had been necessary it was incomplete,
due to lacking any provision for clearing those resowners later.
(Also worth noting in this connection is that a process that hasn't called
InitBufferPoolBackend has no business accessing buffers; so there's more
to do than just add the resowner if we want to touch buffers in processes
not covered by this patch.)

Although this fixes a very old bug, no back-patch, because there's no
evidence of any significant problem in non-assert builds.

Patch by me, pursuant to a report from Justin Pryzby.  Thanks to
Robert Haas and Kyotaro Horiguchi for reviews.

Discussion: https://postgr.es/m/20180627233939.GA10276@telsasoft.com

Branch
------
master

Details
-------
https://git.postgresql.org/pg/commitdiff/3cb646264e8ced9f25557ce271284da512d92043

Modified Files
--------------
src/backend/access/transam/parallel.c          | 16 +++---
src/backend/access/transam/xlog.c              | 18 +++++++
src/backend/bootstrap/bootstrap.c              |  7 +++
src/backend/postmaster/autovacuum.c            | 10 ++--
src/backend/postmaster/bgwriter.c              | 12 +----
src/backend/postmaster/checkpointer.c          | 12 +----
src/backend/postmaster/walwriter.c             | 12 +----
src/backend/replication/logical/logicalfuncs.c |  7 ++-
src/backend/replication/logical/worker.c       |  9 ++--
src/backend/replication/slotfuncs.c            |  8 +--
src/backend/replication/walreceiver.c          |  6 ---
src/backend/replication/walsender.c            |  7 +--
src/backend/utils/init/postinit.c              | 11 ++++
src/backend/utils/resowner/resowner.c          | 74 ++++++++++++++++++++------
src/include/utils/resowner.h                   |  3 ++
src/test/modules/test_shm_mq/worker.c          | 20 +++----
16 files changed, 142 insertions(+), 90 deletions(-)