Re: Some other things about contrib/bloom and generic_xlog.c - Mailing list pgsql-hackers

From Craig Ringer
Subject Re: Some other things about contrib/bloom and generic_xlog.c
Date
Msg-id CAMsr+YFHp4DW9i_aQ11m2o7+NxoM+xxQoSMyVb7Thz3W7tee_g@mail.gmail.com
Whole thread Raw
In response to Re: Some other things about contrib/bloom and generic_xlog.c  (Michael Paquier <michael.paquier@gmail.com>)
List pgsql-hackers
On 12 April 2016 at 10:09, Michael Paquier <michael.paquier@gmail.com> wrote:
On Tue, Apr 12, 2016 at 10:54 AM, Craig Ringer <craig@2ndquadrant.com> wrote: 

> If you rely on shared_preload_libraries, then "oops, I forgot to put
> myextension in shared_preload_libraries on the downstream" becomes a Bad
> Thing. There's no way to go back and redo the work you've passed over. You
> have to recreate the standby. Worse, it's silently wrong. The server has no
> idea it was meant to do anything and no way to tell the user it couldn't do
> what it was meant to do.

Well, one playing with the generic WAL record facility is

Yeah... that's what people say about a lot of things.

If it's there, someone will shoot their foot off with it and blame us.

There's a point where you have to say "well, that was dumb, you had to take the safety off, remove the barrel plug attached to the giant sign saying 'do not remove', find and insert the bolt, and load the ammunition yourself first, maybe you shouldn't have done that?"

However, that doesn't mean we should make it easy for a simple oversight to have serious data correctness implications. I'm on the fence about whether this counts; enough so that I wouldn't fight hard to get it in even if a patch existed, which it doesn't, and we weren't past feature freeze, which we are.
 
 
Any other
infrastructure is going to be one brick shy of a load

I disagree. It's very useful, just not for what you (and I) want to use it for. It's still quite reasonable for custom index representations, and it can be extended later.
 
We could for example force the
hook to set some validation flags that are then checked in the generic
redo routine, and mark in the WAL record itself that this record
should have used the hook. If the record is replayed and this hook is
missing, then do a FATAL and prevent recovery to move on.


Right, but _which_ routine on the standby? How does the standby know which hook must be called? You can't just say "any hook"; there might be multiple ones interested in different generic WAL messages. You need a registration system of some sort and a way for the standby and master to agree on how to identify extensions that have redo hooks for generic WAL. Then you need to be able to look up that registration in some manner during redo.

The simplest registration system would be something like a function called at _PG_init time that passes a globally-unique integer identifier for the extension that maps to some kind of central web-based registry where we hand out IDs. Forever. Then the standby says "this xlog needs extension with generic xlog id 42 but it's missing". But we all know how well those generic global identifier systems work when people are testing and prototyping stuff, want to change versions incompatibly, etc....

So I disagree it's as simple as a validation flag.

--
 Craig Ringer                   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services

pgsql-hackers by date:

Previous
From: Tom Lane
Date:
Subject: Re: [COMMITTERS] pgsql: Allow Pin/UnpinBuffer to operate in a lockfree manner.
Next
From: Andres Freund
Date:
Subject: Re: [COMMITTERS] pgsql: Allow Pin/UnpinBuffer to operate in a lockfree manner.