Re: Some other things about contrib/bloom and generic_xlog.c - Mailing list pgsql-hackers

From Craig Ringer
Subject Re: Some other things about contrib/bloom and generic_xlog.c
Date
Msg-id CAMsr+YGRFJ3cvNbFq0o=vMnHGL5naVR84FGLVSR2kUJQyAMg4w@mail.gmail.com
Whole thread Raw
In response to Re: Some other things about contrib/bloom and generic_xlog.c  (Michael Paquier <michael.paquier@gmail.com>)
Responses Re: Some other things about contrib/bloom and generic_xlog.c  (Michael Paquier <michael.paquier@gmail.com>)
Re: Some other things about contrib/bloom and generic_xlog.c  (Robert Haas <robertmhaas@gmail.com>)
List pgsql-hackers
On 12 April 2016 at 08:36, Michael Paquier <michael.paquier@gmail.com> wrote:
 

> Also, the entire point here is that extensions DON'T
> get to have custom apply routines, because we have no way for
> replay to know about such apply routines.  (It surely cannot look
> in the catalogs for them, but there's no other suitable infrastructure
> either.)  In turn, that means there's little value in having any custom
> data associated with the WAL records.

Well, yes, the startup process has no knowledge of that. That's why it
would need to go through a hook, but the startup process has no
knowledge of routines loaded via _PG_init perhaps?

The only way we really have at the moment is shared_preload_libraries.

This got discussed much earlier, possibly on a previous iteration of the generic xlog discussions rather than this specific thread. IIRC the gist was that we need a way to:

- Flag the need for a redo routine so that replay stops if that redo routine isn't available
- Find out *which* redo routine the generic log message needs during redo
- Get a pointer to that routine to actually call it

... and also possibly allow the admin to force skip of redo calls for a given extension (but not others) in case of a major problem.

If you rely on shared_preload_libraries, then "oops, I forgot to put myextension in shared_preload_libraries on the downstream" becomes a Bad Thing. There's no way to go back and redo the work you've passed over. You have to recreate the standby. Worse, it's silently wrong. The server has no idea it was meant to do anything and no way to tell the user it couldn't do what it was meant to do.

We can't register redo routines in the catalogs because we don't have access to the syscaches etc during redo (right?).

So how do we look at a generic log record, say "ok, the upstream that wrote this says it needs to invoke registered generic xlog hook 42, which is <func> from <extension> at <ptr>" ?

Personally I'd be willing to accept the limitations of the s_p_l and hook approach to the point where I think we should add the hook and accept the caveats of its use ... but I understand the problems with it. I understand not having custom redo routine support in this iteration of the patch. It's somewhat orthogonal job to this patch anyway - while handy for specific relation generic xlog, custom redo routines make most sense when combined with generic xlog that isn't necessarily associated with a specific relation. This patch doesn't support that.

--
 Craig Ringer                   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services

pgsql-hackers by date:

Previous
From: Michael Paquier
Date:
Subject: Re: Optimization for updating foreign tables in Postgres FDW
Next
From: Michael Paquier
Date:
Subject: Re: Some other things about contrib/bloom and generic_xlog.c