Re: pg_background (and more parallelism infrastructure patches) - Mailing list pgsql-hackers

On 2014-11-10 12:10:49 -0500, Robert Haas wrote:
> - Patch 4 has had some review and really, as far as I can tell, the
> only really major issue that has cropped up is the possibility that
> the GUC settings that are legal in backend A aren't legal in backend B
> for some reason; in that case, restoring the GUC settings there will
> probably fail.  While this is a legitimate concern, I think what it
> really boils down to is that there's a possibility that there are
> libraries loaded in the user backend that are not loaded in the
> postmaster and therefore won't get loaded into the background worker.

I'm not too concerned about that one.

> That's an issue to which we may need to engineer some solution, but
> not in this patch.  Overall, the patch's architecture is modeled
> closely on the way we synchronize GUCs to new backends when using
> EXEC_BACKEND, and I don't think we're going do any better than stating
> with that and working to file down the rough edges as we go.  So I'd
> like to go ahead and commit this.

Did you check out whether PGC_BACKEND GUCs work? Other than that I agree
that we can just solve further issues as we notice them. This isn't
particularly intrustive.

> -- There's some code duplication with exec_simple_query() that doesn't
> feel great, but it's not clear that there's a superior alternative.  I
> think we certainly don't want to complicate exec_simple_query() to
> make pg_background happy.

Yea, I think we can live with it if really necessary.

> -- We should add REVOKE to the SQL script that installs it so that
> non-superusers can't use it unless the superuser decides to grant them
> rights.

Right. That seems trivial enough.

> -- It doesn't handle types without send/receive functions.  I thought
> that was tolerable since the functions without such types seem like
> mostly edge cases, but Andres doesn't like it.  Apparently, BDR is
> makes provisions to either ship the tuple as one big blob - if all
> built-in types - or else use binary format where supported and text
> format otherwise.  Since this issue is common to BDR, parallelism, and
> pg_background, we might want to introduce some common infrastructure
> for it, but it's not too clear to me right now what that should look
> like.

I think we definitely need to solve this - but I'm also not at all that
sure how.

For something like pg_background it's pretty trivial to fall back to
in/out when there's no send/recv. It's just a couple of lines, and the
portal code has the necessary provisions. So solving it for
pg_background itself is pretty trivial.  But, as you say, pg_background
itself isn't a primary goal (although a nice demonstration, with some
real world applications).  There's enough differences between the
parallelism and the replication cases that I'm not entirely sure how
much common ground there is. Namely replication has the additional
concerns of version differences, trust (don't use blobs if you don't
fully trust the other side), and differences in the allocated oids
(especially relevant for arrays which, very annoyingly, embed the oid in
the send/recv format).

> 1. Any other opinions for or against pg_background as a concept?  I
> thought the ability to kick of vacuums (or other stuff with
> PreventTransactionChain) asynchronously was pretty cool, as we as the
> ability to use it as an authentication-free loopback dblink.  But
> opinions obviously vary.

I think it's a cool concept, but I'm not sure if it's worth the work to
make it fully usable. Or rather, I think it's worthy enough, but I
personally would priorize other stuff.

> 2. Is anyone sufficiently interested in pg_background as a concept
> that they'd be willing to take over the patch and work on the TODO
> list mentioned above?

I personally won't. If we can come up with a sketch of how to deal with
the data transport encoding issue above, I'd be willing to to work on
that specific part. But not pg_background in itself.

Greetings,

Andres Freund