Re: dynamic background workers - Mailing list pgsql-hackers

From Robert Haas
Subject Re: dynamic background workers
Date
Msg-id CA+TgmoaPNPX6XXedOroj7CgB6FGxKNc0j4Pu5bOpU20WK90sZA@mail.gmail.com
Whole thread Raw
In response to Re: dynamic background workers  (Alvaro Herrera <alvherre@2ndquadrant.com>)
List pgsql-hackers
On Wed, Jul 3, 2013 at 11:15 AM, Alvaro Herrera
<alvherre@2ndquadrant.com> wrote:
> Andres Freund escribió:
>> Just as a datapoint, if you benchmark the numbers of forks that can be
>> performed by a single process (i.e. postmaster) the number is easily in
>> the 10s of thousands. Now forking that much has some scalability
>> implications inside the kernel, but still.
>> I'd be surprised if the actual fork is more than 5-10% of the current
>> cost of starting a new backend.
>
> I played at having some thousands of registered bgworkers on my laptop,
> and there wasn't even that much load.  So yeah, you can have lots of
> forks.

Since no one seems to be objecting to this patch beyond the lack of
documentation, I've added documentation and committed it, with
appropriate rebasing and a few minor cleanups.  One loose end is
around the bgw_sighup and bgw_sigterm structure members.  If you're
registering a background worker for a library that is not loaded in
the postmaster, you can't (safely) use these for anything, because
it's possible (though maybe not likely) for the worker process to map
the shared library at a different address than where they are mapped
in the backend that requests the new process to be started.  However,
that doesn't really matter; AFAICS, you can just as well call pqsignal
to set the handlers to anything you want from the main entrypoint
before unblocking signals.  So I'm inclined to say we should just
remove bgw_sighup and bgw_sigterm altogether and tell people to do it
that way.

Alternatively, we could give them the same treatment that I gave
bgw_main: let the user specify a function name and we'll search the
appropriate DSO for it.  But that's probably less convenient for
anyone using this facility than just calling pqsignal() before
unblocking signals, so I don't see any real reason to go that route.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company



pgsql-hackers by date:

Previous
From: Greg Stark
Date:
Subject: Re: Differences in WHERE clause of SELECT
Next
From: Noah Misch
Date:
Subject: Re: findDependentObjects() mutual exclusion vs. MVCC catalog scans