Re: dynamic background workers - Mailing list pgsql-hackers

From Andres Freund
Subject Re: dynamic background workers
Date
Msg-id 20130620154526.GE16659@awork2.anarazel.de
Whole thread Raw
In response to Re: dynamic background workers  (Robert Haas <robertmhaas@gmail.com>)
Responses Re: dynamic background workers
List pgsql-hackers
On 2013-06-20 11:29:27 -0400, Robert Haas wrote:
> > Do you mean pre-forking and connecting to a specific database? Or really
> > just the forking?
> 
> I've considered both at various times, although in this context I was
> mostly thinking about just the forking. Pre-connecting to a specific
> database would save an unknown but possibly significant amount of
> additional latency.  Against that, it's more complex (because we've
> got to track which preforked workers are associated with which
> databases) and there's some cost to guessing wrong (because then we're
> keeping workers around that we can't use, or maybe even having to turn
> around and kill them to make slots for the workers we actually need).
> I suspect we'll want to pursue the idea at some point but it's not
> near the top of my list.

Just as a datapoint, if you benchmark the numbers of forks that can be
performed by a single process (i.e. postmaster) the number is easily in
the 10s of thousands. Now forking that much has some scalability
implications inside the kernel, but still.
I'd be surprised if the actual fork is more than 5-10% of the current
cost of starting a new backend.

Greetings,

Andres Freund

-- Andres Freund                       http://www.2ndQuadrant.com/PostgreSQL Development, 24x7 Support, Training &
Services



pgsql-hackers by date:

Previous
From: Robert Haas
Date:
Subject: Re: dynamic background workers
Next
From: Peter Eisentraut
Date:
Subject: Re: Patch to add support of "IF NOT EXISTS" to others "CREATE" statements