Thread: bg worker: patch 2 of 6 - job cache

bg worker: patch 2 of 6 - job cache

From
Markus Wanner
Date:
This is the major refactoring patch, which turns the background workers
into more permanent work-horses. After having processed a job, they now
stay connected to their database and wait for more jobs from the
coordinator.

On the coordinator side, a job cache got added, so it is able to queue
requests for jobs on databases which don't currently have an idle
background worker available. The existing autovacuum functionality got
separated from the background worker infrastructure.

The coordinator uses a transaction to read pg_database and start
background workers as appropriate. However, autovacuum still uses
pg_stat, which needs to get initialized after startup. The coordinator
triggers an initial VACUUM on the template database (template1), which
populates the required statistics to kick off autovacuum.

Note that none of the statistics functions differenciate between
autovacuum workers and other background job types, which might or might
not be what we want.

Another open issue is the starvation problem: jobs for a database, which
doesn't currently have any connected background workers might starve, if
the coordinator isn't allowed to fork any new worker (and all others are
idle on other databases).

Attachment