bg worker: patch 2 of 6 - job cache - Mailing list pgsql-hackers

From Markus Wanner
Subject bg worker: patch 2 of 6 - job cache
Date
Msg-id 4C3C78AB.1080707@bluegap.ch
Whole thread Raw
List pgsql-hackers
This is the major refactoring patch, which turns the background workers
into more permanent work-horses. After having processed a job, they now
stay connected to their database and wait for more jobs from the
coordinator.

On the coordinator side, a job cache got added, so it is able to queue
requests for jobs on databases which don't currently have an idle
background worker available. The existing autovacuum functionality got
separated from the background worker infrastructure.

The coordinator uses a transaction to read pg_database and start
background workers as appropriate. However, autovacuum still uses
pg_stat, which needs to get initialized after startup. The coordinator
triggers an initial VACUUM on the template database (template1), which
populates the required statistics to kick off autovacuum.

Note that none of the statistics functions differenciate between
autovacuum workers and other background job types, which might or might
not be what we want.

Another open issue is the starvation problem: jobs for a database, which
doesn't currently have any connected background workers might starve, if
the coordinator isn't allowed to fork any new worker (and all others are
idle on other databases).

Attachment

pgsql-hackers by date:

Previous
From: Markus Wanner
Date:
Subject: bg worker: patch 1 of 6 - permanent process
Next
From: Tom Lane
Date:
Subject: Re: log files and permissions