Re: System load consideration before spawning parallel workers - Mailing list pgsql-hackers

From Haribabu Kommi
Subject Re: System load consideration before spawning parallel workers
Date
Msg-id CAJrrPGd=CJX6VsBtccAG49YC9PvQ-eNQo=uKbN4LZSGXbySsyQ@mail.gmail.com
Whole thread Raw
In response to Re: System load consideration before spawning parallel workers  (Jim Nasby <Jim.Nasby@BlueTreble.com>)
Responses Re: System load consideration before spawning parallel workers  (Peter Eisentraut <peter.eisentraut@2ndquadrant.com>)
List pgsql-hackers
On Fri, Aug 5, 2016 at 9:46 AM, Jim Nasby <Jim.Nasby@bluetreble.com> wrote:
> On 8/1/16 1:08 AM, Haribabu Kommi wrote:
>>
>> There are some utilities and functions that are available to calculate the
>> current system load, based on the available resources and system load,
>> the module can allow the number of parallel workers that can start. In my
>> observation, adding this calculation will add some overhead for simple
>> queries. Because of this reason, i feel this can be hook function, only
>> for
>> the users who want it, can be loaded.
>
>
> I think we need to provide more tools to allow users to control system
> behavior on a more dynamic basis. How many workers to launch is a good
> example. There's more reasons than just CPU that parallel workers can help
> (IO being an obvious one, but possible other things like GPU). Another
> example is allowing users to alter the selection process used by autovac
> workers.

Yes, we need to consider many parameters as a system load, not just only
the CPU. Here I attached a POC patch that implements the CPU load
calculation and decide the number of workers based on the available CPU
load. The load calculation code is not an optimized one, there are many ways
that can used to calculate the system load. This is just for an example.


Regards,
Hari Babu
Fujitsu Australia

Attachment

pgsql-hackers by date:

Previous
From: Michael Meskes
Date:
Subject: Re: Let's get rid of the separate minor version numbers for shlibs
Next
From: Craig Ringer
Date:
Subject: Re: PSA: Systemd will kill PostgreSQL