Re: [v9.3] Extra Daemons (Re: elegant and effective way for running jobs inside a database) - Mailing list pgsql-hackers

From Kohei KaiGai
Subject Re: [v9.3] Extra Daemons (Re: elegant and effective way for running jobs inside a database)
Date
Msg-id CADyhKSXz0AjpDBQn+5apBjOd-ToAb3fWnYh=02AUMBkyQTKHbg@mail.gmail.com
Whole thread Raw
In response to [v9.3] Extra Daemons (Re: elegant and effective way for running jobs inside a database)  (Kohei KaiGai <kaigai@kaigai.gr.jp>)
Responses Re: [v9.3] Extra Daemons (Re: elegant and effective way for running jobs inside a database)
List pgsql-hackers
2012/9/20 Amit Kapila <amit.kapila@huawei.com>:
> On Thursday, September 20, 2012 1:44 AM Simon Riggs wrote:
> On 12 September 2012 04:30, Amit Kapila <amit.kapila@huawei.com> wrote:
>> On Tuesday, September 11, 2012 9:09 PM Alvaro Herrera wrote:
>> Excerpts from Boszormenyi Zoltan's message of vie jun 29 09:11:23 -0400
> 2012:
>>
>>>>> We have some use cases for this patch, when can you post
>>>>> a new version? I would test and review it.
>>
>>>> What use cases do you have in mind?
>>
>>>   Wouldn't it be helpful for some features like parallel query in future?
>
>> Trying to solve that is what delayed this patch, so the scope of this
>> needs to be "permanent daemons" rather than dynamically spawned worker
>> tasks.
>
>   Why can't worker tasks be also permanent, which can be controlled through
>   configuration. What I mean to say is that if user has need for parallel
> operations
>   he can configure max_worker_tasks and those many worker tasks will get
> created.
>   Otherwise without having such parameter, we might not be sure whether such
> deamons
>   will be of use to database users who don't need any background ops.
>
>   The dynamism will come in to scene when we need to allocate such daemons
> for particular ops(query), because
>   might be operation need certain number of worker tasks, but no such task
> is available, at that time it need
>   to be decided whether to spawn a new task or change the parallelism in
> operation such that it can be executed with
>   available number of worker tasks.
>
>   Although I understood and agree that such "permanent daemons" will be
> useful for usecases other than
>   parallel operations. However my thinking is that having "permanent
> daemons" can also be useful for parallel ops.
>   So even currently it is getting developed for certain usecases but the
> overall idea can be enhanced to have them for
>   parallel ops as well.
>
I'm also not sure why "permanent daemons" is more difficult than dynamically
spawned daemons, because I guess all the jobs needed for permanent
daemons are quite similar to what we're now doing to implement bgwriter
or others in postmaster.c, except for it is implemented with loadable modules.

Thanks,
-- 
KaiGai Kohei <kaigai@kaigai.gr.jp>



pgsql-hackers by date:

Previous
From: Alvaro Herrera
Date:
Subject: Re: [v9.3] Extra Daemons (Re: elegant and effective way for running jobs inside a database)
Next
From: Heikki Linnakangas
Date:
Subject: XLogInsert scaling, revisited