Re: TODO : Allow parallel cores to be used by vacuumdb [ WIP ] - Mailing list pgsql-hackers

From Dilip kumar
Subject Re: TODO : Allow parallel cores to be used by vacuumdb [ WIP ]
Date
Msg-id 4205E661176A124FAF891E0A6BA9135266343797@szxeml509-mbs.china.huawei.com
Whole thread Raw
In response to Re: TODO : Allow parallel cores to be used by vacuumdb [ WIP ]  (Magnus Hagander <magnus@hagander.net>)
Responses Re: TODO : Allow parallel cores to be used by vacuumdb [ WIP ]
List pgsql-hackers
On 15 July 2014 19:01, Magnus Hagander Wrote,
 
> I am late to this game, but the first thing to my mind was - do we
> really need the whole forking/threading thing on the client at all? We
> need it for things like pg_dump/pg_restore because they can themselvse
> benefit from parallelism at the client level, but for something like
> this, might the code become a lot simpler if we just use multiple
> database connections and async queries? That would also bring the
> benefit of less platform dependent code, less cleanup needs etc.

Thanks for the review, I understand you point, but I think if we have do this directly by independent connection, 
It's difficult to equally divide the jobs b/w multiple independent connections.

As per this implementation we are able to share the load b/w the processes quite well,
1. If one process finishes the work faster it can take the other load.
2. Specially while vacuuming whole database, it's very difficult to divide the load if they are not centralized
control.

By above points, I think that we can have this patch.. 

> 
> (Oh, and for some reason at my quick review i also noticed - you added
> quoting of the table name, but forgot to do it for the schema name.
> You should probably also look at using something like
> quote_identifier(), that'll make things easier).

Thanks for the comments, I have attached the updated patch.

vacuumdb_parallel_refactor --> No change
vacuumdb_parallel_v9  --> Quotes added for namespace


Thanks & Regards,
Dilip

Attachment

pgsql-hackers by date:

Previous
From: Robert Haas
Date:
Subject: Re: things I learned from working on memory allocation
Next
From: Noah Misch
Date:
Subject: Re: Deadlocks in HS (on 9.0 :( )