Re: parallel joins, and better parallel explain - Mailing list pgsql-hackers

From Amit Kapila
Subject Re: parallel joins, and better parallel explain
Date
Msg-id CAA4eK1L=RQbH3zvHhKTNjbRdy9XQzC+x77BdTQB9JXkbn73hPg@mail.gmail.com
Whole thread Raw
In response to Re: parallel joins, and better parallel explain  (Robert Haas <robertmhaas@gmail.com>)
List pgsql-hackers
On Wed, Dec 9, 2015 at 11:51 PM, Robert Haas <robertmhaas@gmail.com> wrote:
>
> On Fri, Dec 4, 2015 at 3:07 AM, Amit Kapila <amit.kapila16@gmail.com> wrote:
>
> > I think the problem is at Gather node, the number of buffers (read + hit)
> > are greater than the number of pages in relation.  The reason why it
> > is doing so is that in Workers (ParallelQueryMain()), it starts the buffer
> > usage accumulation before ExecutorStart() whereas in master backend
> > it always calculate it for ExecutorRun() phase, on changing it to accumulate
> > for ExecutorRun() phase above problem is fixed.  Attached patch fixes the
> > problem.
>
> Why is it a bad thing to capture the cost of doing ExecutorStart() in
> the worker?  I can see there's an argument that changing this would be
> more consistent, but I'm not totally convinced.  The overhead of
> ExecutorStart() in the leader isn't attributable to any specific
> worker, but the overhead of ExecutorStart() in the worker can fairly
> be blamed on Gather, I think.
>

This boils down to the question why currently buffer usage or other
similar stats (time for node execution) for a node doesn't include
the cost for ExecutorStart().  I think the reason is that ExecutorStart()
does some other miscellaneous works like accessing system tables
or initialization of nodes which we might not even execute, so
accumulating the cost for such work doesn't seems to be meaningful.
Looking at references of InstrStartNode() which actually marks the
beginning of buffer usage stats, it is clear that buffer usage stats are
not counted for ExecutorStart() phase, so I think following the same
for worker stats seems to be more accurate.


With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com

pgsql-hackers by date:

Previous
From: amul sul
Date:
Subject: Re: Passing initially_valid values instead of !skip_validation to StoreRelCheck() in AddRelationNewConstraints()
Next
From: Amit Kapila
Date:
Subject: Re: pg_hba_lookup function to get all matching pg_hba.conf entries