Re: Improve output of BitmapAnd EXPLAIN ANALYZE - Mailing list pgsql-hackers

From Jim Nasby
Subject Re: Improve output of BitmapAnd EXPLAIN ANALYZE
Date
Msg-id 07d8298a-0fe4-e302-1a44-aa2cf0e30288@BlueTreble.com
Whole thread Raw
In response to Re: Improve output of BitmapAnd EXPLAIN ANALYZE  (Stephen Frost <sfrost@snowman.net>)
Responses Re: Improve output of BitmapAnd EXPLAIN ANALYZE  (Stephen Frost <sfrost@snowman.net>)
List pgsql-hackers
On 10/21/16 8:21 AM, Stephen Frost wrote:
> Counting each page as the relation's average number of tuples per page
> seems entirely reasonable to me, for what that is trying to report.

My concern is that still leaves a lot of room for confusion when 
interpreting EXPLAIN ANALYZE. Every other node will tell you exactly 
what happened and it's pretty easy to reason about whether rows should 
have gone up or down based on the type of node. You can't do that for 
Bitmap(And|Or) unless you know the details of how TIDBitmaps work. 
Reporting N/A makes it crystal clear that these nodes operate very 
differently than all the others.

(On a related note, it would also be nice if we reported fractional rows 
when the row count low and loops is high.)

> That said, I'm a big fan of how we have more detail for things like a
> HashJoin (buckets, batches, memory usage) and it might be nice to have
> more information like that for a BitmapAnd (and friends).  In
> particular, I'm thinking of memory usage, exact vs. lossy pages, etc.
> Knowing that the bitmap has gotten to the point of being lossy might
> indicate that a user could up work_mem, for example, and possibly avoid
> recheck costs.

I think that's the best way to handle this: report N/A in the header and 
then provide details on exact vs lossy. That provides a clear indication 
to users that these kinds of nodes are special, as well as a reminder as 
to why they're special. Certainly the node could report an exact 
rowcount in the header if there were no lossy pages too.
-- 
Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX
Experts in Analytics, Data Architecture and PostgreSQL
Data in Trouble? Get it in Treble! http://BlueTreble.com
855-TREBLE2 (855-873-2532)   mobile: 512-569-9461



pgsql-hackers by date:

Previous
From: Tom Lane
Date:
Subject: Re: Aggregate Push Down - Performing aggregation on foreign server
Next
From: Robert Haas
Date:
Subject: Re: Aggregate Push Down - Performing aggregation on foreign server