Re: [HACKERS] Partition-wise join for join between (declaratively)partitioned tables - Mailing list pgsql-hackers

From Rajkumar Raghuwanshi
Subject Re: [HACKERS] Partition-wise join for join between (declaratively)partitioned tables
Date
Msg-id CAKcux6n24b8pEXBzMoUH6mFsCArVgWwtgv-se7+M=rTOXA7Ksg@mail.gmail.com
Whole thread Raw
In response to Re: [HACKERS] Partition-wise join for join between (declaratively)partitioned tables  (Ashutosh Bapat <ashutosh.bapat@enterprisedb.com>)
List pgsql-hackers
On Wed, Sep 20, 2017 at 3:13 PM, Ashutosh Bapat <ashutosh.bapat@enterprisedb.com> wrote:
On Wed, Sep 20, 2017 at 9:44 AM, Thomas Munro
<thomas.munro@enterprisedb.com> wrote:
> 2.  What queries in the 0008 patch are hitting lines that 0007 doesn't hit?
>
> I thought about how to answer questions like this and came up with a
> shell script that (1) makes computers run really hot for quite a long
> time and (2) tells you which blocks of SQL hit which lines of C.
> Please find attached the shell script and its output.  The .sql files
> have been annotated with "block" numbers (blocks being chunks of SQL
> stuff separated by blank lines), and the C files annotated with
> references to those block numbers where A<n> = block <n>
> partition_join.sql and B<n> = block <n> in partition_join_extras.sql.
>
> Then to find lines that B queries hit but A queries don't and know
> which particular queries hit them, you might use something like:
>
>   grep -v 'SQL blocks: .*A[0-9]' < joinpath.c.aggregated_coverage | \
>   grep 'SQL blocks: .*B[0-9]'
>

Thanks for this. It generates a lot of output (970 lines over all the
coverage files). It will take some time for getting anything
meaningful out of this. May be there's some faster way by looking at
the lines that are covered by B but not A. BTW, I checked those lines
to see if there could be any bug there. But I don't see what could go
wrong with those lines.

I have also tried to find test cases in B which hits some extra line which is not
hitting by A, with the help of results attached by Thomas in coverage.tarball_FILES.
It took lot of time but I am able to find some test cases. which if adding in partition_join.sql
increasing no of lines hit by 14. but for hitting these 14 extra line attached patch is doing
900+ line inserts in partition_join.sql and partition_join.out file.

I have used gcov-lcov to find coverage for files changed by partition-wise-join patches
with and without attached patch which is below.


with existing partition_join.sqlpartition_join.sql + some test cases of partition_join_extra.sql
Modifed FilesLine CoverageFunctionsLine CoverageFunctions
src/backend/optimizer/geqo79.4 %269/33996.6 %28/2979.4 %269/33996.6 %28/29
src/backend/optimizer/path/allpaths.c92.3 %787 / 85395.5 %42 / 4492.6 %790 / 85395.5 %42 / 44
src/backend/optimizer/path/costsize.c96.8 %1415 / 146298.4 %61 / 6296.9 %1416 / 146298.4 %61 / 62
src/backend/optimizer/path/joinpath.c95.5 %404 / 423100.0 %16 / 1695.5 %404 / 423100.0 %16 / 16
src/backend/optimizer/path/joinrels.c92.5 %422 / 456100.0 %16 / 1693.0 %424 / 456100.0 %16 / 16
src/backend/optimizer/plan/createplan.c90.9 %1928 / 212296.3 %103 / 10791.0 %1930 / 212296.3 %103 / 107
src/backend/optimizer/plan/planner.c94.9 %1609 / 169697.6 %41 / 4294.9 %1609 / 169697.6 %41 / 42
src/backend/optimizer/plan/setrefs.c91.3 %806 / 88394.3 %33 / 3591.3 %806 / 88394.3 %33 / 35
src/backend/optimizer/prep/prepunion.c95.5 %661 / 692100.0 %25 / 2595.5 %661 / 692100.0 %25 / 25
src/backend/optimizer/util/pathnode.c88.7 %1144 / 129098.1 %52 / 5388.8 %1146 / 129098.1 %52 / 53
src/backend/optimizer/util/placeholder.c96.5 %139 / 144100.0 %10 / 1096.5 %139 / 144100.0 %10 / 10
src/backend/optimizer/util/plancat.c89.0 %540 / 60794.7 %18 / 1989.6 %544 / 60794.7 %18 / 19
src/backend/optimizer/util/relnode.c95.3 %548 / 575100.0 %24 / 2495.3 %548 / 575100.0 %24 / 24
src/backend/utils/misc/guc.c67.4 %1536 / 227889.7 %113 / 12667.4 %1536 / 227889.7 %113 / 126


Thanks & Regards,
Rajkumar Raghuwanshi
QMG, EnterpriseDB Corporation




 
Attachment

pgsql-hackers by date:

Previous
From: Robert Haas
Date:
Subject: Re: [HACKERS] Re: [COMMITTERS] pgsql: Expand partitioned table RTEslevel by level, without flattening
Next
From: Robert Haas
Date:
Subject: Re: [HACKERS] Page Scan Mode in Hash Index