pgsql: Fix mishandling of NaN counts in numeric_[avg_]combine. - Mailing list pgsql-committers

From Tom Lane
Subject pgsql: Fix mishandling of NaN counts in numeric_[avg_]combine.
Date
Msg-id E1jjUv4-0002op-6W@gemulon.postgresql.org
Whole thread Raw
List pgsql-committers
Fix mishandling of NaN counts in numeric_[avg_]combine.

When merging two NumericAggStates, the code missed adding the new
state's NaNcount unless its N was also nonzero; since those counts
are independent, this is wrong.

This would only have visible effect if some partial aggregate scans
found only NaNs while earlier ones found only non-NaNs; then we could
end up falsely deciding that there were no NaNs and fail to return a
NaN final result as expected.  That's pretty improbable, so it's no
surprise this hasn't been reported from the field.  Still, it's a bug.

I didn't try to produce a regression test that would show the bug,
but I did notice that these functions weren't being reached at all
in our regression tests, so I improved the tests to at least
exercise them.  With these additions, I see pretty complete code
coverage on the aggregation-related functions in numeric.c.

Back-patch to 9.6 where this code was introduced.  (I only added
the improved test case as far back as v10, though, since the
relevant part of aggregates.sql isn't there at all in 9.6.)

Branch
------
REL_10_STABLE

Details
-------
https://git.postgresql.org/pg/commitdiff/411a4626e48576a74414d2a24e950f37909134f1

Modified Files
--------------
src/backend/utils/adt/numeric.c          | 12 ++---
src/test/regress/expected/aggregates.out | 80 +++++++++++++++++++++++++++-----
src/test/regress/sql/aggregates.sql      | 32 +++++++++++--
3 files changed, 103 insertions(+), 21 deletions(-)


pgsql-committers by date:

Previous
From: Peter Geoghegan
Date:
Subject: Re: pgsql: Rework HashAgg GUCs.
Next
From: Jeff Davis
Date:
Subject: pgsql: Fix doc build, broken by 13e0fa7a.