Postgres Pro
Downloads
Home   >   mailing lists

Thread: BUG #8451: quantile extension: memory corruption?

BUG #8451: quantile extension: memory corruption?

From
stephane.wustner@lip6.fr
Date:
The following bug has been logged on the website:

Bug reference:      8451
Logged by:          strexxx
Email address:      stephane.wustner@lip6.fr
PostgreSQL version: 9.1.9
Operating system:   Linux 3.8.0-27-generic #40-Ubuntu SMP x86_64
Description:

A query such as this one
COPY (SELECT e,quantile(EXTRACT(EPOCH
FROM(d)),ARRAY[0,0.1,0.2,0.3,0.4,0.5,0.6,0.7,0.8,0.9,1]) AS d FROM (SELECT
e,max(time)-min(time) AS d FROM bb RIGHT JOIN (SELECT s,e FROM ss RIGHT JOIN
(SELECT e FROM ee WHERE h ~* 'blah') AS i0 USING(e)) AS i0  USING
(sessionid) GROUP BY e,ei) AS i1 GROUP BY e) TO 'full_path/d.csv' WITH
CSV;"


 generates this
...
403,"{0.101187,0.139183,0.189162,0.271442,0.405427,0.731997,1.178445,2.52971,4.288747,9.544644,8996.364885}"

404,"{8996.364885,3.16202013338398e-322,3.16202013338398e-322,3.16202013338398e-322,3.16202013338398e-322,3.16202013338398e-322,3.16202013338398e-322,3.16202013338398e-322,3.16202013338398e-322,3.16202013338398e-322,3.16202013338398e-322}"
437,"{0.10007,0.204397,0.266877,0.48238,0.669026,0.911554,1.275652,1.656346,2.445659,3.909159,58915.291602}"

438,"{58915.291602,3.16202013338398e-322,3.16202013338398e-322,3.16202013338398e-322,3.16202013338398e-322,3.16202013338398e-322,3.16202013338398e-322,3.16202013338398e-322,3.16202013338398e-322,3.16202013338398e-322,3.16202013338398e-322}"

440,"{58915.291602,3.16202013338398e-322,3.16202013338398e-322,3.16202013338398e-322,3.16202013338398e-322,3.16202013338398e-322,3.16202013338398e-322,3.16202013338398e-322,3.16202013338398e-322,3.16202013338398e-322,3.16202013338398e-322}"


...


for row that gets corrupted, the value d, for which quantiles are computed
over, is NULL (there is no such row matching conditions and JOIN).
I would expect the quantiles to be NULL as well.


Additionally there is a strange pattern: the last value of the row preceding
a corrupted one, is repeated as the first value of the corrupted rows that
comes after.
The other values correspond to something somewhat close to 2^-1068 (why ?)
...




This subquery corresponding to one of the corrupted case works correctly
(from my understanding)
SELECT s,quantile(EXTRACT(EPOCH
FROM(d)),ARRAY[0,0.1,0.2,0.3,0.4,0.5,0.6,0.7,0.8,0.9,1]) AS d FROM (SELECT
s,max(time)-min(time) AS d FROM bb WHERE s = 440 GROUP BY s,e) AS i1 GROUP
BY s;
                  s | d
-----------+----------
(0 rows)

Re: BUG #8451: quantile extension: memory corruption?

From
Tomas Vondra
Date:
On 13.9.2013 18:07, stephane.wustner@lip6.fr wrote:
> The following bug has been logged on the website:
>
> Bug reference:      8451
> Logged by:          strexxx
> Email address:      stephane.wustner@lip6.fr
> PostgreSQL version: 9.1.9
> Operating system:   Linux 3.8.0-27-generic #40-Ubuntu SMP x86_64
> Description:
>
> A query such as this one
> COPY (SELECT e,quantile(EXTRACT(EPOCH
> FROM(d)),ARRAY[0,0.1,0.2,0.3,0.4,0.5,0.6,0.7,0.8,0.9,1]) AS d FROM (SELECT
> e,max(time)-min(time) AS d FROM bb RIGHT JOIN (SELECT s,e FROM ss RIGHT JOIN
> (SELECT e FROM ee WHERE h ~* 'blah') AS i0 USING(e)) AS i0  USING
> (sessionid) GROUP BY e,ei) AS i1 GROUP BY e) TO 'full_path/d.csv' WITH
> CSV;"

Hi,

this seems like a bug in the "quantile" extension, which is developed
and distributed completely separately from PostgreSQL. So pgsql-bugs is
not the right place to report the issue.

I reported the issue on github and I'll look into that today/tomorrow

   https://github.com/tvondra/quantile/issues/4

so far it seems like a trivial bug in handling NULL values.

regards
Tomas

Re: BUG #8451: quantile extension: memory corruption?

From
Tomas Vondra
Date:
On 14.9.2013 14:12, Tomas Vondra wrote:
> On 13.9.2013 18:07, stephane.wustner@lip6.fr wrote:
>> The following bug has been logged on the website:
>>
>> Bug reference:      8451
>> Logged by:          strexxx
>> Email address:      stephane.wustner@lip6.fr
>> PostgreSQL version: 9.1.9
>> Operating system:   Linux 3.8.0-27-generic #40-Ubuntu SMP x86_64
>> Description:
>>
>> A query such as this one
>> COPY (SELECT e,quantile(EXTRACT(EPOCH
>> FROM(d)),ARRAY[0,0.1,0.2,0.3,0.4,0.5,0.6,0.7,0.8,0.9,1]) AS d FROM (SELECT
>> e,max(time)-min(time) AS d FROM bb RIGHT JOIN (SELECT s,e FROM ss RIGHT JOIN
>> (SELECT e FROM ee WHERE h ~* 'blah') AS i0 USING(e)) AS i0  USING
>> (sessionid) GROUP BY e,ei) AS i1 GROUP BY e) TO 'full_path/d.csv' WITH
>> CSV;"
>
> Hi,
>
> this seems like a bug in the "quantile" extension, which is developed
> and distributed completely separately from PostgreSQL. So pgsql-bugs is
> not the right place to report the issue.
>
> I reported the issue on github and I'll look into that today/tomorrow
>
>    https://github.com/tvondra/quantile/issues/4
>
> so far it seems like a trivial bug in handling NULL values.

And indeed it was a bug in handling NULL values - the PG_ARGISNULL
happened too late. I plan to do few more improvements before pushing new
version to pgxn, but you may grab the sources from github directly.

Tomas