Thread: median and percentile function implementation

median and percentile function implementation

From
Pavel Stehule
Date:
Hello

Three years ago we worked on implementation of median function.

http://www.postgresql.org/message-id/AANLkTimRksUOgGsK7-gTXnvJJGL-1QvuqxidusZQwep6@mail.gmail.com


Can we implement this function now?

There are a few opened questions:

* immutable parameters of aggregate functions

* effective implementation when data are in memory

* effective implementation when data are out of memory

* effective implementation for usage as window function - running median.

Better than before, we can choose between hashing and sorting based aggregation. But tuplesort implementation is not right for using as window function. Second issue can be a not possible reusing of tuplesort for more final functions - median, percentile in one query.

Ideas how to solve this task? Can we move forward in some points?

Regards

Pavel

Re: median and percentile function implementation

From
Vik Fearing
Date:
On 08/26/2013 02:39 PM, Pavel Stehule wrote:
> Hello
>
> Three years ago we worked on implementation of median function.
>
> http://www.postgresql.org/message-id/AANLkTimRksUOgGsK7-gTXnvJJGL-1QvuqxidusZQwep6@mail.gmail.com
>
>
> Can we implement this function now?
>
> There are a few opened questions:
>
> * immutable parameters of aggregate functions
>
> * effective implementation when data are in memory
>
> * effective implementation when data are out of memory
>
> * effective implementation for usage as window function - running median.
>
> Better than before, we can choose between hashing and sorting based
> aggregation. But tuplesort implementation is not right for using as
> window function. Second issue can be a not possible reusing of
> tuplesort for more final functions - median, percentile in one query.
>
> Ideas how to solve this task? Can we move forward in some points?

Take a look at the standard-compliant WITHIN GROUP that is currently in
development for 9.4.
http://www.postgresql.org/message-id/2b8b55b8ba82f83ef4e6070b95fb0cd0@news-out.riddles.org.uk

-- 
Vik




Re: median and percentile function implementation

From
Pavel Stehule
Date:
Hello

very nice! I tested it and works very well.

And I looking for implementation in core.

Two notes:

* should to support string_agg and array_agg

Regards

Pavel



2013/8/26 Vik Fearing <vik.fearing@dalibo.com>
On 08/26/2013 02:39 PM, Pavel Stehule wrote:
> Hello
>
> Three years ago we worked on implementation of median function.
>
> http://www.postgresql.org/message-id/AANLkTimRksUOgGsK7-gTXnvJJGL-1QvuqxidusZQwep6@mail.gmail.com
>
>
> Can we implement this function now?
>
> There are a few opened questions:
>
> * immutable parameters of aggregate functions
>
> * effective implementation when data are in memory
>
> * effective implementation when data are out of memory
>
> * effective implementation for usage as window function - running median.
>
> Better than before, we can choose between hashing and sorting based
> aggregation. But tuplesort implementation is not right for using as
> window function. Second issue can be a not possible reusing of
> tuplesort for more final functions - median, percentile in one query.
>
> Ideas how to solve this task? Can we move forward in some points?

Take a look at the standard-compliant WITHIN GROUP that is currently in
development for 9.4.
http://www.postgresql.org/message-id/2b8b55b8ba82f83ef4e6070b95fb0cd0@news-out.riddles.org.uk

--
Vik


Re: median and percentile function implementation

From
Atri Sharma
Date:
On Mon, Aug 26, 2013 at 10:15 PM, Pavel Stehule <pavel.stehule@gmail.com> wrote:
> Hello
>
> very nice! I tested it and works very well.
>
> And I looking for implementation in core.
>
> Two notes:
>
> * should to support string_agg and array_agg
>
> Regards
>
> Pavel
>
>
>
> 2013/8/26 Vik Fearing <vik.fearing@dalibo.com>
>>
>> On 08/26/2013 02:39 PM, Pavel Stehule wrote:
>> > Hello
>> >
>> > Three years ago we worked on implementation of median function.
>> >
>> >
>> > http://www.postgresql.org/message-id/AANLkTimRksUOgGsK7-gTXnvJJGL-1QvuqxidusZQwep6@mail.gmail.com
>> >
>> >
>> > Can we implement this function now?
>> >
>> > There are a few opened questions:
>> >
>> > * immutable parameters of aggregate functions
>> >
>> > * effective implementation when data are in memory
>> >
>> > * effective implementation when data are out of memory
>> >
>> > * effective implementation for usage as window function - running
>> > median.
>> >
>> > Better than before, we can choose between hashing and sorting based
>> > aggregation. But tuplesort implementation is not right for using as
>> > window function. Second issue can be a not possible reusing of
>> > tuplesort for more final functions - median, percentile in one query.
>> >
>> > Ideas how to solve this task? Can we move forward in some points?
>>
>> Take a look at the standard-compliant WITHIN GROUP that is currently in
>> development for 9.4.
>>
>> http://www.postgresql.org/message-id/2b8b55b8ba82f83ef4e6070b95fb0cd0@news-out.riddles.org.uk
>>
>> --
>> Vik
>>
>

Thanks a ton!

We will definitely look at supporting at the two functions you mentioned.

Regards,

Atri


-- 
Regards,

Atri
l'apprenant