Re: Support of partial decompression for datums - Mailing list pgsql-hackers

From Ildus Kurbangaliev
Subject Re: Support of partial decompression for datums
Date
Msg-id 20151204171726.55024452@lp
Whole thread Raw
In response to Re: Support of partial decompression for datums  (Michael Paquier <michael.paquier@gmail.com>)
List pgsql-hackers
On Fri, 4 Dec 2015 22:13:58 +0900
Michael Paquier <michael.paquier@gmail.com> wrote:

> On Fri, Dec 4, 2015 at 9:47 PM, Ildus Kurbangaliev
> <i.kurbangaliev@postgrespro.ru> wrote:
> > Attached patch adds support of partial decompression for datums.
> > It will be useful in many cases when extracting part of data is
> > enough for big varlena structures.
> >
> > It is especially useful for expanded datums, because it provides
> > storage for partial results.
> >
> > I have another patch, which removes the 1 Mb limit on tsvector using
> > this feature.  
> 
> -1 for changing the shape of pglz_decompress directly and particularly
> use metadata in it. The current format of those routines is close to
> what lz4 offers in terms of compression and decompression of a string,
> let's not break that we had a time hard enough in 9.5 cycle to get
> something clean.

Metadata is not used for current code, only for case with partial
decompression.

> 
> By the way, why don't you compress the multiple chunks and store the
> related metadata at a higher level? There is no need to put that in
> pglz itself.

Yes, but this idea with chunks means that you are creating a whole
new structure, but it can't be used if you want to optimize current
structures. For example you can't just change arrays, but I think there
are places where the partial decompression can be used.

-- 
Ildus Kurbangaliev
Postgres Professional: http://www.postgrespro.com
Russian Postgres Company



pgsql-hackers by date:

Previous
From: Teodor Sigaev
Date:
Subject: Re: Some questions about the array.
Next
From: Kevin Grittner
Date:
Subject: Re: atomic reads & writes (with no barriers)