Re: [HACKERS] Custom compression methods - Mailing list pgsql-hackers

From Dilip Kumar
Subject Re: [HACKERS] Custom compression methods
Date
Msg-id CAFiTN-vwVHdGi36qoF_ZRW5ehD+ea642XH=+EBk50Qc4V1h1TA@mail.gmail.com
Whole thread Raw
In response to Re: [HACKERS] Custom compression methods  (Dilip Kumar <dilipbalaut@gmail.com>)
Responses Re: [HACKERS] Custom compression methods  (Robert Haas <robertmhaas@gmail.com>)
List pgsql-hackers
On Sun, Nov 8, 2020 at 4:29 PM Dilip Kumar <dilipbalaut@gmail.com> wrote:
>
> On Thu, Oct 29, 2020 at 12:07 PM Dilip Kumar <dilipbalaut@gmail.com> wrote:
> >
> > On Thu, Oct 29, 2020 at 12:31 AM Tomas Vondra
> > <tomas.vondra@2ndquadrant.com> wrote:
> > >
> > > On Wed, Oct 28, 2020 at 01:16:31PM +0530, Dilip Kumar wrote:
> > > >>
> > > >> ...
> > > >>
> > > >> I have added the next patch to support the compression options.  I am
> > > >> storing the compression options only for the latest compression
> > > >> method.  Basically, based on this design we would be able to support
> > > >> the options which are used only for compressions.  As of now, the
> > > >> compression option infrastructure is added and the compression options
> > > >> for inbuilt method pglz and the external method zlib are added.  Next,
> > > >> I will work on adding the options for the lz4 method.
> > > >
> > > >In the attached patch set I have also included the compression option
> > > >support for lz4.  As of now, I have only supported the acceleration
> > > >for LZ4_compress_fast.  There is also support for the dictionary-based
> > > >compression but if we try to support that then we will need the
> > > >dictionary for decompression also.  Since we are only keeping the
> > > >options for the current compression methods, we can not support
> > > >dictionary-based options as of now.
> > > >
> > >
> > > OK, thanks. Do you have any other plans to improve this patch series? I
> > > plan to do some testing and review, but if you're likely to post another
> > > version soon then I'd wait a bit.
> >
> > There was some issue in create_compression_method.sgml and the
> > drop_compression_method.sgml was missing.  I have fixed that in the
> > attached patch.  Now I am not planning to change anything soon so you
> > can review.  Thanks.
>
> The patches were not applying on the current head so I have re-based them.

There were a few problems in this rebased version, basically, the
compression options were not passed while compressing values from the
brin_form_tuple, so I have fixed this.

-- 
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com

Attachment

pgsql-hackers by date:

Previous
From: Stephen Frost
Date:
Subject: Re: Disable WAL logging to speed up data loading
Next
From: Alvaro Herrera
Date:
Subject: Re: proposal: possibility to read dumped table's name from file