Re: Speed up Clog Access by increasing CLOG buffers - Mailing list pgsql-hackers

From Peter Geoghegan
Subject Re: Speed up Clog Access by increasing CLOG buffers
Date
Msg-id CAM3SWZRWJp05QvwPtuDL4xmixuYcBeq2ChVZpgwJZdZ_ncGDYQ@mail.gmail.com
Whole thread Raw
In response to Speed up Clog Access by increasing CLOG buffers  (Amit Kapila <amit.kapila16@gmail.com>)
Responses Re: Speed up Clog Access by increasing CLOG buffers  (Amit Kapila <amit.kapila16@gmail.com>)
List pgsql-hackers
On Mon, Aug 31, 2015 at 9:49 PM, Amit Kapila <amit.kapila16@gmail.com> wrote:
> Increasing CLOG buffers to 64 helps in reducing the contention due to second
> reason.  Experiments revealed that increasing CLOG buffers only helps
> once the contention around ProcArrayLock is reduced.

There has been a lot of research on bitmap compression, more or less
for the benefit of bitmap index access methods.

Simple techniques like run length encoding are effective for some
things. If the need to map the bitmap into memory to access the status
of transactions is a concern, there has been work done on that, too.
Byte-aligned bitmap compression is a technique that might offer a good
trade-off between compression clog, and decompression overhead -- I
think that there basically is no decompression overhead, because set
operations can be performed on the "compressed" representation
directly. There are other techniques, too.

Something to consider. There could be multiple benefits to compressing
clog, even beyond simply avoiding managing clog buffers.

-- 
Peter Geoghegan



pgsql-hackers by date:

Previous
From: Stephen Frost
Date:
Subject: Re: CREATE POLICY and RETURNING
Next
From: Robert Haas
Date:
Subject: Re: RFC: replace pg_stat_activity.waiting with something more descriptive