Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM) - Mailing list pgsql-hackers

From Pavan Deolasee
Subject Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)
Date
Msg-id CABOikdPgEqSPQcEJajnLprgZ1ma+m4RHu2trqjneQ8QRWzWBBg@mail.gmail.com
Whole thread Raw
In response to Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)  (Amit Kapila <amit.kapila16@gmail.com>)
Responses Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)  (Amit Kapila <amit.kapila16@gmail.com>)
List pgsql-hackers


On Thu, Mar 23, 2017 at 7:53 PM, Amit Kapila <amit.kapila16@gmail.com> wrote:
On Thu, Mar 23, 2017 at 3:44 PM, Pavan Deolasee

>
> Yes, this is a very fair point. The way I proposed to address this upthread
> is by introducing a set of threshold/scale GUCs specific to WARM. So users
> can control when to invoke WARM cleanup. Only if the WARM cleanup is
> required, we do 2 index scans. Otherwise vacuum will work the way it works
> today without any additional overhead.
>

I am not sure on what basis user can set such parameters, it will be
quite difficult to tune those parameters.  I think the point is
whatever threshold we keep, once that is crossed, it will perform two
scans for all the indexes.

Well, that applies to even vacuum parameters, no? The general sense I've got here is that we're ok to push some work in background if it helps the real-time queries, and I kinda agree with that. If WARM improves things in a significant manner even with these additional maintenance work, it's still worth doing.

Having said that, I see many ways we can improve on this later. Like we can track somewhere else information about tuples which may have received WARM updates (I think it will need to be a per-index bitmap or so) and use that to do WARM chain conversion in a single index pass. But this is clearly not PG 10 material.
 
  IIUC, this conversion of WARM chains is
required so that future updates can be WARM or is there any other
reason?  I see this as a big penalty for future updates.

It's also necessary for index-only-scans. But I don't see this as a big penalty for future updates, because if there are indeed significant WARM updates then not preparing for future updates will result in write-amplification, something we are trying to solve here and something which seems to be showing good gains.

Thanks,
Pavan

--
 Pavan Deolasee                   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services

pgsql-hackers by date:

Previous
From: Andres Freund
Date:
Subject: Re: [HACKERS] Hash support for grouping sets
Next
From: Alvaro Herrera
Date:
Subject: Re: [HACKERS] [PATCH] Move all am-related reloption code intosrc/backend/access/[am-name] and get rid of relopt_kind for custom AM