Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM) - Mailing list pgsql-hackers

From Pavan Deolasee
Subject Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)
Date
Msg-id CABOikdN0Vp1WUxdjSYNp5EcaEN3UQfyJZJQi8J+Of70UGgJufw@mail.gmail.com
Whole thread Raw
In response to Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)  (Robert Haas <robertmhaas@gmail.com>)
Responses Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)  (Bruce Momjian <bruce@momjian.us>)
List pgsql-hackers


On Tue, Mar 21, 2017 at 10:34 PM, Robert Haas <robertmhaas@gmail.com> wrote:
On Tue, Mar 21, 2017 at 12:49 PM, Bruce Momjian <bruce@momjian.us> wrote:
> On Tue, Mar 21, 2017 at 09:25:49AM -0400, Robert Haas wrote:
>> On Tue, Mar 21, 2017 at 8:41 AM, Pavan Deolasee
>> > TBH I see many artificial scenarios here. It will be very useful if he can
>> > rerun the query with some of these restrictions lifted. I'm all for
>> > addressing whatever we can, but I am not sure if this test demonstrates a
>> > real world usage.
>>
>> That's a very fair point, but if these patches - or some of them - are
>> going to get committed then these things need to get discussed.  Let's
>> not just have nothing-nothing-nothing giant unagreed code drop.
>
> First, let me say I love this feature for PG 10, along with
> multi-variate statistics.
>
> However, not to be a bummer on this, but the persistent question I have
> is whether we are locking ourselves into a feature that can only do
> _one_ index-change per WARM chain before a lazy vacuum is required.  Are
> we ever going to figure out how to do more changes per WARM chain in the
> future, and is our use of so many bits for this feature going to
> restrict our ability to do that in the future.
>
> I know we have talked about it, but not recently, and if everyone else
> is fine with it, I am too, but I have to ask these questions.

I think that's a good question.  I previously expressed similar
concerns.  On the one hand, it's hard to ignore the fact that, in the
cases where this wins, it already buys us a lot of performance
improvement.  On the other hand, as you say (and as I said), it eats
up a lot of bits, and that limits what we can do in the future. 

I think we can save a bit few bits, at some additional costs and/or complexity. It all depends on what matters us more. For example, we can choose not to use HEAP_LATEST_TUPLE bit and instead always find the root tuple the hard way. Since only WARM would ever need to find that information, may be it's ok since WARM's other advantage will justify that. Or we cache the information computed during earlier heap_prune_page call and use that (just guessing that we can make it work, no concrete idea at this moment).

We can also save HEAP_WARM_UPDATED flag since this is required only for abort-handling case. We can find a way to push that information down to the old tuple if UPDATE aborts and we detect the broken chain. Again, not fully thought through, but doable. Of course, we will have to carefully evaluate all code paths and make sure that we don't lose that information ever.

If the consumption of bits become a deal breaker then I would first trade the HEAP_LATEST_TUPLE bit and then HEAP_WARM_UPDATED just from correctness perspective.

Thanks,
Pavan
--
 Pavan Deolasee                   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services

pgsql-hackers by date:

Previous
From: David Steele
Date:
Subject: Re: [HACKERS] [WIP] RE: DECLARE STATEMENT setting up a connection in ECPG
Next
From: Andrew Dunstan
Date:
Subject: Re: [HACKERS] Freeze on Cygwin w/ concurrency