Re: Patch: Write Amplification Reduction Method (WARM) - Mailing list pgsql-hackers

From Pavan Deolasee
Subject Re: Patch: Write Amplification Reduction Method (WARM)
Date
Msg-id CABOikdOMczh4+xNE5U87TzJaoewC=9FN_1m_D4ZOZw04=mCyyw@mail.gmail.com
Whole thread Raw
In response to Re: Patch: Write Amplification Reduction Method (WARM)  (Dilip Kumar <dilipbalaut@gmail.com>)
Responses Re: Patch: Write Amplification Reduction Method (WARM)  (Robert Haas <robertmhaas@gmail.com>)
List pgsql-hackers


On Fri, Mar 31, 2017 at 6:47 PM, Robert Haas <robertmhaas@gmail.com> wrote:

2. WARM is a non-optional feature which touches the on-disk format.
There is nothing more dangerous than that.  If hash indexes have bugs,
people can avoid those bugs by not using them; there are good reasons
to suppose that hash indexes have very few existing users.  The
expression evaluation changes, IMHO, are much more dangerous because
everyone will be exposed to them, but they will not likely corrupt
your data because they don't touch the on-disk format.  WARM is even a
little more dangerous than that; everyone is exposed to those bugs,
and in the worst case they could eat your data.


Having worked on it for some time now, I can say that WARM uses pretty much the same infrastructure that HOT uses for cleanup/pruning tuples from the heap. So the risk of having a bug which can eat your data from the heap is very low. Sure, it might mess up with indexes, return duplicate keys, not return a row when it should have. Not saying they are not bad bugs, but probably much less severe than someone removing live rows from the heap.

And we can make it a table level property, keep it off by default, turn it off on system tables in this release and change the defaults only when we get more confidence assuming people use it by explicitly turning it on. Now may be that's not the right approach and keeping it off by default will mean it receives much less testing than we would like. So we can keep it on in the beta cycle and then take a call. I went a good length to make it work on system tables because during HOT development, Tom told me that it better work for everything or it doesn't work at all. But with WARM it works for system tables and I know no known bugs, but if we don't want to risk system tables, we might want to turn it off (just prior to release may be).

Thanks,
Pavan

--
 Pavan Deolasee                   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services

pgsql-hackers by date:

Previous
From: David Steele
Date:
Subject: Re: GUC for cleanup indexes threshold.
Next
From: David Steele
Date:
Subject: Re: [PATCH] Transaction traceability - txid_status(bigint)