Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM) - Mailing list pgsql-hackers
| From | Pavan Deolasee |
|---|---|
| Subject | Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM) |
| Date | |
| Msg-id | CABOikdOe9UKT+QfZd_mzKrNb60hvjYK5z+8kbahPNEvjy9TnxQ@mail.gmail.com Whole thread Raw |
| In response to | Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM) (Robert Haas <robertmhaas@gmail.com>) |
| Responses |
Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)
(Amit Kapila <amit.kapila16@gmail.com>)
Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM) (Robert Haas <robertmhaas@gmail.com>) |
| List | pgsql-hackers |
On Tue, Apr 11, 2017 at 7:10 PM, Robert Haas <robertmhaas@gmail.com> wrote:
Yes, and as Andres says, you don't help with those, and then you're
upset when your own patch doesn't get attention.
I am not upset, I was obviously a bit disappointed which I think is a very natural emotion after spending weeks on it. I am not blaming any one individual (excluding myself) for that and neither the community at large for the outcome. And I've moved on. I know everyone is busy getting the release ready and I see no point discussing this endlessly. We have enough on our plates for next few weeks.
Amit and others who have started to
dig into this patch a little bit found real problems pretty quickly
when they started digging.
And I fixed them as quickly as humanly possible.
Those problems should be addressed, and
review should continue (from whatever source) until no more problems
can be found.
Absolutely.
The patch may
or may not have any data-corrupting bugs, but regressions have been
found and not addressed.
1. Improved the interesting attrs patch that Alvaro wrote to address the regression discovered in fetching more heap attributes. The patch that got committed in fact improved certain synthetic workloads over then master.
2. Based on Petr and your feedback, disabled WARM on toasted attributes to reduce overhead of fetching/decompressing the attributes.
3. Added code to avoid doing second index scan when the index does not contain any WARM pointers. This should address the situation Amit brought up where only one of the indexes receive WARM inserts.
4. Added code to kill wrong index pointers to do online cleanup.
5. Added code to set a CLEAR pointer to a WARM pointer when we know that the entire chain is WARM. This should address the workload Dilip ran and found regression (I don't think he got chance to confirm that)
--
6. Enhanced stats collector to collect information about candidate WARM chains and added mechanism to control WARM cleanup at the heap as well as index level, based on configurable parameters. This gives user better control over the additional work that is required for WARM cleanup.
7. Added table level option to disable WARM if nothing else works.
8. Added mechanism to disable WARM when more than 50% indexes are being updated. I ran some benchmarks with different percentage of indexes getting updated and thought this is a good threshold.
I may have missed something, but there is no intention to ignore known regressions/reviews. Of course, I don't think that every regression will be solvable, like if you run a CPU-bound workload, setting it up in a way such that you repeatedly exercise the area where WARM is doing additional work, without providing any benefit, may be you can still find regression. I am willing to fix them as long as they are fixable and we are comfortable with the additional code complexity. IMHO certain trade-offs are good, but I understand that not everybody will agree with my views and that's ok.
Thanks,
Pavan
Pavan Deolasee http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
PostgreSQL Development, 24x7 Support, Training & Services
pgsql-hackers by date: