Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM) - Mailing list pgsql-hackers

From Pavan Deolasee
Subject Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)
Date
Msg-id CABOikdO3p+NgxDxHSvsZSnaX=Vz6rrOMJMK-uGN7sgmPWHqEsA@mail.gmail.com
Whole thread Raw
In response to Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)  (Alvaro Herrera <alvherre@2ndquadrant.com>)
Responses Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)  (Bruce Momjian <bruce@momjian.us>)
List pgsql-hackers


On Tue, Jan 31, 2017 at 7:21 PM, Alvaro Herrera <alvherre@2ndquadrant.com> wrote:

> > +#define HeapTupleHeaderGetNextTid(tup, next_ctid) \
> > +do { \
> > +     AssertMacro(!((tup)->t_infomask2 & HEAP_LATEST_TUPLE)); \
> > +     ItemPointerCopy(&(tup)->t_ctid, (next_ctid)); \
> > +} while (0)
>
> > Actually, I think this macro could just return the TID so that it can be
> > used as struct assignment, just like ItemPointerCopy does internally --
> > callers can do
> >        ctid = HeapTupleHeaderGetNextTid(tup);
>
> While I agree with your proposal, I wonder why we have ItemPointerCopy() in
> the first place because we freely copy TIDs as struct assignment. Is there
> a reason for that? And if there is, does it impact this specific case?

I dunno.  This macro is present in our very first commit d31084e9d1118b.
Maybe it's an artifact from the Lisp to C conversion.  Even then, we had
some cases of iptrs being copied by struct assignment, so it's not like
it didn't work.  Perhaps somebody envisioned that the internal details
could change, but that hasn't happened in two decades so why should we
worry about it now?  If somebody needs it later, it can be changed then.


May I suggest in that case that we apply the attached patch which removes all references to ItemPointerCopy and its definition as well? This will avoid confusion in future too. No issues noticed in regression tests.
 
> There is one issue that bothers me. The current implementation lacks
> ability to convert WARM chains into HOT chains. The README.WARM has some
> proposal to do that. But it requires additional free bit in tuple header
> (which we don't have) and of course, it needs to be vetted and implemented.
> If the heap ends up with many WARM tuples, then index-only-scans will
> become ineffective because index-only-scan can not skip a heap page, if it
> contains a WARM tuple. Alternate ideas/suggestions and review of the design
> are welcome!

t_infomask2 contains one last unused bit,

Umm, WARM is using 2 unused bits from t_infomask2. You mean there is another free bit after that too?
 
and we could reuse vacuum
full's bits (HEAP_MOVED_OUT, HEAP_MOVED_IN), but that will need some
thinking ahead.  Maybe now's the time to start versioning relations so
that we can ensure clusters upgraded to pg10 do not contain any of those
bits in any tuple headers.

Yeah, IIRC old VACUUM FULL was removed in 9.0, which is good 6 year old. Obviously, there still a chance that a pre-9.0 binary upgraded cluster exists and upgrades to 10. So we still need to do something about them if we reuse these bits. I'm surprised to see that we don't have any mechanism in place to clear those bits. So may be we add something to do that.

I'd some other ideas (and a patch too) to reuse bits from t_ctid.ip_pos given that offset numbers can be represented in just 13 bits, even with the maximum block size. Can look at that if it comes to finding more bits.

Thanks,
Pavan

--
 Pavan Deolasee                   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services
Attachment

pgsql-hackers by date:

Previous
From: Michael Paquier
Date:
Subject: Re: [HACKERS] ICU integration
Next
From: Michael Paquier
Date:
Subject: Re: [HACKERS] [PROPOSAL] Temporal query processing with range types