Thread: Re: [PATCHES] Reduce heap tuple header size
OK, we need to vote on this patch. It reduces the tuple header by 4 bytes (11% decrease). If we apply it, we will not be able to easily use pg_upgrade for 7.3 because the on-disk table format will change. Votes are: 1) Apply it now 2) Wait until August and see if any other table format changes are made. 3) Delay patch until we have other table format changes. --------------------------------------------------------------------------- Manfred Koizar wrote: > On Fri, 14 Jun 2002 10:16:22 -0400, Tom Lane <tgl@sss.pgh.pa.us> wrote: > >As I commented before, I am not in favor of this. I don't think that a > >four-byte savings justifies a forced initdb with no chance of > >pg_upgrade, > > As I don't know other users' preferences, I cannot comment on this. > I just think that four bytes per tuple can amount to a sum that justifies > this effort. Disk space is not my argument here, but reduced disk IO is. > > >plus significantly slower access to > >several critical header fields. The tqual.c routines are already > >hotspots in many scenarios. I believe this will make them noticeably > >slower. > > Significantly slower? I tried to analyze HeapTupleSatisfiesUpdate(), > as I think it is the most complicated of these Satisfies functions > (look for the !! comments): > > > So in the worst case we return after having done four more > compares than without the patch. Note that in the most common > cases there is no additional cost at all. If you still think > we have a performance problem here, we could replace GetCmin > and GetCmax by cheaper macros: > > #define HeapTupleHeaderGetCminKnowingThatNotMoved(tup) \ > ( \ > AssertMacro(!((tup)->t_infomask & HEAP_MOVED)), > ( \ > ((tup)->t_infomask & (HEAP_XMIN_IS_XMAX | HEAP_XMAX_INVALID)) ? \ > (CommandId) (tup)->t_cid \ > : \ > FirstCommandId \ > ) \ > ) > > thus reducing the additional cost to one t_infomask compare, > because the Satisfies functions only access Cmin and Cmax, > when HEAP_MOVED is known to be not set. > > OTOH experimenting with a moderatly sized "out of production" > database I got the following results: > | pages | pages | > relkind | count | tuples | before| after | savings > --------+-------+--------+-------+-------+-------- > i | 31 | 808146 | 8330 | 8330 | 0.00% > r | 32 | 612968 | 13572 | 13184 | 2.86% > all | 63 | | 21902 | 21514 | 1.77% > > 2.86% fewer heap pages mean 2.86% less disk IO caused by heap pages. > Considering that index pages tend to benefit more from caching > we conclude that heap pages contribute more to the overall > IO load, so the total savings in the number of disk IOs should > be better than the 1.77% shown in the table above. I think > this outweighs a few CPU cycles now and then. > > Servus > Manfred > > ---------------------------(end of broadcast)--------------------------- > TIP 2: you can get off all lists at once with the unregister command > (send "unregister YourEmailAddressHere" to majordomo@postgresql.org) > -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 853-3000+ If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup. | Drexel Hill, Pennsylvania19026
On Fri, Jun 28, 2002 at 07:32:00PM -0400, Bruce Momjian wrote: > OK, we need to vote on this patch. It reduces the tuple header by 4 > bytes (11% decrease). > > If we apply it, we will not be able to easily use pg_upgrade for 7.3 > because the on-disk table format will change. I vote for apply it now -- if there are no other on-disk format changes made by late August and pg_upgrade is actually a valid, production-quality upgrade mechanism (which I'm not prepared to assume), consider reverting it. Cheers, Neil -- Neil Conway <neilconway@rogers.com> PGP Key ID: DB3C29FC
On Fri, 28 Jun 2002, Bruce Momjian wrote: > OK, we need to vote on this patch. It reduces the tuple header by 4 > bytes (11% decrease). > > If we apply it, we will not be able to easily use pg_upgrade for 7.3 > because the on-disk table format will change. > > Votes are: > > 1) Apply it now > 2) Wait until August and see if any other table format changes are made. > 3) Delay patch until we have other table format changes. I would tend to say "apply it now" so that we can get more testing of it. It would also be good to see how else we could save space in the header, e.g., by not having an empty OID field when a table is created without OIDs. (That would double the space savings.) I tend to use ID cross reference tables quite a lot, and these tend to have a lot of rows in them. (E.g., group table has group ID; user table has user-id; a group-id + user-id table determines which users are in which groups. In one project a couple of years ago, such a table was 85 million rows.) These types of tables are typically 8 bytes of data and 40 or so bytes of overhead. Ouch! cjs -- Curt Sampson <cjs@cynic.net> +81 90 7737 2974 http://www.netbsd.org Don't you know, in this new Dark Age, we're alllight. --XTC
Curt Sampson wrote: > On Fri, 28 Jun 2002, Bruce Momjian wrote: > > > OK, we need to vote on this patch. It reduces the tuple header by 4 > > bytes (11% decrease). > > > > If we apply it, we will not be able to easily use pg_upgrade for 7.3 > > because the on-disk table format will change. > > > > Votes are: > > > > 1) Apply it now > > 2) Wait until August and see if any other table format changes are made. > > 3) Delay patch until we have other table format changes. > > I would tend to say "apply it now" so that we can get more testing > of it. OK, I have heard enough votes to add this. If more votes come in while it is in the queue, we can reevaluate. Also, Manfred is working on making the OID field optional, so it seems we may have more format changes coming. Time to focus on any other data format changes we want to be in 7.3. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 853-3000+ If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup. | Drexel Hill, Pennsylvania19026
On Mon, 1 Jul 2002 10:15:42 -0400 (EDT), Bruce Momjian <pgman@candle.pha.pa.us> wrote: >OK, I have heard enough votes to add this. In a second version of this patch posted on 2002-06-26 you can control the tuple format by #define/#undef PG72FORMAT. While there have been voices saying that exposing this choice to the user viaconfigure --enable-pg72format is not a good idea [well, it was one voice and the idea was called "really, really bad" ;-) but the argument is still valid], I wonder whether we shouldn't apply this second version (without the configure parts) and put all forthcoming format changes under #ifndef PG72FORMAT. This way you can decide to go back to v7.2 format immediately before going beta, if the changes are too hot to handle. And I think if I wouldn't volunteer to cleanup the #ifdef PG72FORMAT stuff after the new format has been accepted, I would be nominated to do it ... ServusManfred
Manfred Koizar <mkoi-pg@aon.at> writes:
> ... I wonder
> whether we shouldn't apply this second version (without the configure
> parts) and put all forthcoming format changes under #ifndef
> PG72FORMAT.
Seems reasonable. I generally dislike #ifdef clutter, but the #ifs
would only be around a couple of macro definitions AFAICT, so the
readability hit would be minimal. And someone who wanted
back-compatibility would be able to have it, whichever way we jump
on the decision for 7.3.
At the rate Manfred is going, he'll have patches for all the tuple and
page header related issues before August anyway ... so my original gripe
about wanting to group those changes into a single release will become
moot ;-). I certainly have no objection to doing them all in 7.3
instead of 7.4 if we can get it done.
regards, tom lane
Tom Lane wrote: > Manfred Koizar <mkoi-pg@aon.at> writes: > > ... I wonder > > whether we shouldn't apply this second version (without the configure > > parts) and put all forthcoming format changes under #ifndef > > PG72FORMAT. > > Seems reasonable. I generally dislike #ifdef clutter, but the #ifs > would only be around a couple of macro definitions AFAICT, so the > readability hit would be minimal. And someone who wanted > back-compatibility would be able to have it, whichever way we jump > on the decision for 7.3. I committed the version with no #ifdef's. If we need them, we can add them later, but it is likely we will never need them. > At the rate Manfred is going, he'll have patches for all the tuple and > page header related issues before August anyway ... so my original gripe > about wanting to group those changes into a single release will become > moot ;-). I certainly have no objection to doing them all in 7.3 > instead of 7.4 if we can get it done. Yes. Manfred, keep going. ;-) -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 853-3000+ If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup. | Drexel Hill, Pennsylvania19026
On Tue, 2 Jul 2002 02:16:29 -0400 (EDT), Bruce Momjian <pgman@candle.pha.pa.us> wrote: >I committed the version with no #ifdef's. If we need them, we can add >them later, but it is likely we will never need them. My point was, if there is a need to fallback to v7.2 format, it can be done by changing a single line from #undef to #define. IMO the next patch I'm going to submit is a bit more risky. But if everyone else is confident we can make it stable for v7.3, it's fine by me too. >Yes. Manfred, keep going. ;-) Can't guarantee to keep the rate. You know, the kids need a bit more attention when they don't go to school :-) ServusManfred
Manfred Koizar wrote: > On Tue, 2 Jul 2002 02:16:29 -0400 (EDT), Bruce Momjian > <pgman@candle.pha.pa.us> wrote: > >I committed the version with no #ifdef's. If we need them, we can add > >them later, but it is likely we will never need them. > > My point was, if there is a need to fallback to v7.2 format, it can be > done by changing a single line from #undef to #define. IMO the next > patch I'm going to submit is a bit more risky. But if everyone else > is confident we can make it stable for v7.3, it's fine by me too. Yes, with your recent pages, I think we are committed to changing the format for 7.3. > >Yes. Manfred, keep going. ;-) > > Can't guarantee to keep the rate. You know, the kids need a bit more > attention when they don't go to school :-) Let me send over my kids. Where are you located? Austria? Hmmm... -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 853-3000+ If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup. | Drexel Hill, Pennsylvania19026