Thread: Re: [PATCHES] Reduce heap tuple header size

Re: [PATCHES] Reduce heap tuple header size

From
Bruce Momjian
Date:
OK, we need to vote on this patch.  It reduces the tuple header by 4
bytes (11% decrease).

If we apply it, we will not be able to easily use pg_upgrade for 7.3
because the on-disk table format will change.

Votes are:

1) Apply it now
2) Wait until August and see if any other table format changes are made.
3) Delay patch until we have other table format changes.


---------------------------------------------------------------------------

Manfred Koizar wrote:
> On Fri, 14 Jun 2002 10:16:22 -0400, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> >As I commented before, I am not in favor of this.  I don't think that a
> >four-byte savings justifies a forced initdb with no chance of
> >pg_upgrade,
> 
> As I don't know other users' preferences, I cannot comment on this.
> I just think that four bytes per tuple can amount to a sum that justifies
> this effort.  Disk space is not my argument here, but reduced disk IO is.
> 
> >plus significantly slower access to
> >several critical header fields.  The tqual.c routines are already
> >hotspots in many scenarios.  I believe this will make them noticeably
> >slower.
> 
> Significantly slower?  I tried to analyze HeapTupleSatisfiesUpdate(),
> as I think it is the most complicated of these Satisfies functions
> (look for the !! comments):
> 
> 
> So in the worst case we return after having done four more
> compares than without the patch.  Note that in the most common
> cases there is no additional cost at all.  If you still think
> we have a performance problem here, we could replace GetCmin
> and GetCmax by cheaper macros:
> 
> #define HeapTupleHeaderGetCminKnowingThatNotMoved(tup) \
> ( \
>     AssertMacro(!((tup)->t_infomask & HEAP_MOVED)),
>     ( \
>         ((tup)->t_infomask & (HEAP_XMIN_IS_XMAX | HEAP_XMAX_INVALID)) ? \
>             (CommandId) (tup)->t_cid \
>         : \
>             FirstCommandId \
>     ) \
> )
> 
> thus reducing the additional cost to one t_infomask compare,
> because the Satisfies functions only access Cmin and Cmax,
> when HEAP_MOVED is known to be not set.
> 
> OTOH experimenting with a moderatly sized "out of production"
> database I got the following results:
>                          | pages | pages |
> relkind | count | tuples | before| after | savings
> --------+-------+--------+-------+-------+--------
> i       |    31 | 808146 |  8330 |  8330 |   0.00%
> r       |    32 | 612968 | 13572 | 13184 |   2.86%
> all     |    63 |        | 21902 | 21514 |   1.77%
> 
> 2.86% fewer heap pages mean 2.86% less disk IO caused by heap pages.
> Considering that index pages tend to benefit more from caching
> we conclude that heap pages contribute more to the overall
> IO load, so the total savings in the number of disk IOs should
> be better than the 1.77% shown in the table above.  I think
> this outweighs a few CPU cycles now and then.
> 
> Servus
>  Manfred
> 
> ---------------------------(end of broadcast)---------------------------
> TIP 2: you can get off all lists at once with the unregister command
>     (send "unregister YourEmailAddressHere" to majordomo@postgresql.org)
> 

--  Bruce Momjian                        |  http://candle.pha.pa.us pgman@candle.pha.pa.us               |  (610)
853-3000+  If your life is a hard drive,     |  830 Blythe Avenue +  Christ can be your backup.        |  Drexel Hill,
Pennsylvania19026
 




Re: [PATCHES] Reduce heap tuple header size

From
nconway@klamath.dyndns.org (Neil Conway)
Date:
On Fri, Jun 28, 2002 at 07:32:00PM -0400, Bruce Momjian wrote:
> OK, we need to vote on this patch.  It reduces the tuple header by 4
> bytes (11% decrease).
> 
> If we apply it, we will not be able to easily use pg_upgrade for 7.3
> because the on-disk table format will change.

I vote for apply it now -- if there are no other on-disk format changes
made by late August and pg_upgrade is actually a valid, production-quality
upgrade mechanism (which I'm not prepared to assume), consider
reverting it.

Cheers,

Neil

-- 
Neil Conway <neilconway@rogers.com>
PGP Key ID: DB3C29FC




Re: [PATCHES] Reduce heap tuple header size

From
Curt Sampson
Date:
On Fri, 28 Jun 2002, Bruce Momjian wrote:

> OK, we need to vote on this patch.  It reduces the tuple header by 4
> bytes (11% decrease).
>
> If we apply it, we will not be able to easily use pg_upgrade for 7.3
> because the on-disk table format will change.
>
> Votes are:
>
> 1) Apply it now
> 2) Wait until August and see if any other table format changes are made.
> 3) Delay patch until we have other table format changes.

I would tend to say "apply it now" so that we can get more testing
of it.

It would also be good to see how else we could save space in the
header, e.g., by not having an empty OID field when a table is
created without OIDs. (That would double the space savings.)

I tend to use ID cross reference tables quite a lot, and these tend to
have a lot of rows in them. (E.g., group table has group ID; user table
has user-id; a group-id + user-id table determines which users are in
which groups. In one project a couple of years ago, such a table was 85
million rows.) These types of tables are typically 8 bytes of data and
40 or so bytes of overhead. Ouch!

cjs
-- 
Curt Sampson  <cjs@cynic.net>   +81 90 7737 2974   http://www.netbsd.org   Don't you know, in this new Dark Age, we're
alllight.  --XTC
 





Re: [PATCHES] Reduce heap tuple header size

From
Bruce Momjian
Date:
Curt Sampson wrote:
> On Fri, 28 Jun 2002, Bruce Momjian wrote:
> 
> > OK, we need to vote on this patch.  It reduces the tuple header by 4
> > bytes (11% decrease).
> >
> > If we apply it, we will not be able to easily use pg_upgrade for 7.3
> > because the on-disk table format will change.
> >
> > Votes are:
> >
> > 1) Apply it now
> > 2) Wait until August and see if any other table format changes are made.
> > 3) Delay patch until we have other table format changes.
> 
> I would tend to say "apply it now" so that we can get more testing
> of it.

OK, I have heard enough votes to add this.  If more votes come in while
it is in the queue, we can reevaluate.

Also, Manfred is working on making the OID field optional, so it seems
we may have more format changes coming.  Time to focus on any other data
format changes we want to be in 7.3.

--  Bruce Momjian                        |  http://candle.pha.pa.us pgman@candle.pha.pa.us               |  (610)
853-3000+  If your life is a hard drive,     |  830 Blythe Avenue +  Christ can be your backup.        |  Drexel Hill,
Pennsylvania19026
 




Re: [PATCHES] Reduce heap tuple header size

From
Manfred Koizar
Date:
On Mon, 1 Jul 2002 10:15:42 -0400 (EDT), Bruce Momjian
<pgman@candle.pha.pa.us> wrote:
>OK, I have heard enough votes to add this.

In a second version of this patch posted on 2002-06-26 you can control
the tuple format by #define/#undef PG72FORMAT.  While there have been
voices saying that exposing this choice to the user viaconfigure --enable-pg72format
is not a good idea [well, it was one voice and the idea was called
"really, really bad" ;-) but the argument is still valid], I wonder
whether we shouldn't apply this second version (without the configure
parts) and put all forthcoming format changes under #ifndef
PG72FORMAT.

This way you can decide to go back to v7.2 format immediately before
going beta, if the changes are too hot to handle.  And I think if I
wouldn't volunteer to cleanup the #ifdef PG72FORMAT stuff after the
new format has been accepted, I would be nominated to do it ...

ServusManfred




Re: [PATCHES] Reduce heap tuple header size

From
Tom Lane
Date:
Manfred Koizar <mkoi-pg@aon.at> writes:
> ... I wonder
> whether we shouldn't apply this second version (without the configure
> parts) and put all forthcoming format changes under #ifndef
> PG72FORMAT.

Seems reasonable.  I generally dislike #ifdef clutter, but the #ifs
would only be around a couple of macro definitions AFAICT, so the
readability hit would be minimal.  And someone who wanted
back-compatibility would be able to have it, whichever way we jump
on the decision for 7.3.

At the rate Manfred is going, he'll have patches for all the tuple and
page header related issues before August anyway ... so my original gripe
about wanting to group those changes into a single release will become
moot ;-).  I certainly have no objection to doing them all in 7.3
instead of 7.4 if we can get it done.
        regards, tom lane




Re: [PATCHES] Reduce heap tuple header size

From
Bruce Momjian
Date:
Tom Lane wrote:
> Manfred Koizar <mkoi-pg@aon.at> writes:
> > ... I wonder
> > whether we shouldn't apply this second version (without the configure
> > parts) and put all forthcoming format changes under #ifndef
> > PG72FORMAT.
> 
> Seems reasonable.  I generally dislike #ifdef clutter, but the #ifs
> would only be around a couple of macro definitions AFAICT, so the
> readability hit would be minimal.  And someone who wanted
> back-compatibility would be able to have it, whichever way we jump
> on the decision for 7.3.

I committed the version with no #ifdef's.  If we need them, we can add
them later, but it is likely we will never need them.

> At the rate Manfred is going, he'll have patches for all the tuple and
> page header related issues before August anyway ... so my original gripe
> about wanting to group those changes into a single release will become
> moot ;-).  I certainly have no objection to doing them all in 7.3
> instead of 7.4 if we can get it done.

Yes.  Manfred, keep going.  ;-)

--  Bruce Momjian                        |  http://candle.pha.pa.us pgman@candle.pha.pa.us               |  (610)
853-3000+  If your life is a hard drive,     |  830 Blythe Avenue +  Christ can be your backup.        |  Drexel Hill,
Pennsylvania19026
 




Re: [PATCHES] Reduce heap tuple header size

From
Manfred Koizar
Date:
On Tue, 2 Jul 2002 02:16:29 -0400 (EDT), Bruce Momjian
<pgman@candle.pha.pa.us> wrote:
>I committed the version with no #ifdef's.  If we need them, we can add
>them later, but it is likely we will never need them.

My point was, if there is a need to fallback to v7.2 format, it can be
done by changing a single line from #undef to #define.  IMO the next
patch I'm going to submit is a bit more risky.  But if everyone else
is confident we can make it stable for v7.3, it's fine by me too.

>Yes.  Manfred, keep going.  ;-)

Can't guarantee to keep the rate.  You know, the kids need a bit more
attention when they don't go to school :-)

ServusManfred




Re: [PATCHES] Reduce heap tuple header size

From
Bruce Momjian
Date:
Manfred Koizar wrote:
> On Tue, 2 Jul 2002 02:16:29 -0400 (EDT), Bruce Momjian
> <pgman@candle.pha.pa.us> wrote:
> >I committed the version with no #ifdef's.  If we need them, we can add
> >them later, but it is likely we will never need them.
> 
> My point was, if there is a need to fallback to v7.2 format, it can be
> done by changing a single line from #undef to #define.  IMO the next
> patch I'm going to submit is a bit more risky.  But if everyone else
> is confident we can make it stable for v7.3, it's fine by me too.

Yes, with your recent pages, I think we are committed to changing the
format for 7.3.

> >Yes.  Manfred, keep going.  ;-)
> 
> Can't guarantee to keep the rate.  You know, the kids need a bit more
> attention when they don't go to school :-)

Let me send over my kids.  Where are you located?  Austria?  Hmmm...

--  Bruce Momjian                        |  http://candle.pha.pa.us pgman@candle.pha.pa.us               |  (610)
853-3000+  If your life is a hard drive,     |  830 Blythe Avenue +  Christ can be your backup.        |  Drexel Hill,
Pennsylvania19026