Re: Trivial HugeTLB Benchmark - Mailing list pgsql-hackers

On Sun, 2007-03-04 at 10:14 -0800, Tom Lane wrote:
> If you did this only once, the results are not really trustworthy;
> you need to average several similar runs before you can have much
> confidence.  pgbench's inter-run variation is usually upwards of 10%,
> so trying to draw conclusions about half-percentage-point differences
> without averaging is a waste of time.
Good point, thanks

> Also, if scaling factor < number of clients then what you're mostly
> measuring is update-contention behavior.  Try it with -s 10 and -c 5;
> and don't forget to reinitialize the database for each run of tests
> to be sure it's fair.
> 
I did another 18 runs, 9 each for huge pages and normal shared memory.
The database was reinitialized before every third run with "pgbench -i
-s 10". The runs themselves were done with "pgbench -s 10 -c 5 -t 10000"

Normal shared memory:
tps = 1835.929043 (including connections establishing)
tps = 1697.455165 (including connections establishing)
tps = 1378.393001 (including connections establishing)

tps = 1834.802729 (including connections establishing)
tps = 1630.100895 (including connections establishing)
tps = 1415.504943 (including connections establishing)

tps = 1864.908838 (including connections establishing)
tps = 1726.295622 (including connections establishing)
tps = 1323.679649 (including connections establishing)

Average: 1634.19 tps

Huge pages:
tps = 1867.400381 (including connections establishing)
tps = 1715.269338 (including connections establishing)
tps = 1259.314139 (including connections establishing)

tps = 1880.803336 (including connections establishing)
tps = 1885.351404 (including connections establishing)
tps = 1603.302855 (including connections establishing)

tps = 1884.888431 (including connections establishing)
tps = 1563.452093 (including connections establishing)
tps = 1361.896887 (including connections establishing)

Average: 1669.08

That works out to approximately a 2.1% performance boost for huge pages.
It still doesn't seem very compelling but I could try to put together a
patch for inclusion if people were interested in such a thing.

-Ryan
This electronic mail transmission and any accompanying attachments contain confidential information intended only for
theuse of the individual or entity named above. Any dissemination, distribution, copying or action taken in reliance on
thecontents of this communication by anyone other than the intended recipient is strictly prohibited. If you have
receivedthis communication in error please immediately delete the e-mail and either notify the sender at the above
e-mailaddress or by telephone at 250.386.5323.