Thread: to fsync or not to fsync
Hello there, I'm trying to make sure my postgres 7.4 is running as fast as it can in my box. My hardware configuration is: HP ML-350G3 Dup processor XEON 2.8 Three U320, 10000 rpm disks, RAID-5 HP 641 Raid Controller. 1GB RAM My Software config is: RedHat 7.3 - 2.4.20-28.7smp Kernel, reporting four processors because of hyper threading. Postgres 7.4 Data directory is on a ext3 journaled filesystem (data=journal) Log directory is on another partition. It is to be a dedicated to databases server, this means there are no other heavy processes running. My tests with pgbench with fsync turned off: [root@dbs pgbench]# ./pgbench -U dba -P 4ghinec osdb -c 10 -s 11 -t 1000 starting vacuum...end. transaction type: TPC-B (sort of) scaling factor: 1 number of clients: 10 number of transactions per client: 1000 number of transactions actually processed: 10000/10000 tps = 408.520028 (including connections establishing) tps = 409.697088 (excluding connections establishing) with fsync turned on: [root@dbs pgbench]# ./pgbench -U dba -P 4ghinec osdb starting vacuum...end. transaction type: TPC-B (sort of) scaling factor: 1 number of clients: 1 number of transactions per client: 10 number of transactions actually processed: 10/10 tps = 43.366451 (including connections establishing) tps = 44.867394 (excluding connections establishing) I did a lot of these tests and results are consistent. Now then, without fsync on I get a 1000% improvment!!!! Questions now: 1) since I'm using ext3 with data=journal, do I need to use fsync=true? 2) Is there not a problem with RedHat? should fsyncs asked by postgres to redhat be such a burden? 3) Any other tests you would suggest me to do? thank you all Rodrigo Filgueira Prates IT@CINTERFOR/OIT http://www.cinterfor.org.uy
Rodrigo, > Hello there, I'm trying to make sure my postgres 7.4 is running as fast > as it can in my box. > ... > My Software config is: > > RedHat 7.3 - 2.4.20-28.7smp Kernel, reporting four processors because of > hyper threading. > Postgres 7.4 > Data directory is on a ext3 journaled filesystem (data=journal) Isn't ext3 a really slow journaled filesystem [1, 2] to begin with? [1] http://slashdot.org/article.pl?sid=04/05/11/134214 [2] http://209.81.41.149/~jpiszcz/index.html It might be a good experiment to figure out how different file systems affect perf. -- Frank Hsueh, EngSci Elec 0T3 + 1 email: frank.hsueh@gmail.com
Frank, thanks for your answer, This article http://www.linuxjournal.com/print.php?sid=5841 evaluates performance from a relational database point of view and it concludes that ext3 is faster. The articles you provided evaluate filesystems by using basic shell commands, copy, tar, touch. I really don't know why ext3 would be faster for databases but here are some tests that suggest this is true, I run pgbench with data=writeback for ext3, this is as ext2 as an ext3 fs can get, and results are more or less the same as with data=journal Best run with data=journal ------------------------------------ [root@dbs pgbench]# ./pgbench -U dba -P 4ghinec osdb -c 10 -s 11 -t 1000 starting vacuum...end. transaction type: TPC-B (sort of) scaling factor: 1 number of clients: 10 number of transactions per client: 1000 number of transactions actually processed: 10000/10000 tps = 349.844978 (including connections establishing) tps = 350.715286 (excluding connections establishing) Best run with data=writeback ---------------------------------------- [root@dbs pgbench]# ./pgbench -U dba -P 4ghinec osdb -c 10 -s 11 -t 1000 starting vacuum...end. transaction type: TPC-B (sort of) scaling factor: 1 number of clients: 10 number of transactions per client: 1000 number of transactions actually processed: 10000/10000 tps = 319.239210 (including connections establishing) tps = 319.961564 (excluding connections establishing) anybody else can throw some light on this? Frank Hsueh wrote: >Rodrigo, > > >>Hello there, I'm trying to make sure my postgres 7.4 is running as fast >>as it can in my box. >> >> >>My Software config is: >> >>RedHat 7.3 - 2.4.20-28.7smp Kernel, reporting four processors because of >>hyper threading. >>Postgres 7.4 >>Data directory is on a ext3 journaled filesystem (data=journal) >> >> >Isn't ext3 a really slow journaled filesystem [1, 2] to begin with? > >[1] http://slashdot.org/article.pl?sid=04/05/11/134214 >[2] http://209.81.41.149/~jpiszcz/index.html > >It might be a good experiment to figure out how different file systems >affect perf. > > Rodrigo Filgueira Prates IT@CINTERFOR/OIT http://www.cinterfor.org.uy