Thread: JFS fastest filesystem for PostgreSQL?
[reposted due to delivery error -jwb]
I just took delivery of a new system, and used the opportunity to
benchmark postgresql 8.0 performance on various filesystems. The system
in question runs Linux 2.6.12, has one CPU and 1GB of system memory, and
5 7200RPM SATA disks attached to an Areca hardware RAID controller
having 128MB of cache. The caches are all write-back.
I ran pgbench with a scale factor of 1000 and a total of 100,000
transactions per run. I varied the number of clients between 10 and
100. It appears from my test JFS is much faster than both ext3 and XFS
for this workload. JFS and XFS were made with the mkfs defaults. ext3
was made with -T largefile4 and -E stride=32. The deadline scheduler
was used for all runs (anticipatory scheduler is much worse).
Here's the result, in transactions per second.
ext3 jfs xfs
-----------------------------
10 Clients 55 81 68
100 Clients 61 100 64
----------------------------
-jwb
I was wondering - have you had a chance to run the same benchmarks on
ReiserFS (ideally both 3 and 4, with notail)?
I'd be quite interested to see how it performs in this situation since
it's my fs of choice for most things.
Thanks,
Dmitri
-----Original Message-----
From: pgsql-performance-owner@postgresql.org
[mailto:pgsql-performance-owner@postgresql.org] On Behalf Of Jeffrey W.
Baker
Sent: Thursday, July 14, 2005 2:34 AM
To: pgsql-performance@postgresql.org
Subject: [PERFORM] JFS fastest filesystem for PostgreSQL?
[reposted due to delivery error -jwb]
I just took delivery of a new system, and used the opportunity to
benchmark postgresql 8.0 performance on various filesystems. The system
in question runs Linux 2.6.12, has one CPU and 1GB of system memory, and
5 7200RPM SATA disks attached to an Areca hardware RAID controller
having 128MB of cache. The caches are all write-back.
I ran pgbench with a scale factor of 1000 and a total of 100,000
transactions per run. I varied the number of clients between 10 and
100. It appears from my test JFS is much faster than both ext3 and XFS
for this workload. JFS and XFS were made with the mkfs defaults. ext3
was made with -T largefile4 and -E stride=32. The deadline scheduler
was used for all runs (anticipatory scheduler is much worse).
Here's the result, in transactions per second.
ext3 jfs xfs
-----------------------------
10 Clients 55 81 68
100 Clients 61 100 64
----------------------------
-jwb
---------------------------(end of broadcast)---------------------------
TIP 9: In versions below 8.0, the planner will ignore your desire to
choose an index scan if your joining column's datatypes do not
match
The information transmitted is intended only for the person or entity to which it is addressed and may contain
confidentialand/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any
actionin reliance upon, this information by persons or entities other than the intended recipient is prohibited. If you
receivedthis in error, please contact the sender and delete the material from any computer
On 7/14/05, Jeffrey W. Baker <jwbaker@acm.org> wrote:
> [reposted due to delivery error -jwb]
>
> I just took delivery of a new system, and used the opportunity to
> benchmark postgresql 8.0 performance on various filesystems. The system
> in question runs Linux 2.6.12, has one CPU and 1GB of system memory, and
> 5 7200RPM SATA disks attached to an Areca hardware RAID controller
> having 128MB of cache. The caches are all write-back.
>
> I ran pgbench with a scale factor of 1000 and a total of 100,000
> transactions per run. I varied the number of clients between 10 and
> 100. It appears from my test JFS is much faster than both ext3 and XFS
> for this workload. JFS and XFS were made with the mkfs defaults. ext3
> was made with -T largefile4 and -E stride=32. The deadline scheduler
> was used for all runs (anticipatory scheduler is much worse).
>
> Here's the result, in transactions per second.
>
> ext3 jfs xfs
> -----------------------------
> 10 Clients 55 81 68
> 100 Clients 61 100 64
> ----------------------------
If you still have a chance, could you do tests with other journaling
options for ext3 (journal=writeback, journal=data)? And could you
give figures about performace of other IO elevators? I mean, you
wrote that anticipatory is much wore -- how much worse? :) Could
you give numbers for deadline,anticipatory,cfq elevators? :)
And, additionally would it be possible to give numbers for bonnie++
results? To see how does pgbench to bonnie++ relate?
Regards,
Dawid
Quoting "Jeffrey W. Baker" <jwbaker@acm.org>: > > Here's the result, in transactions per second. > > ext3 jfs xfs > ---------------------- ------- > 10 Clients 55 81 68 > 100 Clients 61 100 64 > ---------------------------- Was fsync true? And have you tried ext2? Legend has it that ext2 is the fastest thing going for synchronous writes (besides O_DIRECT or raw) because there's no journal. > > -jwb > > ---------------------------(end of broadcast)--------------------------- > TIP 9: In versions below 8.0, the planner will ignore your desire to > choose an index scan if your joining column's datatypes do not > match >
Did you seperate the data & the transaction log? I've noticed less than optimal performance on xfs if the transaction log is on the xfs data partition, and it's silly to put the xlog on a journaled filesystem anyway. Try putting xlog on an ext2 for all the tests. Mike Stone
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Dawid Kuroczko wrote: | | If you still have a chance, could you do tests with other journaling | options for ext3 (journal=writeback, journal=data)? And could you | give figures about performace of other IO elevators? I mean, you | wrote that anticipatory is much wore -- how much worse? :) Could | you give numbers for deadline,anticipatory,cfq elevators? :) | | And, additionally would it be possible to give numbers for bonnie++ | results? To see how does pgbench to bonnie++ relate? | Hello, list. I've been thinking on this one for a while - I'm not sure as to what ratio pgbench has with regard to stressing CPU vs. I/O. There is one thing that's definitely worth mentioning though: in the tests that I've been doing with bonnie++ and iozone at my former job, while building a distributed indexing engine, jfs was the one filesystem with the least strain on the CPU, which might be one of the deciding factors in making it look good for a particular workload. I'm afraid I don't have any concrete figures to offer as the material itself was classified. I can tell though that we've been comparing it with both ext2 and ext3, as well as xfs, and notably, xfs was the worst CPU hog of all. The CPU load difference between jfs and xfs was about 10% in favor of jfs in all random read/write tests, and the interesting thing is, jfs managed to shuffle around quite a lot of data: the mbps/cpu% ratio in xfs was much worse. As expected, there wasn't much difference in block transfer tests, but jfs was slightly winning in the area of CPU consumption and slightly lagging in the transfer rate field. What is a little bit concerning though, is the fact that some Linux distributors like SuSE have removed jfs support from their admin tooling <quote>due to technical problems with jfs</quote> (http://your-local-suse-mirror/.../suse/i386/9.3/docu/RELEASE-NOTES.en.html#14) I'm curious as to what this means - did they have problems integrating it into their toolchain or are there actual problems going on in jfs currently? Kind regards, - -- Grega Bremec gregab at p0f dot net -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (GNU/Linux) iD8DBQFC1ld4fu4IwuB3+XoRAqEyAJ0TS9son+brhbQGtV7Cw7T8wa9W2gCfZ02/ dWm/E/Dc99TyKbxxl2tKaZc= =nvv3 -----END PGP SIGNATURE-----
On Thu, Jul 14, 2005 at 02:15:52PM +0200, Grega Bremec wrote: >I'm curious as to what this means - did they have problems integrating >it into their toolchain or are there actual problems going on in jfs >currently? I've found jfs to be the least stable linux filesystem and won't allow it anywhere near an important system. YMMV. Mike Stone
On Thu, 2005-07-14 at 10:03 +0200, Dawid Kuroczko wrote:
> On 7/14/05, Jeffrey W. Baker <jwbaker@acm.org> wrote:
> > [reposted due to delivery error -jwb]
> >
> > I just took delivery of a new system, and used the opportunity to
> > benchmark postgresql 8.0 performance on various filesystems. The system
> > in question runs Linux 2.6.12, has one CPU and 1GB of system memory, and
> > 5 7200RPM SATA disks attached to an Areca hardware RAID controller
> > having 128MB of cache. The caches are all write-back.
> >
> > I ran pgbench with a scale factor of 1000 and a total of 100,000
> > transactions per run. I varied the number of clients between 10 and
> > 100. It appears from my test JFS is much faster than both ext3 and XFS
> > for this workload. JFS and XFS were made with the mkfs defaults. ext3
> > was made with -T largefile4 and -E stride=32. The deadline scheduler
> > was used for all runs (anticipatory scheduler is much worse).
> >
> > Here's the result, in transactions per second.
> >
> > ext3 jfs xfs
> > -----------------------------
> > 10 Clients 55 81 68
> > 100 Clients 61 100 64
> > ----------------------------
>
> If you still have a chance, could you do tests with other journaling
> options for ext3 (journal=writeback, journal=data)? And could you
> give figures about performace of other IO elevators? I mean, you
> wrote that anticipatory is much wore -- how much worse? :) Could
> you give numbers for deadline,anticipatory,cfq elevators? :)
>
> And, additionally would it be possible to give numbers for bonnie++
> results? To see how does pgbench to bonnie++ relate?
Phew, that's a lot of permutations. At 20-30 minutes per run, I'm
thinking 5-8 hours or so. Still, for you dear readers, I'll somehow
accomplish this tedious feat.
As for Bonnie, JFS is a good 60-80% faster than ext3. See my message to
ext3-users yesterday.
Using bonnie++ with a 10GB fileset, in MB/s:
ext3 jfs xfs
Read 112 188 141
Write 97 157 167
Rewrite 51 71 60
-jwb
On Wed, Jul 13, 2005 at 11:33:41PM -0700, Jeffrey W. Baker wrote: > [reposted due to delivery error -jwb] > > I just took delivery of a new system, and used the opportunity to > benchmark postgresql 8.0 performance on various filesystems. The system > in question runs Linux 2.6.12, has one CPU and 1GB of system memory, and > 5 7200RPM SATA disks attached to an Areca hardware RAID controller > having 128MB of cache. The caches are all write-back. > > I ran pgbench with a scale factor of 1000 and a total of 100,000 > transactions per run. I varied the number of clients between 10 and > 100. It appears from my test JFS is much faster than both ext3 and XFS > for this workload. JFS and XFS were made with the mkfs defaults. ext3 > was made with -T largefile4 and -E stride=32. The deadline scheduler > was used for all runs (anticipatory scheduler is much worse). > > Here's the result, in transactions per second. > > ext3 jfs xfs > ----------------------------- > 10 Clients 55 81 68 > 100 Clients 61 100 64 > ---------------------------- BTW, it'd be interesting to see how UFS on FreeBSD compared. -- Jim C. Nasby, Database Consultant decibel@decibel.org Give your computer some brain candy! www.distributed.net Team #1828 Windows: "Where do you want to go today?" Linux: "Where do you want to go tomorrow?" FreeBSD: "Are you guys coming, or what?"
>>I ran pgbench with a scale factor of 1000 and a total of 100,000 >>transactions per run. I varied the number of clients between 10 and >>100. It appears from my test JFS is much faster than both ext3 and XFS >>for this workload. JFS and XFS were made with the mkfs defaults. ext3 >>was made with -T largefile4 and -E stride=32. The deadline scheduler >>was used for all runs (anticipatory scheduler is much worse). >> >>Here's the result, in transactions per second. >> >> ext3 jfs xfs >>----------------------------- >> 10 Clients 55 81 68 >>100 Clients 61 100 64 >>---------------------------- I would be curious as to what options were passed to jfs and xfs. Sincerely, Joshua D. Drake > > > BTW, it'd be interesting to see how UFS on FreeBSD compared. -- Your PostgreSQL solutions company - Command Prompt, Inc. 1.800.492.2240 PostgreSQL Replication, Consulting, Custom Programming, 24x7 support Managed Services, Shared and Dedicated Hosting Co-Authors: plPHP, plPerlNG - http://www.commandprompt.com/