Thread: High Performance/High Reliability File system on SuSE64
Hello All
Just wanted to gather opinions on what file system has the best balance between performance and reliability when used on a quad processor machine running SuSE64. Thanks
DAve
Dave Thompson wrote:
XFS.. hands down.
XFS.. hands down.
DAve
-- Command Prompt, Inc., home of Mammoth PostgreSQL - S/ODBC and S/JDBC Postgresql support, programming shared hosting and dedicated hosting. +1-503-667-4564 - jd@commandprompt.com - http://www.commandprompt.com PostgreSQL Replicator -- production quality replication for PostgreSQL
On Fri, Jan 23, 2004 at 08:51:03AM -0800, Joshua D. Drake wrote:
>
>
> XFS.. hands down.
I thought it was you who recently said you thought there was some
sort of possible caching problem there?
A
--
Andrew Sullivan | ajs@crankycanuck.ca
The plural of anecdote is not data.
--Roger Brinner
>>XFS.. hands down. >> >> > >I thought it was you who recently said you thought there was some >sort of possible caching problem there? > > > Not I. We have had issues with JFS and data corruption on a powerout but XFS has been rock solid in all of our tests. XFS also has the interesting ability (although I have yet to test it) that will allow you to take a snapshot of the filesystem. Thus you can have filesystem level backups of the PGDATA directory that are consistent even though the database is running. There is nothing else on Linux that comes close to that. Plus XFS has been proven in a 64 bit environment (Irix). Sincerely, Joshua D. Drake >A > > > -- Command Prompt, Inc., home of Mammoth PostgreSQL - S/ODBC and S/JDBC Postgresql support, programming shared hosting and dedicated hosting. +1-503-667-4564 - jd@commandprompt.com - http://www.commandprompt.com Mammoth PostgreSQL Replicator. Integrated Replication for PostgreSQL
On Fri, Jan 23, 2004 at 10:18:35AM -0800, Joshua D. Drake wrote: > > > Not I. We have had issues with JFS and data corruption on a powerout but > XFS has been rock solid in all of our tests. Sorry, it was Josh Berkus: http://archives.postgresql.org/pgsql-performance/2004-01/msg00086.php > There is nothing else on Linux that comes close to that. Plus XFS has been > proven in a 64 bit environment (Irix). I had lots of happy experiences with XFS when administering IRIX boxes[1], but I don't know what differences the Linux port entailed. Do you have details on that? We're certainly looking for an option over JFS at the moment. A [1] I will note, however, that it was practically the only happy experience I had with them. IRIX made the early Debian installer look positively user-friendly, and SGI's desire to make everything whiz-bang nifty by running practically every binary setuid root gave me fits. But XFS was nice. -- Andrew Sullivan | ajs@crankycanuck.ca In the future this spectacle of the middle classes shocking the avant- garde will probably become the textbook definition of Postmodernism. --Brad Holland
>>There is nothing else on Linux that comes close to that. Plus XFS has been >>proven in a 64 bit environment (Irix). >> >> > >I had lots of happy experiences with XFS when administering IRIX >boxes[1], but I don't know what differences the Linux port entailed. >Do you have details on that? > http://oss.sgi.com/projects/xfs/ >We're certainly looking for an option >over JFS at the moment. > >A > >[1] I will note, however, that it was practically the only happy >experience I had with them. IRIX made the early Debian installer >look positively user-friendly, and SGI's desire to make everything >whiz-bang nifty by running practically every binary setuid root gave >me fits. But XFS was nice. > > > -- Command Prompt, Inc., home of Mammoth PostgreSQL - S/ODBC and S/JDBC Postgresql support, programming shared hosting and dedicated hosting. +1-503-667-4564 - jd@commandprompt.com - http://www.commandprompt.com Mammoth PostgreSQL Replicator. Integrated Replication for PostgreSQL
On Fri, Jan 23, 2004 at 11:05:41AM -0800, Joshua D. Drake wrote: > http://oss.sgi.com/projects/xfs/ Yes, I guess I shoulda thought of that, eh? Thanks. The docs do suggest that there are some significant differences between the two versions of the filesystem, so I'm not sure how sanguine I'd be about the degree of "testing" the filesystem has received on Linux. On the other hand, I wouldn't be surprised if it were no worse than the other options. A -- Andrew Sullivan | ajs@crankycanuck.ca The fact that technology doesn't work is no bar to success in the marketplace. --Philip Greenspun
>Yes, I guess I shoulda thought of that, eh? Thanks. The docs do >suggest that there are some significant differences between the two >versions of the filesystem, so I'm not sure how sanguine I'd be about >the degree of "testing" the filesystem has received on Linux. On the > > Well SuSE ships with XFS and SuSE tends to be really good about testing. Better than RedHat IMHO. Just the fact that RedHat uses ext3 as the default is a black eye. XFS has been around a LONG time, and on Linux for a couple of years now. Plus I believe it is the default FS for all of the really high end stuff SGI is doing with Linux. I would (and do) trust XFS currently over ANY other journalled option on Linux. Sincerely, Joshua D. Drake >other hand, I wouldn't be surprised if it were no worse than the >other options. > >A > > > -- Command Prompt, Inc., home of Mammoth PostgreSQL - S/ODBC and S/JDBC Postgresql support, programming shared hosting and dedicated hosting. +1-503-667-4564 - jd@commandprompt.com - http://www.commandprompt.com Mammoth PostgreSQL Replicator. Integrated Replication for PostgreSQL
jd@commandprompt.com ("Joshua D. Drake") writes:
>>Yes, I guess I shoulda thought of that, eh? Thanks. The docs do
>>suggest that there are some significant differences between the two
>>versions of the filesystem, so I'm not sure how sanguine I'd be about
>>the degree of "testing" the filesystem has received on Linux. On the
>>
> Well SuSE ships with XFS and SuSE tends to be really good about
> testing. Better than RedHat IMHO. Just the fact that RedHat uses
> ext3 as the default is a black eye.
Well, I'd point to one major factor with RHAT; they employ Stephen
Tweedie, creator of ext3, and have been paying him to work on it for
some time now. If they _didn't_ promote use of ext3, they would be
very much vulnerable to the "won't eat their own dogfood" criticism.
> XFS has been around a LONG time, and on Linux for a couple of years
> now. Plus I believe it is the default FS for all of the really high
> end stuff SGI is doing with Linux.
Ah, but there is a bit of a 'problem' nonetheless; XFS is not
'officially supported' as part of the Linux kernel until version 2.6,
which is still pretty "bleeding edge." Until 2.6 solidifies a bit
more (aside: based on experiences with 2.6.0, "quite a lot more"), it
is a "patchy" add-on to the 'stable' 2.4 kernel series.
Do the patches work? As far as I have heard, quite well indeed. But
the fact of it not having been 'official' is a fair little bit of a
downside.
> I would (and do) trust XFS currently over ANY other journalled
> option on Linux.
I'm getting less and less inclined to trust ext3 or JFS, which "floats
upwards" any other boats that are lingering around...
--
let name="cbbrowne" and tld="libertyrms.info" in String.concat "@" [name;tld];;
<http://dev6.int.libertyrms.com/>
Christopher Browne
(416) 646 3304 x124 (land)
>Well, I'd point to one major factor with RHAT; they employ Stephen >Tweedie, creator of ext3, and have been paying him to work on it for >some time now. If they _didn't_ promote use of ext3, they would be >very much vulnerable to the "won't eat their own dogfood" criticism. > > > True but frankly, they shouldn't. EXT3 has some serious issues. In fact if you are running a stock RH kernel before 2.4.20 you can destroy your PostgreSQL database with it. Not to mention how slow it is ;) >>XFS has been around a LONG time, and on Linux for a couple of years >>now. Plus I believe it is the default FS for all of the really high >>end stuff SGI is doing with Linux. >> >> > >Ah, but there is a bit of a 'problem' nonetheless; XFS is not >'officially supported' as part of the Linux kernel until version 2.6, >which is still pretty "bleeding edge." > That is not true see: http://kerneltrap.org/node/view/1751 >Until 2.6 solidifies a bit >more (aside: based on experiences with 2.6.0, "quite a lot more"), it >is a "patchy" add-on to the 'stable' 2.4 kernel series. > > > Again see above :) >Do the patches work? As far as I have heard, quite well indeed. But >the fact of it not having been 'official' is a fair little bit of a >downside. > > What is official? Sincerely, Joshua D. Drake >>I would (and do) trust XFS currently over ANY other journalled >>option on Linux. >> >> > >I'm getting less and less inclined to trust ext3 or JFS, which "floats >upwards" any other boats that are lingering around... > > -- Command Prompt, Inc., home of Mammoth PostgreSQL - S/ODBC and S/JDBC Postgresql support, programming shared hosting and dedicated hosting. +1-503-667-4564 - jd@commandprompt.com - http://www.commandprompt.com Mammoth PostgreSQL Replicator. Integrated Replication for PostgreSQL
>You can do snapshots in FreeBSD 5.x with UFS2 as well but that ( >nor XFS snapshots ) will let you backup with the database server >running. Just because you will get the file exactly as it was at >a particular instant does not mean that the postmaster did not >still have some some data that was not flushed to disk yet. > > Ahh... isn't that what fsync is for? -- Command Prompt, Inc., home of Mammoth PostgreSQL - S/ODBC and S/JDBC Postgresql support, programming shared hosting and dedicated hosting. +1-503-667-4564 - jd@commandprompt.com - http://www.commandprompt.com Mammoth PostgreSQL Replicator. Integrated Replication for PostgreSQL
They seem pretty clean (have patched vanilla kernels + xfs for Mandrake 9.2/9.0). And yes, I would recommend xfs - noticeably faster than ext3, and no sign of any mysterious hangs under load. best wishes Mark Christopher Browne wrote: >Do the patches work? As far as I have heard, quite well indeed. But >the fact of it not having been 'official' is a fair little bit of a >downside. > > >
Mark Kirkwood wrote: > They seem pretty clean (have patched vanilla kernels + xfs for > Mandrake 9.2/9.0). > > And yes, I would recommend xfs - noticeably faster than ext3, and no > sign of any mysterious hangs under load. The hangs you are having are due to several issues... one of them is the way ext3 syncs. What kernel version are you running? Sincerely, Joshua Drake > > best wishes > > Mark > > Christopher Browne wrote: > >> Do the patches work? As far as I have heard, quite well indeed. But >> the fact of it not having been 'official' is a fair little bit of a >> downside. >> >> >> > > > ---------------------------(end of broadcast)--------------------------- > TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org -- Command Prompt, Inc., home of Mammoth PostgreSQL - S/ODBC and S/JDBC Postgresql support, programming shared hosting and dedicated hosting. +1-503-667-4564 - jd@commandprompt.com - http://www.commandprompt.com PostgreSQL Replicator -- production quality replication for PostgreSQL
jd@commandprompt.com says... > XFS.. hands down. Off topic question here, but I'm a bit at a loss to understand exactly what sgi are doing. I thought that they were removing IRIX and going with Linux as the OS to their high end graphical workstations, yet I see they still have IRIX on their site. What, exactly, is the story with this? Paul... -- plinehan y_a_h_o_o and d_o_t com C++ Builder 5 SP1, Interbase 6.0.1.6 IBX 5.04 W2K Pro Please do not top-post. "XML avoids the fundamental question of what we should do, by focusing entirely on how we should do it." quote from http://www.metatorial.com
Paul Ganainm <paulsnewsgroups@hotmail.com> writes: > jd@commandprompt.com says... > > >> XFS.. hands down. > > > Off topic question here, but I'm a bit at a loss to understand exactly > what sgi are doing. > > > I thought that they were removing IRIX and going with Linux as the OS to > their high end graphical workstations, yet I see they still have IRIX on > their site. > > > What, exactly, is the story with this? Well, they do have a very large installed base of Irix systems, and a certain obligation to keep supporting them... As long as there are people whi still want MIPS/Irix they will probably keep selling at least a few models. -Doug
> Mark Kirkwood wrote a little unclearly: > >> >> And yes, I would recommend xfs - noticeably faster than ext3, and no >> sign of any mysterious hangs under load. > > I was thinking about the reported mini-hangs that folks are seeing with jfs, except the all important keyword "jfs" didnt make it out of my head and into the 2nd bit of the message :-( Sorry for the confusion. So I meant "faster that ext3, and without strange mini hangs like jfs...". Joshua wrote: > > The hangs you are having are due to several issues... one of them is > the way ext3 syncs. Never suffered this personally, it was an unexpected filesystem corruption with ext3 that "encouraged" me to try out xfs instead. best wishes Mark
jd@commandprompt.com ("Joshua D. Drake") writes:
>>Well, I'd point to one major factor with RHAT; they employ Stephen
>>Tweedie, creator of ext3, and have been paying him to work on it for
>>some time now. If they _didn't_ promote use of ext3, they would be
>>very much vulnerable to the "won't eat their own dogfood" criticism.
>
> True but frankly, they shouldn't. EXT3 has some serious issues. In fact
> if you are running a stock RH kernel before 2.4.20 you can destroy your
> PostgreSQL database with it.
>
> Not to mention how slow it is ;)
I'm not defending ext3's merits; just the clear reason why RHAT uses
it :-).
>>>XFS has been around a LONG time, and on Linux for a couple of years
>>>now. Plus I believe it is the default FS for all of the really high
>>>end stuff SGI is doing with Linux.
>>>
>>>
>>
>>Ah, but there is a bit of a 'problem' nonetheless; XFS is not
>>'officially supported' as part of the Linux kernel until version 2.6,
>> which is still pretty "bleeding edge."
>>
> That is not true see:
>
> http://kerneltrap.org/node/view/1751
Well, I just downloaded 2.4.24 this week, and I don't see XFS included
in it. I see ReiserFS, ext3, and JFS, but not XFS.
>>Until 2.6 solidifies a bit more (aside: based on experiences with
>>2.6.0, "quite a lot more"), it is a "patchy" add-on to the 'stable'
>>2.4 kernel series.
>>
> Again see above :)
>>Do the patches work? As far as I have heard, quite well indeed. But
>>the fact of it not having been 'official' is a fair little bit of a
>>downside.
>
> What is official?
"Is it included in the kernel sources hosted at ftp.kernel.org?"
--
let name="cbbrowne" and tld="libertyrms.info" in name ^ "@" ^ tld;;
<http://dev6.int.libertyrms.com/>
Christopher Browne
(416) 646 3304 x124 (land)
Christopher Browne <cbbrowne@libertyrms.info> writes: > Ah, but there is a bit of a 'problem' nonetheless; XFS is not > 'officially supported' as part of the Linux kernel until version 2.6, > which is still pretty "bleeding edge." Until 2.6 solidifies a bit > more (aside: based on experiences with 2.6.0, "quite a lot more"), it > is a "patchy" add-on to the 'stable' 2.4 kernel series. Actually I think XFS is in 2.4.25. -- greg
> Hello All > > Just wanted to gather opinions on what file system has the best balance between performance and > reliability when used on a quad processor machine running SuSE64. Thanks > > DAve I was reading the article 'Behind the ALTIX 3000' in the Feb. 2003 Linux Journal, and it mentioned the following paper which compares Ext2, Ext3, ReiserFS, XFS, and JFS: Filesystem Performance and Scalability in Linux 2.4.17 Originally published in Proceedings of the FREENIX Track: 2002 USENIX Annual Technical Conference http://oss.sgi.com/projects/xfs/papers/filesystem-perf-tm.pdf Quoting the abtract: Although the best-performing filesystem varies depending on the benchmark and system used, some larger trends are evident in the data. On the smaller systems, the best-performing file system is often Ext2, Ext3 or ReiserFS. For the larger systems and higher loads, XFS can provide the best overall performance. George Essig
On 23 Jan, Dave Thompson wrote:
> Hello All
>
> Just wanted to gather opinions on what file system has the best balance between performance and reliability when used
ona quad processor machine running SuSE64. Thanks
>
> DAve
Hi Dave,
I have some data for performance using our DBT-2 workload (OLTP type
transactions) with Linux-2.6 with various filesystems and i/o
schedulers. I know it doesn't address the reliability part of your
question, and it's in a completely different environment (32-bit as well
as a different distro), but if you think you'll find the results
interesting:
http://developer.osdl.org/markw/fs/project_results.html
Mark
On 01/23/2004-10:18AM, Joshua D. Drake wrote: > > XFS also has the interesting ability (although I have yet to test it) > that will allow you > to take a snapshot of the filesystem. Thus you can have filesystem level > backups > of the PGDATA directory that are consistent even though the database is > running. You can do snapshots in FreeBSD 5.x with UFS2 as well but that ( nor XFS snapshots ) will let you backup with the database server running. Just because you will get the file exactly as it was at a particular instant does not mean that the postmaster did not still have some some data that was not flushed to disk yet.
Christopher Weimann <cweimann@k12hq.com> writes:
> You can do snapshots in FreeBSD 5.x with UFS2 as well but that (
> nor XFS snapshots ) will let you backup with the database server
> running. Just because you will get the file exactly as it was at
> a particular instant does not mean that the postmaster did not
> still have some some data that was not flushed to disk yet.
It *will* work, if you have an instantaneous filesystem snapshot
covering the entire $PGDATA directory tree (both data files and WAL).
Restarting the postmaster on the backup will result in a WAL replay
sequence, and at the end the data files will be consistent. If this
were not so, we'd not be crash-proof. The instantaneous snapshot
is exactly equivalent to the on-disk state at the moment of a kernel
crash or power failure, no?
regards, tom lane
Christopher Weimann wrote: > On 01/23/2004-10:18AM, Joshua D. Drake wrote: > > > > XFS also has the interesting ability (although I have yet to test it) > > that will allow you > > to take a snapshot of the filesystem. Thus you can have filesystem level > > backups > > of the PGDATA directory that are consistent even though the database is > > running. > > You can do snapshots in FreeBSD 5.x with UFS2 as well but that ( > nor XFS snapshots ) will let you backup with the database server > running. Just because you will get the file exactly as it was at > a particular instant does not mean that the postmaster did not > still have some some data that was not flushed to disk yet. Uh, yea, it does. If the snapshot includes all of /data, including WAL/xlog, you can then back up the snapshot and restore it on another machine. It will restart just like a crash. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania 19073