Thread: High Performance/High Reliability File system on SuSE64

High Performance/High Reliability File system on SuSE64

From
"Dave Thompson"
Date:
Hello All
 
Just wanted to gather opinions on what file system has the best balance between performance and reliability when used on a quad processor machine running SuSE64.  Thanks
 
DAve

Re: High Performance/High Reliability File system on SuSE64

From
"Joshua D. Drake"
Date:
Dave Thompson wrote:
Hello All
 
Just wanted to gather opinions on what file system has the best balance between performance and reliability when used on a quad processor machine running SuSE64.  Thanks

XFS.. hands down.

 
DAve


-- 
Command Prompt, Inc., home of Mammoth PostgreSQL - S/ODBC and S/JDBC
Postgresql support, programming shared hosting and dedicated hosting.
+1-503-667-4564 - jd@commandprompt.com - http://www.commandprompt.com
PostgreSQL Replicator -- production quality replication for PostgreSQL

Re: High Performance/High Reliability File system on SuSE64

From
Andrew Sullivan
Date:
On Fri, Jan 23, 2004 at 08:51:03AM -0800, Joshua D. Drake wrote:
>
>
> XFS.. hands down.

I thought it was you who recently said you thought there was some
sort of possible caching problem there?

A

--
Andrew Sullivan  | ajs@crankycanuck.ca
The plural of anecdote is not data.
        --Roger Brinner

Re: High Performance/High Reliability File system on SuSE64

From
"Joshua D. Drake"
Date:
>>XFS.. hands down.
>>
>>
>
>I thought it was you who recently said you thought there was some
>sort of possible caching problem there?
>
>
>
Not I. We have had issues with JFS and data corruption on a powerout but
XFS has been rock solid in all of our tests.

XFS also has the interesting ability (although I have yet to test it)
that will allow you
to take a snapshot of the filesystem. Thus you can have filesystem level
backups
of the PGDATA directory that are consistent even though the database is
running.
There is nothing else on Linux that comes close to that. Plus XFS has been
proven in a 64 bit environment (Irix).

Sincerely,

Joshua D. Drake




>A
>
>
>


--
Command Prompt, Inc., home of Mammoth PostgreSQL - S/ODBC and S/JDBC
Postgresql support, programming shared hosting and dedicated hosting.
+1-503-667-4564 - jd@commandprompt.com - http://www.commandprompt.com
Mammoth PostgreSQL Replicator. Integrated Replication for PostgreSQL


Re: High Performance/High Reliability File system on SuSE64

From
Andrew Sullivan
Date:
On Fri, Jan 23, 2004 at 10:18:35AM -0800, Joshua D. Drake wrote:
> >
> Not I. We have had issues with JFS and data corruption on a powerout but
> XFS has been rock solid in all of our tests.

Sorry, it was Josh Berkus:

http://archives.postgresql.org/pgsql-performance/2004-01/msg00086.php

> There is nothing else on Linux that comes close to that. Plus XFS has been
> proven in a 64 bit environment (Irix).

I had lots of happy experiences with XFS when administering IRIX
boxes[1], but I don't know what differences the Linux port entailed.
Do you have details on that?  We're certainly looking for an option
over JFS at the moment.

A

[1] I will note, however, that it was practically the only happy
experience I had with them.  IRIX made the early Debian installer
look positively user-friendly, and SGI's desire to make everything
whiz-bang nifty by running practically every binary setuid root gave
me fits.  But XFS was nice.

--
Andrew Sullivan  | ajs@crankycanuck.ca
In the future this spectacle of the middle classes shocking the avant-
garde will probably become the textbook definition of Postmodernism.
                --Brad Holland

Re: High Performance/High Reliability File system on SuSE64

From
"Joshua D. Drake"
Date:
>>There is nothing else on Linux that comes close to that. Plus XFS has been
>>proven in a 64 bit environment (Irix).
>>
>>
>
>I had lots of happy experiences with XFS when administering IRIX
>boxes[1], but I don't know what differences the Linux port entailed.
>Do you have details on that?
>

http://oss.sgi.com/projects/xfs/


>We're certainly looking for an option
>over JFS at the moment.
>
>A
>
>[1] I will note, however, that it was practically the only happy
>experience I had with them.  IRIX made the early Debian installer
>look positively user-friendly, and SGI's desire to make everything
>whiz-bang nifty by running practically every binary setuid root gave
>me fits.  But XFS was nice.
>
>
>


--
Command Prompt, Inc., home of Mammoth PostgreSQL - S/ODBC and S/JDBC
Postgresql support, programming shared hosting and dedicated hosting.
+1-503-667-4564 - jd@commandprompt.com - http://www.commandprompt.com
Mammoth PostgreSQL Replicator. Integrated Replication for PostgreSQL


Re: High Performance/High Reliability File system on SuSE64

From
Andrew Sullivan
Date:
On Fri, Jan 23, 2004 at 11:05:41AM -0800, Joshua D. Drake wrote:
> http://oss.sgi.com/projects/xfs/

Yes, I guess I shoulda thought of that, eh?  Thanks.  The docs do
suggest that there are some significant differences between the two
versions of the filesystem, so I'm not sure how sanguine I'd be about
the degree of "testing" the filesystem has received on Linux.  On the
other hand, I wouldn't be surprised if it were no worse than the
other options.

A

--
Andrew Sullivan  | ajs@crankycanuck.ca
The fact that technology doesn't work is no bar to success in the marketplace.
        --Philip Greenspun

Re: High Performance/High Reliability File system on SuSE64

From
"Joshua D. Drake"
Date:
>Yes, I guess I shoulda thought of that, eh?  Thanks.  The docs do
>suggest that there are some significant differences between the two
>versions of the filesystem, so I'm not sure how sanguine I'd be about
>the degree of "testing" the filesystem has received on Linux.  On the
>
>
Well SuSE ships with XFS and SuSE tends to be really good about testing.
Better than RedHat IMHO. Just the fact that RedHat uses ext3 as the default
is a black eye.

XFS has been around a LONG time, and on Linux for a couple of years now.
Plus I believe it is the default FS for all of the really high end stuff
SGI is doing
with Linux.

I would (and do) trust XFS currently over ANY other journalled option on
Linux.

Sincerely,

Joshua D. Drake



>other hand, I wouldn't be surprised if it were no worse than the
>other options.
>
>A
>
>
>


--
Command Prompt, Inc., home of Mammoth PostgreSQL - S/ODBC and S/JDBC
Postgresql support, programming shared hosting and dedicated hosting.
+1-503-667-4564 - jd@commandprompt.com - http://www.commandprompt.com
Mammoth PostgreSQL Replicator. Integrated Replication for PostgreSQL


Re: High Performance/High Reliability File system on SuSE64

From
Christopher Browne
Date:
jd@commandprompt.com ("Joshua D. Drake") writes:
>>Yes, I guess I shoulda thought of that, eh?  Thanks.  The docs do
>>suggest that there are some significant differences between the two
>>versions of the filesystem, so I'm not sure how sanguine I'd be about
>>the degree of "testing" the filesystem has received on Linux.  On the
>>
> Well SuSE ships with XFS and SuSE tends to be really good about
> testing.  Better than RedHat IMHO. Just the fact that RedHat uses
> ext3 as the default is a black eye.

Well, I'd point to one major factor with RHAT; they employ Stephen
Tweedie, creator of ext3, and have been paying him to work on it for
some time now.  If they _didn't_ promote use of ext3, they would be
very much vulnerable to the "won't eat their own dogfood" criticism.

> XFS has been around a LONG time, and on Linux for a couple of years
> now.  Plus I believe it is the default FS for all of the really high
> end stuff SGI is doing with Linux.

Ah, but there is a bit of a 'problem' nonetheless; XFS is not
'officially supported' as part of the Linux kernel until version 2.6,
which is still pretty "bleeding edge."  Until 2.6 solidifies a bit
more (aside: based on experiences with 2.6.0, "quite a lot more"), it
is a "patchy" add-on to the 'stable' 2.4 kernel series.

Do the patches work?  As far as I have heard, quite well indeed.  But
the fact of it not having been 'official' is a fair little bit of a
downside.

> I would (and do) trust XFS currently over ANY other journalled
> option on Linux.

I'm getting less and less inclined to trust ext3 or JFS, which "floats
upwards" any other boats that are lingering around...
--
let name="cbbrowne" and tld="libertyrms.info" in String.concat "@" [name;tld];;
<http://dev6.int.libertyrms.com/>
Christopher Browne
(416) 646 3304 x124 (land)

Re: High Performance/High Reliability File system on SuSE64

From
"Joshua D. Drake"
Date:
>Well, I'd point to one major factor with RHAT; they employ Stephen
>Tweedie, creator of ext3, and have been paying him to work on it for
>some time now.  If they _didn't_ promote use of ext3, they would be
>very much vulnerable to the "won't eat their own dogfood" criticism.
>
>
>
True but frankly, they shouldn't. EXT3 has some serious issues. In fact
if you are running a stock RH kernel before 2.4.20 you can destroy your
PostgreSQL database with it.

Not to mention how slow it is ;)

>>XFS has been around a LONG time, and on Linux for a couple of years
>>now.  Plus I believe it is the default FS for all of the really high
>>end stuff SGI is doing with Linux.
>>
>>
>
>Ah, but there is a bit of a 'problem' nonetheless; XFS is not
>'officially supported' as part of the Linux kernel until version 2.6,
>which is still pretty "bleeding edge."
>
That is not true see:

http://kerneltrap.org/node/view/1751



>Until 2.6 solidifies a bit
>more (aside: based on experiences with 2.6.0, "quite a lot more"), it
>is a "patchy" add-on to the 'stable' 2.4 kernel series.
>
>
>
Again see above :)

>Do the patches work?  As far as I have heard, quite well indeed.  But
>the fact of it not having been 'official' is a fair little bit of a
>downside.
>
>

What is official?


Sincerely,

Joshua D. Drake

>>I would (and do) trust XFS currently over ANY other journalled
>>option on Linux.
>>
>>
>
>I'm getting less and less inclined to trust ext3 or JFS, which "floats
>upwards" any other boats that are lingering around...
>
>


--
Command Prompt, Inc., home of Mammoth PostgreSQL - S/ODBC and S/JDBC
Postgresql support, programming shared hosting and dedicated hosting.
+1-503-667-4564 - jd@commandprompt.com - http://www.commandprompt.com
Mammoth PostgreSQL Replicator. Integrated Replication for PostgreSQL


Re: High Performance/High Reliability File system on SuSE64

From
"Joshua D. Drake"
Date:
>You can do snapshots in FreeBSD 5.x with UFS2 as well but that (
>nor XFS snapshots ) will let you backup with the database server
>running.  Just because you will get the file exactly as it was at
>a particular instant does not mean that the postmaster did not
>still have some some data that was not flushed to disk yet.
>
>
Ahh... isn't that what fsync is for?

--
Command Prompt, Inc., home of Mammoth PostgreSQL - S/ODBC and S/JDBC
Postgresql support, programming shared hosting and dedicated hosting.
+1-503-667-4564 - jd@commandprompt.com - http://www.commandprompt.com
Mammoth PostgreSQL Replicator. Integrated Replication for PostgreSQL


Re: High Performance/High Reliability File system on SuSE64

From
Mark Kirkwood
Date:
They seem pretty clean (have patched vanilla kernels + xfs for Mandrake
9.2/9.0).

And yes, I would recommend xfs - noticeably faster than ext3, and no
sign of any mysterious hangs under load.

best wishes

Mark

Christopher Browne wrote:

>Do the patches work?  As far as I have heard, quite well indeed.  But
>the fact of it not having been 'official' is a fair little bit of a
>downside.
>
>
>


Re: High Performance/High Reliability File system on SuSE64

From
"Joshua D. Drake"
Date:
Mark Kirkwood wrote:

> They seem pretty clean (have patched vanilla kernels + xfs for
> Mandrake 9.2/9.0).
>
> And yes, I would recommend xfs - noticeably faster than ext3, and no
> sign of any mysterious hangs under load.


The hangs you are having are due to several issues... one of them is the
way ext3 syncs. What kernel version are
you running?

Sincerely,

Joshua Drake



>
> best wishes
>
> Mark
>
> Christopher Browne wrote:
>
>> Do the patches work?  As far as I have heard, quite well indeed.  But
>> the fact of it not having been 'official' is a fair little bit of a
>> downside.
>>
>>
>>
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org



--
Command Prompt, Inc., home of Mammoth PostgreSQL - S/ODBC and S/JDBC
Postgresql support, programming shared hosting and dedicated hosting.
+1-503-667-4564 - jd@commandprompt.com - http://www.commandprompt.com
PostgreSQL Replicator -- production quality replication for PostgreSQL


Re: High Performance/High Reliability File system on SuSE64

From
Paul Ganainm
Date:
jd@commandprompt.com says...


> XFS.. hands down.


Off topic question here, but I'm a bit at a loss to understand exactly
what sgi are doing.


I thought that they were removing IRIX and going with Linux as the OS to
their high end graphical workstations, yet I see they still have IRIX on
their site.


What, exactly, is the story with this?


Paul...


--
plinehan  y_a_h_o_o  and d_o_t  com
C++ Builder 5 SP1, Interbase 6.0.1.6 IBX 5.04 W2K Pro
Please do not top-post.

"XML avoids the fundamental question of what we should do,
by focusing entirely on how we should do it."

quote from http://www.metatorial.com

Re: High Performance/High Reliability File system on SuSE64

From
Doug McNaught
Date:
Paul Ganainm <paulsnewsgroups@hotmail.com> writes:

> jd@commandprompt.com says...
>
>
>> XFS.. hands down.
>
>
> Off topic question here, but I'm a bit at a loss to understand exactly
> what sgi are doing.
>
>
> I thought that they were removing IRIX and going with Linux as the OS to
> their high end graphical workstations, yet I see they still have IRIX on
> their site.
>
>
> What, exactly, is the story with this?

Well, they do have a very large installed base of Irix systems, and
a certain obligation to keep supporting them...  As long as there are
people whi still want MIPS/Irix they will probably keep selling at
least a few models.

-Doug

Re: High Performance/High Reliability File system on SuSE64

From
Mark Kirkwood
Date:

> Mark Kirkwood wrote a little unclearly:
>
>>
>> And yes, I would recommend xfs - noticeably faster than ext3, and no
>> sign of any mysterious hangs under load.
>
>
I was thinking about the reported mini-hangs that folks are seeing with
jfs, except the all important keyword "jfs" didnt make it out of my head
and into the 2nd bit of the message  :-( Sorry for the confusion.

So I meant "faster that ext3, and without strange mini hangs like jfs...".


Joshua wrote:

>
> The hangs you are having are due to several issues... one of them is
> the way ext3 syncs.

Never suffered this personally, it was an unexpected filesystem
corruption with ext3 that "encouraged" me to try out xfs instead.

best wishes

Mark


Re: High Performance/High Reliability File system on SuSE64

From
Christopher Browne
Date:
jd@commandprompt.com ("Joshua D. Drake") writes:
>>Well, I'd point to one major factor with RHAT; they employ Stephen
>>Tweedie, creator of ext3, and have been paying him to work on it for
>>some time now.  If they _didn't_ promote use of ext3, they would be
>>very much vulnerable to the "won't eat their own dogfood" criticism.
>
> True but frankly, they shouldn't. EXT3 has some serious issues. In fact
> if you are running a stock RH kernel before 2.4.20 you can destroy your
> PostgreSQL database with it.
>
> Not to mention how slow it is ;)

I'm not defending ext3's merits; just the clear reason why RHAT uses
it :-).

>>>XFS has been around a LONG time, and on Linux for a couple of years
>>>now.  Plus I believe it is the default FS for all of the really high
>>>end stuff SGI is doing with Linux.
>>>
>>>
>>
>>Ah, but there is a bit of a 'problem' nonetheless; XFS is not
>>'officially supported' as part of the Linux kernel until version 2.6,
>> which is still pretty "bleeding edge."
>>
> That is not true see:
>
> http://kerneltrap.org/node/view/1751

Well, I just downloaded 2.4.24 this week, and I don't see XFS included
in it.  I see ReiserFS, ext3, and JFS, but not XFS.

>>Until 2.6 solidifies a bit more (aside: based on experiences with
>>2.6.0, "quite a lot more"), it is a "patchy" add-on to the 'stable'
>>2.4 kernel series.
>>
> Again see above :)

>>Do the patches work?  As far as I have heard, quite well indeed.  But
>>the fact of it not having been 'official' is a fair little bit of a
>>downside.
>
> What is official?

"Is it included in the kernel sources hosted at ftp.kernel.org?"
--
let name="cbbrowne" and tld="libertyrms.info" in name ^ "@" ^ tld;;
<http://dev6.int.libertyrms.com/>
Christopher Browne
(416) 646 3304 x124 (land)

Re: High Performance/High Reliability File system on SuSE64

From
Greg Stark
Date:
Christopher Browne <cbbrowne@libertyrms.info> writes:

> Ah, but there is a bit of a 'problem' nonetheless; XFS is not
> 'officially supported' as part of the Linux kernel until version 2.6,
> which is still pretty "bleeding edge."  Until 2.6 solidifies a bit
> more (aside: based on experiences with 2.6.0, "quite a lot more"), it
> is a "patchy" add-on to the 'stable' 2.4 kernel series.

Actually I think XFS is in 2.4.25.

--
greg

Re: High Performance/High Reliability File system on SuSE64

From
George Essig
Date:
> Hello All
>
> Just wanted to gather opinions on what file system has the best balance between performance and
> reliability when used on a quad processor machine running SuSE64.  Thanks
>
> DAve

I was reading the article 'Behind the ALTIX 3000' in the Feb. 2003 Linux Journal, and it mentioned
the following paper which compares Ext2, Ext3, ReiserFS, XFS, and JFS:

Filesystem Performance and Scalability in Linux 2.4.17
Originally published in Proceedings of the FREENIX Track:
2002 USENIX Annual Technical Conference
http://oss.sgi.com/projects/xfs/papers/filesystem-perf-tm.pdf

Quoting the abtract:

Although the best-performing filesystem varies depending on the benchmark and system used, some
larger trends are evident in the data. On the smaller systems, the best-performing file system is
often Ext2, Ext3 or ReiserFS. For the larger systems and higher loads, XFS can provide the best
overall performance.

George Essig

Re: High Performance/High Reliability File system on

From
markw@osdl.org
Date:
On 23 Jan, Dave Thompson wrote:
> Hello All
>
> Just wanted to gather opinions on what file system has the best balance between performance and reliability when used
ona quad processor machine running SuSE64.  Thanks 
>
> DAve

Hi Dave,

I have some data for performance using our DBT-2 workload (OLTP type
transactions) with Linux-2.6 with various filesystems and i/o
schedulers.  I know it doesn't address the reliability part of your
question, and it's in a completely different environment (32-bit as well
as a different distro), but if you think you'll find the results
interesting:

    http://developer.osdl.org/markw/fs/project_results.html

Mark

Re: High Performance/High Reliability File system on SuSE64

From
Christopher Weimann
Date:
On 01/23/2004-10:18AM, Joshua D. Drake wrote:
>
> XFS also has the interesting ability (although I have yet to test it)
> that will allow you
> to take a snapshot of the filesystem. Thus you can have filesystem level
> backups
> of the PGDATA directory that are consistent even though the database is
> running.

You can do snapshots in FreeBSD 5.x with UFS2 as well but that (
nor XFS snapshots ) will let you backup with the database server
running.  Just because you will get the file exactly as it was at
a particular instant does not mean that the postmaster did not
still have some some data that was not flushed to disk yet.


Re: High Performance/High Reliability File system on SuSE64

From
Tom Lane
Date:
Christopher Weimann <cweimann@k12hq.com> writes:
> You can do snapshots in FreeBSD 5.x with UFS2 as well but that (
> nor XFS snapshots ) will let you backup with the database server
> running.  Just because you will get the file exactly as it was at
> a particular instant does not mean that the postmaster did not
> still have some some data that was not flushed to disk yet.

It *will* work, if you have an instantaneous filesystem snapshot
covering the entire $PGDATA directory tree (both data files and WAL).
Restarting the postmaster on the backup will result in a WAL replay
sequence, and at the end the data files will be consistent.  If this
were not so, we'd not be crash-proof.  The instantaneous snapshot
is exactly equivalent to the on-disk state at the moment of a kernel
crash or power failure, no?

            regards, tom lane

Re: High Performance/High Reliability File system on SuSE64

From
Bruce Momjian
Date:
Christopher Weimann wrote:
> On 01/23/2004-10:18AM, Joshua D. Drake wrote:
> >
> > XFS also has the interesting ability (although I have yet to test it)
> > that will allow you
> > to take a snapshot of the filesystem. Thus you can have filesystem level
> > backups
> > of the PGDATA directory that are consistent even though the database is
> > running.
>
> You can do snapshots in FreeBSD 5.x with UFS2 as well but that (
> nor XFS snapshots ) will let you backup with the database server
> running.  Just because you will get the file exactly as it was at
> a particular instant does not mean that the postmaster did not
> still have some some data that was not flushed to disk yet.

Uh, yea, it does.  If the snapshot includes all of /data, including
WAL/xlog, you can then back up the snapshot and restore it on another
machine.  It will restart just like a crash.

--
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073