Re: Anything to be gained from a 'Postgres Filesystem'? - Mailing list pgsql-performance

Hi, Christopher,
[sorry for the delay of my answer, we were rather busy last weks]

On Thu, 04 Nov 2004 21:29:04 -0500
Christopher Browne <cbbrowne@acm.org> wrote:

> In an attempt to throw the authorities off his trail, schabios@logi-track.com (Markus Schaber) transmitted:
> > We should create a list of those needs, and then communicate those
> > to the kernel/fs developers. Then we (as well as other apps) can
> > make use of those features where they are available, and use the old
> > way everywhere else.
>
> Which kernel/fs developers did you have in mind?  The ones working on
> Linux?  Or FreeBSD?  Or DragonflyBSD?  Or Solaris?  Or AIX?

All of them, and others (e. G. Windows).

Once we have a list of those needs, the advocates can talk to the OS
developers. Some OS developers will follow, others not.

Then the postgres folks (and other application developers that benefit
from this capabilities) can point interested users to our benchmarks and
tell them that Foox performs 3 times as fast as BaarOs because they
provide better support for database needs.

> Please keep in mind that many of the PostgreSQL developers are BSD
> folk that aren't particularly interested in creating bleeding edge
> Linux capabilities.

Then this should be motivation to add those things to BSD, maybe as a
patch or loadable module so it does not bloat mainstream. I personally
would prefer it to appear in BSD first, because in case it really pays
of, it won't be long until it appears in Linux as well :-)

> Jumping into a customized filesystem that neither hardware nor
> software vendors would remotely consider supporting just doesn't look
> like a viable strategy to me.

I did not vote for a custom filesystem, as the OP did. I did vote for
isolating a set of useful capabilities PostgreSQL could exploit, and
then try to confince the kernel developers to include this capabilities,
so they are likely to be included in the main distributions.

I don't know about the BSD market, but I know that Redhat and SuSE often
ship their patched versions of the kernels (so then they officially
support the extensions), and most of this is likely to be included in
main stream later.

> > Maybe Reiser4 is a step into the right way, and maybe even a
> > postgres plugin for Reiser4 will be worth the effort. Maybe XFS/JFS
> > etc. already have such capabilities. Maybe that's completely wrong.
>
> The capabilities tend to be redundant.  They tend to implement vaguely
> similar transactional capabilities to what databases have to
> implement.  The similarities are not close enough to eliminate either
> variety of "commit" as redundant.

But a speed gain may be possible by coordinating DB and FS tansactions.

Thanks,
Markus

--
markus schaber | dipl. informatiker
logi-track ag | rennweg 14-16 | ch 8001 zürich
phone +41-43-888 62 52 | fax +41-43-888 62 53
mailto:schabios@logi-track.com | www.logi-track.com