Thread: virtual filesystem atop a PostgreSQL database

virtual filesystem atop a PostgreSQL database

From
Frank Joerdens
Date:
I am wondering whether anyone has already tried it, or if not, looking
for starting points as to how to go about doing it:

The idea would be to have some kind of tree implementation (e.g.
pointers or nested sets) for an SQL database and then to write a Linux
driver that would make it possible to create a device file so that you
could do something like

mount -t (specify filesystem: e.g. ext2, vfat) /dev/posttree /mountpoint

so that the SQL tree would look like a normal filesystem. Read-only
would be cool to start with, although it'd become really useful if you
had an rw implementation with permissions etc.. (Richard Jones has done
something which is kind of similar for his ftp server: You can use a
PostgreSQL database as a backend for it, rather than a filesystem:
http://www.cpan.org/modules/by-authors/id/R/RW/RWMJ/)

How difficult would that be? Where to start? Where to find code snippets
to work with? Which filesystem would be the most suited for an emulation
- ext2,vfat . .  . ? Whould that need to be done in C or could you write
a wrapper/driver in something like e.g. Perl?

The inspiration for this idea was Hans Reiser's manifesto 'The Naming
System Venture' where he argues that the future belongs to filesystems
with database-like extensions, rather than databases. He may be
right or not; but what kept me thinking above all was that I
do encounter the problem that he describes: Whenever I want to put
something into a database, or retrieve something from it, I am always
depending on more or less specialized interfaces (I use PHP) which may
not be available to some user at some point, and things then tend to
become cumbersome. Hans Reiser's argument is actually somewhat more
sophistaced and lenghty, which is why I am not trying to reproduce it
here. A tool like the one I tried describe would make it possible to
combine the flexibility of a filesystem with the more specialized
functionality of an SQL database.

Regards, Frank

Re: virtual filesystem atop a PostgreSQL database

From
Thomas Lockhart
Date:
> I am wondering whether anyone has already tried it, or if not, looking
> for starting points as to how to go about doing it:

I recall seeing an article in a magazine a couple of years ago by
someone using PostgreSQL to implement a versioning file system on Linux.
I *think* it was in Linux Journal.

                       - Thomas

Re: virtual filesystem atop a PostgreSQL database

From
Frank Joerdens
Date:
On Tue, Sep 25, 2001 at 02:45:37PM +0200, Jan Pruner wrote:
> Hmm, filesystem IS database.
> If you need SQL-like functionality to ask  for something in your fs why do
> you want to mount db like fs?
> You can build a sql-like shell !?! EnhancedBASH?

I couldn't possibly explain it any better than Hans Reiser (BTW, he's
the author of ReiserFS, just in case that is not known) in

http://www.namesys.com/whitepaper.html

In particular, look at the section entitled 'The Little Inconveniences
Dominate What We Do'. To be sure, what I suggested is emphatically NOT
what Reiser is aiming for; it just adresses some of the issues he raises.

Regards, Frank

Re: virtual filesystem atop a PostgreSQL database

From
Frank Joerdens
Date:
On Tue, Sep 25, 2001 at 01:29:58PM +0000, Thomas Lockhart wrote:
> > I am wondering whether anyone has already tried it, or if not, looking
> > for starting points as to how to go about doing it:
>
> I recall seeing an article in a magazine a couple of years ago by
> someone using PostgreSQL to implement a versioning file system on Linux.
> I *think* it was in Linux Journal.

Yep. Found it. It looks more or less like what I was suggesting.

http://www2.linuxjournal.com/lj-issues/issue42/1383.html

Unfortunately, the link to the sources is defunct. Does anyone still
have a copy of Pgfs (which is what the author, Brian Bartholomew, called
it)? I am going to write to the author as well.

Cheers, Frank

Re: virtual filesystem atop a PostgreSQL database

From
Jan Pruner
Date:
Hmm, filesystem IS database.
If you need SQL-like functionality to ask  for something in your fs why do
you want to mount db like fs?
You can build a sql-like shell !?! EnhancedBASH?


JP

On Tue 25. September 2001 14:22, you wrote:
> I am wondering whether anyone has already tried it, or if not, looking
> for starting points as to how to go about doing it:
>
> The idea would be to have some kind of tree implementation (e.g.
> pointers or nested sets) for an SQL database and then to write a Linux
> driver that would make it possible to create a device file so that you
> could do something like
>
> mount -t (specify filesystem: e.g. ext2, vfat) /dev/posttree /mountpoint
>
> so that the SQL tree would look like a normal filesystem. Read-only
> would be cool to start with, although it'd become really useful if you
> had an rw implementation with permissions etc.. (Richard Jones has done
> something which is kind of similar for his ftp server: You can use a
> PostgreSQL database as a backend for it, rather than a filesystem:
> http://www.cpan.org/modules/by-authors/id/R/RW/RWMJ/)
>
> How difficult would that be? Where to start? Where to find code snippets
> to work with? Which filesystem would be the most suited for an emulation
> - ext2,vfat . .  . ? Whould that need to be done in C or could you write
> a wrapper/driver in something like e.g. Perl?
>
> The inspiration for this idea was Hans Reiser's manifesto 'The Naming
> System Venture' where he argues that the future belongs to filesystems
> with database-like extensions, rather than databases. He may be
> right or not; but what kept me thinking above all was that I
> do encounter the problem that he describes: Whenever I want to put
> something into a database, or retrieve something from it, I am always
> depending on more or less specialized interfaces (I use PHP) which may
> not be available to some user at some point, and things then tend to
> become cumbersome. Hans Reiser's argument is actually somewhat more
> sophistaced and lenghty, which is why I am not trying to reproduce it
> here. A tool like the one I tried describe would make it possible to
> combine the flexibility of a filesystem with the more specialized
> functionality of an SQL database.
>
> Regards, Frank
>
> ---------------------------(end of broadcast)---------------------------
> TIP 4: Don't 'kill -9' the postmaster

Re: virtual filesystem atop a PostgreSQL database

From
Jan Pruner
Date:
Hmm, filesystem IS database.
If you need SQL-like functionality to ask  for something in your fs why do
you want to mount db like fs?
You can build a sql-like shell !?! EnhancedBASH?


JP

On Tue 25. September 2001 14:22, you wrote:
> I am wondering whether anyone has already tried it, or if not, looking
> for starting points as to how to go about doing it:
>
> The idea would be to have some kind of tree implementation (e.g.
> pointers or nested sets) for an SQL database and then to write a Linux
> driver that would make it possible to create a device file so that you
> could do something like
>
> mount -t (specify filesystem: e.g. ext2, vfat) /dev/posttree /mountpoint
>
> so that the SQL tree would look like a normal filesystem. Read-only
> would be cool to start with, although it'd become really useful if you
> had an rw implementation with permissions etc.. (Richard Jones has done
> something which is kind of similar for his ftp server: You can use a
> PostgreSQL database as a backend for it, rather than a filesystem:
> http://www.cpan.org/modules/by-authors/id/R/RW/RWMJ/)
>
> How difficult would that be? Where to start? Where to find code snippets
> to work with? Which filesystem would be the most suited for an emulation
> - ext2,vfat . .  . ? Whould that need to be done in C or could you write
> a wrapper/driver in something like e.g. Perl?
>
> The inspiration for this idea was Hans Reiser's manifesto 'The Naming
> System Venture' where he argues that the future belongs to filesystems
> with database-like extensions, rather than databases. He may be
> right or not; but what kept me thinking above all was that I
> do encounter the problem that he describes: Whenever I want to put
> something into a database, or retrieve something from it, I am always
> depending on more or less specialized interfaces (I use PHP) which may
> not be available to some user at some point, and things then tend to
> become cumbersome. Hans Reiser's argument is actually somewhat more
> sophistaced and lenghty, which is why I am not trying to reproduce it
> here. A tool like the one I tried describe would make it possible to
> combine the flexibility of a filesystem with the more specialized
> functionality of an SQL database.
>
> Regards, Frank
>
> ---------------------------(end of broadcast)---------------------------
> TIP 4: Don't 'kill -9' the postmaster

Re: virtual filesystem atop a PostgreSQL database

From
Jan Pruner
Date:
Hmm, filesystem IS database.
If you need SQL-like functionality to ask  for something in your fs why do
you want to mount db like fs?
You can build a sql-like shell !?! EnhancedBASH?


JP

On Tue 25. September 2001 14:22, you wrote:
> I am wondering whether anyone has already tried it, or if not, looking
> for starting points as to how to go about doing it:
>
> The idea would be to have some kind of tree implementation (e.g.
> pointers or nested sets) for an SQL database and then to write a Linux
> driver that would make it possible to create a device file so that you
> could do something like
>
> mount -t (specify filesystem: e.g. ext2, vfat) /dev/posttree /mountpoint
>
> so that the SQL tree would look like a normal filesystem. Read-only
> would be cool to start with, although it'd become really useful if you
> had an rw implementation with permissions etc.. (Richard Jones has done
> something which is kind of similar for his ftp server: You can use a
> PostgreSQL database as a backend for it, rather than a filesystem:
> http://www.cpan.org/modules/by-authors/id/R/RW/RWMJ/)
>
> How difficult would that be? Where to start? Where to find code snippets
> to work with? Which filesystem would be the most suited for an emulation
> - ext2,vfat . .  . ? Whould that need to be done in C or could you write
> a wrapper/driver in something like e.g. Perl?
>
> The inspiration for this idea was Hans Reiser's manifesto 'The Naming
> System Venture' where he argues that the future belongs to filesystems
> with database-like extensions, rather than databases. He may be
> right or not; but what kept me thinking above all was that I
> do encounter the problem that he describes: Whenever I want to put
> something into a database, or retrieve something from it, I am always
> depending on more or less specialized interfaces (I use PHP) which may
> not be available to some user at some point, and things then tend to
> become cumbersome. Hans Reiser's argument is actually somewhat more
> sophistaced and lenghty, which is why I am not trying to reproduce it
> here. A tool like the one I tried describe would make it possible to
> combine the flexibility of a filesystem with the more specialized
> functionality of an SQL database.
>
> Regards, Frank
>
> ---------------------------(end of broadcast)---------------------------
> TIP 4: Don't 'kill -9' the postmaster