Thread: virtual filesystem atop a PostgreSQL database
I am wondering whether anyone has already tried it, or if not, looking for starting points as to how to go about doing it: The idea would be to have some kind of tree implementation (e.g. pointers or nested sets) for an SQL database and then to write a Linux driver that would make it possible to create a device file so that you could do something like mount -t (specify filesystem: e.g. ext2, vfat) /dev/posttree /mountpoint so that the SQL tree would look like a normal filesystem. Read-only would be cool to start with, although it'd become really useful if you had an rw implementation with permissions etc.. (Richard Jones has done something which is kind of similar for his ftp server: You can use a PostgreSQL database as a backend for it, rather than a filesystem: http://www.cpan.org/modules/by-authors/id/R/RW/RWMJ/) How difficult would that be? Where to start? Where to find code snippets to work with? Which filesystem would be the most suited for an emulation - ext2,vfat . . . ? Whould that need to be done in C or could you write a wrapper/driver in something like e.g. Perl? The inspiration for this idea was Hans Reiser's manifesto 'The Naming System Venture' where he argues that the future belongs to filesystems with database-like extensions, rather than databases. He may be right or not; but what kept me thinking above all was that I do encounter the problem that he describes: Whenever I want to put something into a database, or retrieve something from it, I am always depending on more or less specialized interfaces (I use PHP) which may not be available to some user at some point, and things then tend to become cumbersome. Hans Reiser's argument is actually somewhat more sophistaced and lenghty, which is why I am not trying to reproduce it here. A tool like the one I tried describe would make it possible to combine the flexibility of a filesystem with the more specialized functionality of an SQL database. Regards, Frank
> I am wondering whether anyone has already tried it, or if not, looking
> for starting points as to how to go about doing it:
I recall seeing an article in a magazine a couple of years ago by
someone using PostgreSQL to implement a versioning file system on Linux.
I *think* it was in Linux Journal.
- Thomas
On Tue, Sep 25, 2001 at 02:45:37PM +0200, Jan Pruner wrote: > Hmm, filesystem IS database. > If you need SQL-like functionality to ask for something in your fs why do > you want to mount db like fs? > You can build a sql-like shell !?! EnhancedBASH? I couldn't possibly explain it any better than Hans Reiser (BTW, he's the author of ReiserFS, just in case that is not known) in http://www.namesys.com/whitepaper.html In particular, look at the section entitled 'The Little Inconveniences Dominate What We Do'. To be sure, what I suggested is emphatically NOT what Reiser is aiming for; it just adresses some of the issues he raises. Regards, Frank
On Tue, Sep 25, 2001 at 01:29:58PM +0000, Thomas Lockhart wrote: > > I am wondering whether anyone has already tried it, or if not, looking > > for starting points as to how to go about doing it: > > I recall seeing an article in a magazine a couple of years ago by > someone using PostgreSQL to implement a versioning file system on Linux. > I *think* it was in Linux Journal. Yep. Found it. It looks more or less like what I was suggesting. http://www2.linuxjournal.com/lj-issues/issue42/1383.html Unfortunately, the link to the sources is defunct. Does anyone still have a copy of Pgfs (which is what the author, Brian Bartholomew, called it)? I am going to write to the author as well. Cheers, Frank
Hmm, filesystem IS database. If you need SQL-like functionality to ask for something in your fs why do you want to mount db like fs? You can build a sql-like shell !?! EnhancedBASH? JP On Tue 25. September 2001 14:22, you wrote: > I am wondering whether anyone has already tried it, or if not, looking > for starting points as to how to go about doing it: > > The idea would be to have some kind of tree implementation (e.g. > pointers or nested sets) for an SQL database and then to write a Linux > driver that would make it possible to create a device file so that you > could do something like > > mount -t (specify filesystem: e.g. ext2, vfat) /dev/posttree /mountpoint > > so that the SQL tree would look like a normal filesystem. Read-only > would be cool to start with, although it'd become really useful if you > had an rw implementation with permissions etc.. (Richard Jones has done > something which is kind of similar for his ftp server: You can use a > PostgreSQL database as a backend for it, rather than a filesystem: > http://www.cpan.org/modules/by-authors/id/R/RW/RWMJ/) > > How difficult would that be? Where to start? Where to find code snippets > to work with? Which filesystem would be the most suited for an emulation > - ext2,vfat . . . ? Whould that need to be done in C or could you write > a wrapper/driver in something like e.g. Perl? > > The inspiration for this idea was Hans Reiser's manifesto 'The Naming > System Venture' where he argues that the future belongs to filesystems > with database-like extensions, rather than databases. He may be > right or not; but what kept me thinking above all was that I > do encounter the problem that he describes: Whenever I want to put > something into a database, or retrieve something from it, I am always > depending on more or less specialized interfaces (I use PHP) which may > not be available to some user at some point, and things then tend to > become cumbersome. Hans Reiser's argument is actually somewhat more > sophistaced and lenghty, which is why I am not trying to reproduce it > here. A tool like the one I tried describe would make it possible to > combine the flexibility of a filesystem with the more specialized > functionality of an SQL database. > > Regards, Frank > > ---------------------------(end of broadcast)--------------------------- > TIP 4: Don't 'kill -9' the postmaster
Hmm, filesystem IS database. If you need SQL-like functionality to ask for something in your fs why do you want to mount db like fs? You can build a sql-like shell !?! EnhancedBASH? JP On Tue 25. September 2001 14:22, you wrote: > I am wondering whether anyone has already tried it, or if not, looking > for starting points as to how to go about doing it: > > The idea would be to have some kind of tree implementation (e.g. > pointers or nested sets) for an SQL database and then to write a Linux > driver that would make it possible to create a device file so that you > could do something like > > mount -t (specify filesystem: e.g. ext2, vfat) /dev/posttree /mountpoint > > so that the SQL tree would look like a normal filesystem. Read-only > would be cool to start with, although it'd become really useful if you > had an rw implementation with permissions etc.. (Richard Jones has done > something which is kind of similar for his ftp server: You can use a > PostgreSQL database as a backend for it, rather than a filesystem: > http://www.cpan.org/modules/by-authors/id/R/RW/RWMJ/) > > How difficult would that be? Where to start? Where to find code snippets > to work with? Which filesystem would be the most suited for an emulation > - ext2,vfat . . . ? Whould that need to be done in C or could you write > a wrapper/driver in something like e.g. Perl? > > The inspiration for this idea was Hans Reiser's manifesto 'The Naming > System Venture' where he argues that the future belongs to filesystems > with database-like extensions, rather than databases. He may be > right or not; but what kept me thinking above all was that I > do encounter the problem that he describes: Whenever I want to put > something into a database, or retrieve something from it, I am always > depending on more or less specialized interfaces (I use PHP) which may > not be available to some user at some point, and things then tend to > become cumbersome. Hans Reiser's argument is actually somewhat more > sophistaced and lenghty, which is why I am not trying to reproduce it > here. A tool like the one I tried describe would make it possible to > combine the flexibility of a filesystem with the more specialized > functionality of an SQL database. > > Regards, Frank > > ---------------------------(end of broadcast)--------------------------- > TIP 4: Don't 'kill -9' the postmaster
Hmm, filesystem IS database. If you need SQL-like functionality to ask for something in your fs why do you want to mount db like fs? You can build a sql-like shell !?! EnhancedBASH? JP On Tue 25. September 2001 14:22, you wrote: > I am wondering whether anyone has already tried it, or if not, looking > for starting points as to how to go about doing it: > > The idea would be to have some kind of tree implementation (e.g. > pointers or nested sets) for an SQL database and then to write a Linux > driver that would make it possible to create a device file so that you > could do something like > > mount -t (specify filesystem: e.g. ext2, vfat) /dev/posttree /mountpoint > > so that the SQL tree would look like a normal filesystem. Read-only > would be cool to start with, although it'd become really useful if you > had an rw implementation with permissions etc.. (Richard Jones has done > something which is kind of similar for his ftp server: You can use a > PostgreSQL database as a backend for it, rather than a filesystem: > http://www.cpan.org/modules/by-authors/id/R/RW/RWMJ/) > > How difficult would that be? Where to start? Where to find code snippets > to work with? Which filesystem would be the most suited for an emulation > - ext2,vfat . . . ? Whould that need to be done in C or could you write > a wrapper/driver in something like e.g. Perl? > > The inspiration for this idea was Hans Reiser's manifesto 'The Naming > System Venture' where he argues that the future belongs to filesystems > with database-like extensions, rather than databases. He may be > right or not; but what kept me thinking above all was that I > do encounter the problem that he describes: Whenever I want to put > something into a database, or retrieve something from it, I am always > depending on more or less specialized interfaces (I use PHP) which may > not be available to some user at some point, and things then tend to > become cumbersome. Hans Reiser's argument is actually somewhat more > sophistaced and lenghty, which is why I am not trying to reproduce it > here. A tool like the one I tried describe would make it possible to > combine the flexibility of a filesystem with the more specialized > functionality of an SQL database. > > Regards, Frank > > ---------------------------(end of broadcast)--------------------------- > TIP 4: Don't 'kill -9' the postmaster