Thread: module archive
From time to time people have raised the idea of a CPAN-like mechanism for downloading, building and installing extensions and the like (types, functions, sample dbs, anything not requiring Postgres itself to be rebuilt), and I have been thinking on this for the last few days. What sort of requirements would people have of such a mechanism? How do people envision it working? cheers andrew
On 10/25/07, Andrew Dunstan <andrew@dunslane.net> wrote: > > From time to time people have raised the idea of a CPAN-like mechanism > for downloading, building and installing extensions and the like (types, > functions, sample dbs, anything not requiring Postgres itself to be > rebuilt), and I have been thinking on this for the last few days. What > sort of requirements would people have of such a mechanism? How do > people envision it working? From a user's point of view, it would work like this : $ pgsql-pkg list Installed packages : ------------------------------ pgpool-II v.1.2.1 plR v.8.2.0.5 $ cd /usr/src/pgsql-packages/ $ cvs update $ cd languages/plR $ make upgrade $ cd ../../compatibility/orafce $ make install $ pgsql-pkg list Installed packages : ------------------------------ pgpool-II v.1.2.1 plR v.8.2.0.7 orafce v.2.1.2 A starting point worth considering could be pkgsrc , the packaging system used by NetBSD ( http://www.netbsd.org/docs/software/packages.html ) . Pkgsrc is not limited to NetBSD : it works on other OS'es as well (Linux, *BSD, Solaris, HP-UX, ... ) . The pkgsrc is based on a directory tree organized in categories. For each package there is a directory that contains the Makefile, the description, and sometimes patches . The user can do : make build , make install , make upgrade , etc. The sources are downloaded from the server where the project is originally hosted : NetBSD is not duplicating the source archives on their server. If necessary, the build system can apply patches before compiling the package. This allows the NetBSD folks to adapt the package without requiring modifications to the original sources So, the mechanism for PostgreSQL extensions could work this way : (A) The source tree would be an adapted version of pkgsrc : - the extensions are organized into directories (based on categories) - each extension would be a directory that basically contains a Makefile that contains enough information for downloading the source tarball, optionally apply patches, and install the package(extension). - probably it will be necessary to create a separate source tree for each PostgreSQL version , so that the build system wouldn't be cluttered with keeping information about which package version is compatible with which PostgreSQL version. - with such a system it is possible to include an extension even if the maintainer of that particular project doesn't modify anything for making it compatible with the extensions system . (B) Some extensions are only sql scripts that install new objects into a database, while others could also contain binaries, libraries, or maybe other stuff as well. Therefore, the installation probably has to be two-step : step 1 : 'make install' would copy the files in a new subdirectory called "extensions" : /usr/local/pgsql/extensions/bin /usr/local/pgsql/extensions/lib /usr/local/pgsql/extensions/install_scripts /usr/local/pgsql/extensions/remove_scripts /usr/local/pgsql/extensions/examples and add the extension to the list of 'installed extensions'. step 2 : for activating the extension inside a particular database the user would execute the extension's install_script. The first step is installing the extension at the database cluster level , while the second step is installing the extension inside a particular database. I think that this distinction is an important one : the user will need full control in installing the extensions only in the databases where she/he wants to. (C) It would be nice to have a table containing information about the extensions currently installed . Cheers, Adrian Maier
Am Donnerstag, 25. Oktober 2007 schrieb Andrew Dunstan: > From time to time people have raised the idea of a CPAN-like mechanism > for downloading, building and installing extensions and the like (types, > functions, sample dbs, anything not requiring Postgres itself to be > rebuilt), and I have been thinking on this for the last few days. What > sort of requirements would people have of such a mechanism? How do > people envision it working? Downloading, building, and installing extensions is actually fairly standardized already (well, perhaps there are 2 or 3 standards, but CPAN has that as well). I think the inhibitions relate more to the management of what is installed. I imagine we need a package manager inside of PostgreSQL to manage installation, setup, removal, dependencies and so on. Much like rpm or dpkg really. That should replace the current "run this .sql file" mechanism, much like rpm and dpkg replaced the "run make install and trust me" mechanism. I have some of this mapped out in my head if there is interest. We'd also need easy integration with the real rpm and dpkg, so that distribution packages can be built easily and I can run apt-get install postgresql extension1 extension2 and it's all set up. -- Peter Eisentraut http://developer.postgresql.org/~petere/
On Thu, Oct 25, 2007 at 12:31:30PM -0400, Andrew Dunstan wrote: > > From time to time people have raised the idea of a CPAN-like mechanism > for downloading, building and installing extensions and the like (types, > functions, sample dbs, anything not requiring Postgres itself to be > rebuilt), and I have been thinking on this for the last few days. What > sort of requirements would people have of such a mechanism? How do > people envision it working? There's been some discussion before: http://archives.postgresql.org/pgsql-patches/2006-05/msg00327.php The most useful feature IMHO would be if the installed modules could be recognised as a block (using pg_depends) so that: 1. pg_dump can see them and emit only INSTALL <module> 2. Uninstall removes everything that was installed This only covers installtion though, not building or compiling. Have a nice day, -- Martijn van Oosterhout <kleptog@svana.org> http://svana.org/kleptog/ > From each according to his ability. To each according to his ability to litigate.
Peter Eisentraut wrote: > Am Donnerstag, 25. Oktober 2007 schrieb Andrew Dunstan: >> From time to time people have raised the idea of a CPAN-like mechanism for >> downloading, building and installing extensions and the like (types, >> functions, sample dbs, anything not requiring Postgres itself to be >> rebuilt), and I have been thinking on this for the last few days. What sort >> of requirements would people have of such a mechanism? How do people >> envision it working? > > Downloading, building, and installing extensions is actually fairly > standardized already (well, perhaps there are 2 or 3 standards, but CPAN has > that as well). I think the inhibitions relate more to the management of > what is installed. > > I imagine we need a package manager inside of PostgreSQL to manage > installation, setup, removal, dependencies and so on. Much like rpm or dpkg > really. That should replace the current "run this .sql file" mechanism, > much like rpm and dpkg replaced the "run make install and trust me" > mechanism. I have some of this mapped out in my head if there is interest. The major challenge that I see is getting updates right, especially when the package/module contains tables which the user might have added data to (Like for example pre-8.3 tsearch). Both for updates of the packages, and for upgrading postgres to a new major revision. Maybe there are some schema-versioning tools available already that might help, though... > We'd also need easy integration with the real rpm and dpkg, so that > distribution packages can be built easily and I can run > > apt-get install postgresql extension1 extension2 Wow, that is ambitious ;-) I haven't yet seen a single distribution that gets this right for CPAN, ruby gems, or anything the like - if you know one, I'd be very interested in trying it out. Speaking for myself, I'd already be very happy if I could do apt-get install postgresql postgresql-pkg and then postpkg <database> install <whatever module>. That'd also allow postpkg to deal with the database-specific requirements of a package manager (Like specifying which db to add the module too). regards, Florian Pflug