Thread: module archive

module archive

From
Andrew Dunstan
Date:
From time to time people have raised the idea of a CPAN-like mechanism 
for downloading, building and installing extensions and the like (types, 
functions, sample dbs, anything not requiring Postgres itself to be 
rebuilt), and I have been thinking on this for the last few days. What 
sort of requirements would people have of such a mechanism? How do 
people envision it working?

cheers

andrew


Re: module archive

From
"Adrian Maier"
Date:
On 10/25/07, Andrew Dunstan <andrew@dunslane.net> wrote:
>
>  From time to time people have raised the idea of a CPAN-like mechanism
> for downloading, building and installing extensions and the like (types,
> functions, sample dbs, anything not requiring Postgres itself to be
> rebuilt), and I have been thinking on this for the last few days. What
> sort of requirements would people have of such a mechanism? How do
> people envision it working?

From a user's point of view,  it would work like this :

$ pgsql-pkg   list
Installed packages :
------------------------------
pgpool-II      v.1.2.1
plR              v.8.2.0.5

$ cd /usr/src/pgsql-packages/
$ cvs  update

$ cd languages/plR
$ make upgrade

$ cd ../../compatibility/orafce
$ make install

$ pgsql-pkg   list
Installed packages :
------------------------------
pgpool-II     v.1.2.1
plR             v.8.2.0.7
orafce         v.2.1.2


A starting point worth considering could be pkgsrc , the packaging system used
by NetBSD   ( http://www.netbsd.org/docs/software/packages.html ) .
Pkgsrc is not limited to NetBSD :  it works on other OS'es  as well
(Linux, *BSD,
Solaris, HP-UX, ... ) .

The pkgsrc is  based on a directory tree organized in categories. For
each package
there is a directory that contains the Makefile, the description, and sometimes
patches .  The user can do :  make build , make install ,  make upgrade , etc.
The sources are downloaded from the server where the project is originally
hosted : NetBSD is not duplicating the source archives on their server.
If necessary,  the build system can apply patches before compiling the
package.  This allows the NetBSD folks to adapt the package without requiring
modifications to the original sources


So,  the mechanism for PostgreSQL extensions could work this way :
(A)  The source tree would be an adapted version of pkgsrc :
- the extensions are organized into directories (based on categories)

- each extension would be a directory that basically contains a Makefile
that contains enough information for downloading the source tarball,
optionally apply patches,  and install the package(extension).

- probably it will be necessary to create a separate source tree for each
PostgreSQL version , so that the build system wouldn't be cluttered with
keeping information about which package version is compatible with
which PostgreSQL version.

- with such a system it is possible to include an extension even if the
maintainer of that particular project doesn't modify anything for making it
compatible with the extensions system .

(B) Some extensions are only sql scripts that install new objects into
a database,
while others could also contain binaries, libraries, or maybe other
stuff as well.
Therefore, the installation probably has to be two-step :
step 1 :   'make install' would copy the files in a new subdirectory
called  "extensions"  :
/usr/local/pgsql/extensions/bin
/usr/local/pgsql/extensions/lib
/usr/local/pgsql/extensions/install_scripts
/usr/local/pgsql/extensions/remove_scripts
/usr/local/pgsql/extensions/examples
and add the extension to the list of 'installed extensions'.

step 2 : for activating the extension inside a particular database
the user would
execute the extension's install_script.

The first step is installing the extension at the database cluster
level ,  while the
second step is installing the extension inside a particular database.
I think that
this distinction is an important one :  the user will need full control in
installing the extensions only in the databases where she/he wants to.

(C) It would be nice to have a table containing information about the extensions
currently installed .


Cheers,
Adrian Maier


Re: module archive

From
Peter Eisentraut
Date:
Am Donnerstag, 25. Oktober 2007 schrieb Andrew Dunstan:
>  From time to time people have raised the idea of a CPAN-like mechanism
> for downloading, building and installing extensions and the like (types,
> functions, sample dbs, anything not requiring Postgres itself to be
> rebuilt), and I have been thinking on this for the last few days. What
> sort of requirements would people have of such a mechanism? How do
> people envision it working?

Downloading, building, and installing extensions is actually fairly 
standardized already (well, perhaps there are 2 or 3 standards, but CPAN has 
that as well).  I think the inhibitions relate more to the management of what 
is installed.

I imagine we need a package manager inside of PostgreSQL to manage 
installation, setup, removal, dependencies and so on.  Much like rpm or dpkg 
really.  That should replace the current "run this .sql file" mechanism, much 
like rpm and dpkg replaced the "run make install and trust me" mechanism.  I 
have some of this mapped out in my head if there is interest.

We'd also need easy integration with the real rpm and dpkg, so that 
distribution packages can be built easily and I can run

apt-get install postgresql extension1 extension2

and it's all set up.

-- 
Peter Eisentraut
http://developer.postgresql.org/~petere/


Re: module archive

From
Martijn van Oosterhout
Date:
On Thu, Oct 25, 2007 at 12:31:30PM -0400, Andrew Dunstan wrote:
>
> From time to time people have raised the idea of a CPAN-like mechanism
> for downloading, building and installing extensions and the like (types,
> functions, sample dbs, anything not requiring Postgres itself to be
> rebuilt), and I have been thinking on this for the last few days. What
> sort of requirements would people have of such a mechanism? How do
> people envision it working?

There's been some discussion before:
http://archives.postgresql.org/pgsql-patches/2006-05/msg00327.php

The most useful feature IMHO would be if the installed modules could be
recognised as a block (using pg_depends) so that:

1. pg_dump can see them and emit only INSTALL <module>
2. Uninstall removes everything that was installed

This only covers installtion though, not building or compiling.

Have a nice day,
--
Martijn van Oosterhout   <kleptog@svana.org>   http://svana.org/kleptog/
> From each according to his ability. To each according to his ability to litigate.

Re: module archive

From
"Florian G. Pflug"
Date:
Peter Eisentraut wrote:
> Am Donnerstag, 25. Oktober 2007 schrieb Andrew Dunstan:
>> From time to time people have raised the idea of a CPAN-like mechanism for
>> downloading, building and installing extensions and the like (types, 
>> functions, sample dbs, anything not requiring Postgres itself to be 
>> rebuilt), and I have been thinking on this for the last few days. What sort
>> of requirements would people have of such a mechanism? How do people
>> envision it working?
> 
> Downloading, building, and installing extensions is actually fairly 
> standardized already (well, perhaps there are 2 or 3 standards, but CPAN has
>  that as well).  I think the inhibitions relate more to the management of
> what is installed.
> 
> I imagine we need a package manager inside of PostgreSQL to manage 
> installation, setup, removal, dependencies and so on.  Much like rpm or dpkg
>  really.  That should replace the current "run this .sql file" mechanism,
> much like rpm and dpkg replaced the "run make install and trust me"
> mechanism.  I have some of this mapped out in my head if there is interest.

The major challenge that I see is getting updates right, especially when the
package/module contains tables which the user might have added data to (Like for
example pre-8.3 tsearch). Both for updates of the packages, and for upgrading
postgres to a new major revision.

Maybe there are some schema-versioning tools available already that might help,
though...

> We'd also need easy integration with the real rpm and dpkg, so that 
> distribution packages can be built easily and I can run
> 
> apt-get install postgresql extension1 extension2
Wow, that is ambitious ;-)
I haven't yet seen a single distribution that gets this right for CPAN, ruby
gems, or anything the like - if you know one, I'd be very interested in trying
it out.

Speaking for myself, I'd already be very happy if I could do
apt-get install postgresql postgresql-pkg
and then
postpkg <database> install <whatever module>.

That'd also allow postpkg to deal with the database-specific requirements of a
package manager (Like specifying which db to add the module too).

regards, Florian Pflug