Thread: pgxs: build infrastructure for extensions v1

Dear patchers,


Please find attached a patch which provides a working infrastructure for
pg extensions such as new gist-based indexes, functions, types...

The infrastructure is a simple reworking of the already available internal
infrastructure for contrib, so that it can be used outside of the
postgresql source tree after installation, without gory details being in
sight of the user...

As a test case and show how, I provided new Makefile.pgxs for most contrib
subdirectories. After postgresql has been configure, compiled and
installed, simply try them with:

    "cd contrib/foo ; make -f Makefile.pgxs install"


BEWARE: The patch does not include an update of "configure", as I do not
have the same version of "autoconf". So configure must be regenerated:
"autoconf configure.in > configure"

There is an initial documentation in "pgxs.sgml".


How it works:

 - necessary files (includes, scripts, makefiles...) are copied in lib/build
   on the initial "make install". The adds 2MB of stuff. The former
   installation can be obtained with "make light-install", but then
   no extensions could be added to the installation, so this is more for
   packagers that would like to provide a separate -dev package, IMHO.

   Due to gnu-make restriction on how its includes work, these files must
   be copied with the same directory structure as the pg source tree.
   The fact does not appear at all in the actual infrastructure from the
   user point of view, but it explains why subdirectories are necessary
   under the build subdir...

 - the makefile of any extension is expected to set macro PGXS to
   "pg_config --pgxs", to include a special pgxs.mk makefile, and to
   set some macros depending on what is to be built, just like in
   current contrib. See the examples under contrib.

 - I've added two PGXS-triggered conditionnals in Makefile.global,
   so that includes and libraries are taken where needed...


It works for me.


Questions and thoughts:

 - maybe more files should be copied? if so, which ones?

 - should this actually replace the current "contrib" infrastructure?

   I would tend to say "yes"...  that would require to replace all makefiles
   there by the Makefile.pgxs version, to remove "contrib-global.mk", and
   also possibly to update some documentations.

 - more documentation?
   integration of the documentation wrt pg doc?

 - validation? one could thing of installing some contribs
   when "make check" is performed, so as to validate that the
   extension infrastructure is not broken?


Have a nice day,


--
Fabien Coelho - coelho@cri.ensmp.fr

Have folks looked this over?  Is this the direction we want to go?

---------------------------------------------------------------------------

Fabien COELHO wrote:
>
> Dear patchers,
>
>
> Please find attached a patch which provides a working infrastructure for
> pg extensions such as new gist-based indexes, functions, types...
>
> The infrastructure is a simple reworking of the already available internal
> infrastructure for contrib, so that it can be used outside of the
> postgresql source tree after installation, without gory details being in
> sight of the user...
>
> As a test case and show how, I provided new Makefile.pgxs for most contrib
> subdirectories. After postgresql has been configure, compiled and
> installed, simply try them with:
>
>     "cd contrib/foo ; make -f Makefile.pgxs install"
>
>
> BEWARE: The patch does not include an update of "configure", as I do not
> have the same version of "autoconf". So configure must be regenerated:
> "autoconf configure.in > configure"
>
> There is an initial documentation in "pgxs.sgml".
>
>
> How it works:
>
>  - necessary files (includes, scripts, makefiles...) are copied in lib/build
>    on the initial "make install". The adds 2MB of stuff. The former
>    installation can be obtained with "make light-install", but then
>    no extensions could be added to the installation, so this is more for
>    packagers that would like to provide a separate -dev package, IMHO.
>
>    Due to gnu-make restriction on how its includes work, these files must
>    be copied with the same directory structure as the pg source tree.
>    The fact does not appear at all in the actual infrastructure from the
>    user point of view, but it explains why subdirectories are necessary
>    under the build subdir...
>
>  - the makefile of any extension is expected to set macro PGXS to
>    "pg_config --pgxs", to include a special pgxs.mk makefile, and to
>    set some macros depending on what is to be built, just like in
>    current contrib. See the examples under contrib.
>
>  - I've added two PGXS-triggered conditionnals in Makefile.global,
>    so that includes and libraries are taken where needed...
>
>
> It works for me.
>
>
> Questions and thoughts:
>
>  - maybe more files should be copied? if so, which ones?
>
>  - should this actually replace the current "contrib" infrastructure?
>
>    I would tend to say "yes"...  that would require to replace all makefiles
>    there by the Makefile.pgxs version, to remove "contrib-global.mk", and
>    also possibly to update some documentations.
>
>  - more documentation?
>    integration of the documentation wrt pg doc?
>
>  - validation? one could thing of installing some contribs
>    when "make check" is performed, so as to validate that the
>    extension infrastructure is not broken?
>
>
> Have a nice day,
>
>
> --
> Fabien Coelho - coelho@cri.ensmp.fr

Content-Description:

[ Attachment, skipping... ]

>
> ---------------------------(end of broadcast)---------------------------
> TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org

--
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Bruce Momjian <pgman@candle.pha.pa.us> writes:
> Have folks looked this over?  Is this the direction we want to go?

I've been hoping Peter would comment; he did most of the work on the
present contrib infrastructure and so probably has the best knowledge
of where the gotchas are.

In the absence of Peter looking it over, I will, but I have a few other
things to do first ...

            regards, tom lane

Fabien COELHO wrote:
> Please find attached a patch which provides a working infrastructure
> for pg extensions such as new gist-based indexes, functions, types...

Comments:

- Regarding GNUmakefile.in changes: files in directory foo/ must be
handled by a makefile in directory foo/, so these changes must be moved
to a makefile in config/.

- Please don't invent new targets like light-install, install-local.
Just install everything in the install target.

- The uninstall target may only remove what it installed, not "*".

- Don't invent new installation directories; there are enough already.
Platform dependent data files (which these are) go into/under
$(pkglibdir).

- Referring to the renaming of all makefiles under contrib/: I'm not in
favor of naming makefiles Makefile.foo; it should be foo.mk.  This
makes it easier for tools that recognize files by extension.

- I think this is a good approach

PGXS  := $(shell pg_config --pgxs)
include $(PGXS)/pgxs.mk

but why not

PGXS  := $(shell pg_config --pgxs)
include $(PGXS)

- Incidentally, the above will fail if pg_config is not installed (yet),
so it can't be used in the contrib/ directories.  (The contrib
directories are at least cleaned by the top-level makefile, this this
is a must-fix failure.)

- In Makefile.global: -L$(pkglibdir) is not necessary.  There are not
libraries to link at build time in there.

- -I$(includedir) and -L$(libdir) cannot be used.  In an average
installation that resolves to -I/usr/include and -L/usr/lib, which
breaks GCC and other compilers.

Dear Peter,

Thanks a lot for all these comments. I'll try to update my patch in the
coming week, or maybe this weekend.

Some responses to some of your comments:

> - Please don't invent new targets like light-install, install-local.
> Just install everything in the install target.

The current status is that there are two targets: "install" and
"server-install". As I think that "server-install" should be the default,
I renamed it "install", but I wanted to let the old "install" still
available, thus I had to find a name for it, hence "light-install". It
is the install a packager would like to chose, to have a separate -dev
package.

> - Referring to the renaming of all makefiles under contrib/: I'm not in
> favor of naming makefiles Makefile.foo; it should be foo.mk.  This
> makes it easier for tools that recognize files by extension.

Ok. Actually, the "Makefile.pgxs" is the template "example" makefile, so
its status is different from others that are to be included.

Thanks again,

--
Fabien Coelho - coelho@cri.ensmp.fr