Re: RFC: Additional Directory for Extensions - Mailing list pgsql-hackers

Hi Hackers,

Apologies for the delay in reply; I’ve been at the XOXO Festival and almost completely unplugged for the first time in
ages.Happy to see this thread coming alive, though. Thank you Gabriele, Craig, and Jelte! 

On Aug 21, 2024, at 19:07, Craig Ringer <craig.ringer@enterprisedb.com> wrote:

> So IMO this should be a _path_ to search for extension control files
> and SQL scripts.
>
> If the current built-in default extension dir was exposed as a var
> $extdir like we do for $libdir, this might look something like this
> for local development and testing while working with a packaged
> postgres build:
>
>    SET extension_search_path = $extsdir, /opt/myapp/extensions,
> /usr/local/postgres/my-custom-extension/extensions;
>    SET dynamic_library_path = $libdir, /opt/myapp/lib,
> /usr/local/postgres/my-custom-extension/lib

I would very much like something like this, but I’m not sure how feasible it is for a few reasons. The first, and most
important,is that extensions are not limited to just a control file and SQL file. They also very often include: 

* one or more shared library files
* documentation files
* binary files

And maybe more? How many of these directories might an extension install files into:

✦ ❯ pg_config | grep DIR | awk '{print $1}'
BINDIR
DOCDIR
HTMLDIR
INCLUDEDIR
PKGINCLUDEDIR
INCLUDEDIR-SERVER
LIBDIR
PKGLIBDIR
LOCALEDIR
MANDIR
SHAREDIR
SYSCONFDIR

I would assume BINDIR, DOCDIR, HTMLDIR, PKGLIBDIR, MANDIR, SHAREDIR, and perhaps LOCALEDIR.

But even if it’s just one or two, the only proper way an extension directory would work, IME, is to define a
directory-basedstructure for extensions, where every file for an extension is in a directory named for the extension,
andsubdirectories are defined for each of the above requisite file types. Something like: 

extension_name
├── control.ini
├── bin
├── doc
├── html
├── lib
├── local
├── man
└── share

This would allow multiple paths to work and keep all the files for an extension bundled together. It could also
potentiallyallow for multiple versions of an extension to be installed at once, if we required the version to be part
ofthe directory name. 

I think this would be a much nicer layout for packaging, installing, and managing extensions versus the current method
ofstrewing files around to a slew of different directories. But it would come at some cost, in terms of backward with
theexisting layout (or migration to it), significant modification of the server to use the new layout (and
extension_search_path),and other annoyances like PATH and MANPATH management. 

Long term I think it would be worthwhile, but the current patch feels like a decent interim step we could live with,
solvingmost of the integration problems (immutable servers, packaging testing, etc.) at the cost of a slightly
unexpecteddirectory layout. What I mean by that is that the current patch is pretty much just using extension_destdir
asa prefix to all of those directories from pg_config, so they never have to change, but it does mean that you end up
installingextensions in something like 

/mnt/extensions/pg16/usr/share/postgresql/16
/mnt/extensions/pg16/usr/include/postgresql

etc.

Best,

David