Thread: Configurable path to look up dynamic libraries

About to be implemented, for your approval...

Variable name: dynamic_library_path

Permissions: superuser

Default value: empty string

Specification:

When the dynamic loader attempts to load a file (initiated by create
function, for example) and the file name does not contain a slash
(anywhere) and this variable is not set to the empty string, the dynamic
loader will look for the file in the search path specified by this
variable.

The search path is the usual colon-separated style.  Empty components will
be ignored.  If the directory name is not absolute, an error will be
raised.

If no appropriate file is found in this path walk, the dynamic loader will
try to load the file as given, which may invoke a system-dependent lookup
mechanism (e.g., LD_LIBRARY_PATH).

(The fine points of this specification are intended to be compatible with
Libtool's libltdl dynamic loading interface.)

-- 
Peter Eisentraut   peter_e@gmx.net   http://funkturm.homeip.net/~peter

Peter Eisentraut <peter_e@gmx.net> writes:
> About to be implemented, for your approval...
> Variable name: dynamic_library_path
> Permissions: superuser
> Default value: empty string

This is of little value unless the default is intelligently chosen.
The default should be "$PGLIB", IMHO (inserted from configure's data).
Unless there is a usable default, we cannot start recommending that
people not use absolute paths in CREATE FUNCTION commands.

I do not believe that it's a good idea to allow the value to be changed
at runtime, either --- do you expect that backends will remove
already-loaded libraries in response to a change in the variable?
I think setting the path at postmaster start is necessary and sufficient.

Also, it'd be really nice if a platform-dependent suffix (.so, .sl,
etc) were automatically added to the given library name, if one's not
present already.
        regards, tom lane

Tom Lane writes:

> This is of little value unless the default is intelligently chosen.
> The default should be "$PGLIB", IMHO (inserted from configure's data).

This default has little value as well.  Users don't generally put their
loadable modules in the same directory as a PostgreSQL installation.
Maybe they do for general-purpose contrib-like stuff, but then they might
as well use an absolute path. (Remember that a PostgreSQL installation
could well be under /usr/lib; think of all the things that reside there
that we have no business with.)

This also ties in somewhat with the fact that we have no default for
PGDATA, on purpose.  If we can have arbitrarily located data locations the
system should not have a hard-wired in default for libraries (which are
usually tied to particular databases in particular database clusters).

> I do not believe that it's a good idea to allow the value to be changed
> at runtime, either --- do you expect that backends will remove
> already-loaded libraries in response to a change in the variable?

No, I would expect it to use the path for loading new libraries from then
on.  People that use loadable libraries and C functions are superusers and
experienced enough to cope with this little (logical) fact.  (Analogy:
When I change the PATH in my shell, the shell does not kill all processes
already running.)

The way I think this is most useful is in third-party provided
load_all_my_stuff.sql scripts, like:

set dynamic_library_path='/usr/local/foo/lib'; -- inserted by the package's build process

create function foo_xxx() ...

(Yes, you could do the same "inserted by package's build process" into
each of the create function's, but this way it's much cleaner.)

I also envision this to be used as part of dump/restore.  pg_dump might
have an option "do not dump full path", and it would insert a 'set
dynamic_library_path'.  This would work like the previous case, really.

Also think of a developer wanting to try out different sets of libraries
with a common load script.

If we make this parameter postmaster start only then we really don't gain
anything.  We don't even gain the minimal expected convenience in pg_dump
because you would force all modules to reside in a certain place where
administrators would least like them to be.


> Also, it'd be really nice if a platform-dependent suffix (.so, .sl,
> etc) were automatically added to the given library name, if one's not
> present already.

Yes.

-- 
Peter Eisentraut   peter_e@gmx.net   http://funkturm.homeip.net/~peter

Peter Eisentraut <peter_e@gmx.net> writes:
> Tom Lane writes:
>> This is of little value unless the default is intelligently chosen.
>> The default should be "$PGLIB", IMHO (inserted from configure's data).

> This default has little value as well.  Users don't generally put their
> loadable modules in the same directory as a PostgreSQL installation.

That's a sweeping statement with little to back it up.  How do you know
that the usual procedure isn't to put things in $PGLIB?  That's
certainly what all our contrib packages do.  Even more to the point,
that's certainly where the PL call handler functions are.  I will
consider this feature utterly without value unless it allows the
standard declaration of plpgsql_call_handler to become
installation-independent, vizCREATE FUNCTION plpgsql_call_handler () RETURNS OPAQUEAS 'plpgsql' LANGUAGE 'C';

> This also ties in somewhat with the fact that we have no default for
> PGDATA, on purpose.  If we can have arbitrarily located data locations the
> system should not have a hard-wired in default for libraries (which are
> usually tied to particular databases in particular database clusters).

I'd be willing to accept a default path that points to somewhere under
$PGDATA, although I consider this rather less useful.  Maybe we could
agree on a compromise two-entry default path: "$PGDATA/functions:$PGLIB"?
That would require some initdb-time shenanigans to set up, but if you
want to do it...

> I also envision this to be used as part of dump/restore.  pg_dump might
> have an option "do not dump full path", and it would insert a 'set
> dynamic_library_path'.  This would work like the previous case, really.

What?  What value would it have for pg_dump to do a set path operation?
The dump script would be unlikely to actually invoke any of the
functions it's loading.  By the time anyone tries to use the functions,
they'd be running in a different backend with a different path setting,
namely the default for the installation.

> If we make this parameter postmaster start only then we really don't gain
> anything.  We don't even gain the minimal expected convenience in pg_dump
> because you would force all modules to reside in a certain place where
> administrators would least like them to be.

I fail to follow this claim also.  The point as far as I'm concerned is
that paths mentioned in CREATE FUNCTION ought to be relative to
someplace that's installation-dependent.  That way, when you dump out
and reload a CREATE FUNCTION command, the declaration is still good,
you just have to have put a copy of the function's shlib in the right
place for the new installation.
        regards, tom lane

Tom Lane writes:

> The point as far as I'm concerned is that paths mentioned in CREATE
> FUNCTION ought to be relative to someplace that's
> installation-dependent.  That way, when you dump out and reload a
> CREATE FUNCTION command, the declaration is still good, you just have
> to have put a copy of the function's shlib in the right place for the
> new installation.

Okay, I'm convinced that $libdir can be a useful default.  But given the
case where users might want to *add* his directory to the path he needs to
have knowledge of what the default path is.  (Unfortunately we can't do
PATH=$PATH:xxx.)  Perhaps it would be good to make the empty path
component equivalent to $libdir, e.g.,

''            default, search libdir
':/my/own'        search libdir before my own
'/my/own:'        search libdir after my own
'/my/own'        don't seach libdir

But I think there are enough possibly useful applications for changing
this while the postmaster is running at no real harm.

-- 
Peter Eisentraut   peter_e@gmx.net   http://funkturm.homeip.net/~peter

Peter Eisentraut <peter_e@gmx.net> writes:
> Perhaps it would be good to make the empty path
> component equivalent to $libdir, e.g.,

Hmm, that would work, and also avoid having to figure out how to stuff
$PGLIB into postgresql.conf during initdb.

Sold as far as I'm concerned ...
        regards, tom lane

Tom Lane <tgl@sss.pgh.pa.us> writes:

> Peter Eisentraut <peter_e@gmx.net> writes:
> > Perhaps it would be good to make the empty path
> > component equivalent to $libdir, e.g.,
> 
> Hmm, that would work, and also avoid having to figure out how to stuff
> $PGLIB into postgresql.conf during initdb.

While on the subject of postgresql conf... shouldn't it be in
sysconfdir instead of the database directory? And there's no switch to
the postmaster to tell it you've put it somewhere else either.

-- 
Trond Eivind Glomsrød
Red Hat, Inc.

Tom Lane <tgl@sss.pgh.pa.us> writes:

> teg@redhat.com (Trond Eivind Glomsrød) writes:
> > While on the subject of postgresql conf... shouldn't it be in
> > sysconfdir instead of the database directory?
> 
> No.  That would (a) not allow different postmasters to have different
> config files;

You could search in a path... first sysconfdir, then datadir. 

> (b) not allow a person to create an unprivileged
> installation (assuming that sysconfdir is root-owned).

Sysconfdir defaults to $prefix/etc, so that's not a problem.

-- 
Trond Eivind Glomsrød
Red Hat, Inc.

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Tuesday 15 May 2001 14:44, Trond Eivind Glomsrød wrote:
> Tom Lane <tgl@sss.pgh.pa.us> writes:
> > Peter Eisentraut <peter_e@gmx.net> writes:
> > > Perhaps it would be good to make the empty path
> > > component equivalent to $libdir, e.g.,

> > Hmm, that would work, and also avoid having to figure out how to stuff
> > $PGLIB into postgresql.conf during initdb.

> While on the subject of postgresql conf... shouldn't it be in
> sysconfdir instead of the database directory? And there's no switch to
> the postmaster to tell it you've put it somewhere else either.

While I understand and, to an extent, agree with this sentiment, it would be 
unworkable at present unless the postgresql.conf file contained constructs 
that differentiated between multiple datadirs.  While the RPM currently 
doesn't support that possibility (not that it will never support such a 
possibility :-)), there are many who do use PostgreSQL with multiple 
postmasters and datadirs.

I personally wouldn't mind a construct similar to that of a webserver that 
supported multiple domain hosting -- you have a master config file or config 
file section that is in a standard place, and you have either a separate 
config file or config file section for each datadir -- in the case of the 
multiple config files, the master would point to each one.

BUT, given the current mindset in the postgresql.conf file, keeping it with 
the datadir is presently the only practical option.
- --
Lamar Owen
WGCR Internet Radio
1 Peter 4:11
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.4 (GNU/Linux)
Comment: For info see http://www.gnupg.org

iD8DBQE7AXvy5kGGI8vV9eERAhS+AKDGhUchvGN5AEkBqE11wEq8xskrGwCgmIFs
FEDp+xn6e9rdVskMOlhtEKI=
=B0tI
-----END PGP SIGNATURE-----

teg@redhat.com (Trond Eivind Glomsrød) writes:
> You could search in a path... first sysconfdir, then datadir. 

Surely the other way around.
        regards, tom lane

teg@redhat.com (Trond Eivind Glomsrød) writes:
> Tom Lane <tgl@sss.pgh.pa.us> writes:
>> teg@redhat.com (Trond Eivind Glomsrød) writes:
> You could search in a path... first sysconfdir, then datadir. 
>> 
>> Surely the other way around.

> Which could work as well - or just a switch to postmaster to tell it
> which file to use.

I could live with a datadir-then-sysconfdir path search.  (It should be
datadir first, since the sysconfdir file would serve as a system-wide
default for multiple postmasters.)  Given that approach I see no real
need for a postmaster switch.

Possibly the same approach should apply to all the config files we
currently store in datadir?

There is a security issue here: stuff stored in datadir is not visible
to random other users on the machine (since datadir is mode 700), but
I would not expect sysconfdir to be mode 700.  We'd need to think about
the implications of allowing Postgres config files to be world-visible.
        regards, tom lane

Tom Lane <tgl@sss.pgh.pa.us> writes:

> teg@redhat.com (Trond Eivind Glomsrød) writes:
> > Tom Lane <tgl@sss.pgh.pa.us> writes:
> >> teg@redhat.com (Trond Eivind Glomsrød) writes:
> > You could search in a path... first sysconfdir, then datadir. 
> >> 
> >> Surely the other way around.
> 
> > Which could work as well - or just a switch to postmaster to tell it
> > which file to use.
> 
> I could live with a datadir-then-sysconfdir path search.  (It should be
> datadir first, since the sysconfdir file would serve as a system-wide
> default for multiple postmasters.)  Given that approach I see no real
> need for a postmaster switch.
> 
> Possibly the same approach should apply to all the config files we
> currently store in datadir?
> 
> There is a security issue here: stuff stored in datadir is not visible
> to random other users on the machine (since datadir is mode 700), but
> I would not expect sysconfdir to be mode 700.  

It could be (the RPMs specify a sysconfdir of /etc/pgsql)

> We'd need to think about the implications of allowing Postgres
> config files to be world-visible. 

The files doesn't need to be visible to others...

-- 
Trond Eivind Glomsrød
Red Hat, Inc.

Tom Lane <tgl@sss.pgh.pa.us> writes:

> teg@redhat.com (Trond Eivind Glomsrød) writes:
> > You could search in a path... first sysconfdir, then datadir. 
> 
> Surely the other way around.

Which could work as well - or just a switch to postmaster to tell it
which file to use.
-- 
Trond Eivind Glomsrød
Red Hat, Inc.

teg@redhat.com (Trond Eivind Glomsrød) writes:
>> There is a security issue here: stuff stored in datadir is not visible
>> to random other users on the machine (since datadir is mode 700), but
>> I would not expect sysconfdir to be mode 700.  

> It could be (the RPMs specify a sysconfdir of /etc/pgsql)

The usual install procedure would probably leave sysconfdir owned by
root, if one likes to install in such a way that the binaries are owned
by root (ie make, su root, make install).  I'd object to a setup that's
insecure for people who aren't using RPMs.

The real bottom line here, though, is that you haven't shown me any
positive reason to move the config files out of datadir.  They're not
broken where they are; and arguably they *are* data.
        regards, tom lane

Trond Eivind Glomsrød writes:

> Tom Lane <tgl@sss.pgh.pa.us> writes:
>
> > teg@redhat.com (Trond Eivind Glomsrød) writes:
> > > You could search in a path... first sysconfdir, then datadir.
> >
> > Surely the other way around.
>
> Which could work as well - or just a switch to postmaster to tell it
> which file to use.

Might as well use a symlink in this case.

I could go for a solution that processed both files in order (possibly
even ${sysconfdir}/postgresql.conf, $PGDATA/postgresql.conf,
${sysconfdir}/postgresql.conf.fixed in order, à la PINE).  It could be as
easy as adding two or three lines in postmaster.c.  However, I'm afraid
users will interpret a file in $sysconfdir as something clients should
process as well, which is not part of this deal.

-- 
Peter Eisentraut   peter_e@gmx.net   http://funkturm.homeip.net/~peter

teg@redhat.com (Trond Eivind Glomsrød) writes:
>> The real bottom line here, though, is that you haven't shown me any
>> positive reason to move the config files out of datadir. 

> It conflicts with the FHS -

AFAIK, the FHS is not designed to support multiple instances of
unprivileged daemons.  I'm not interested in forcing Postgres into
that straitjacket ...
        regards, tom lane

Tom Lane <tgl@sss.pgh.pa.us> writes:

> teg@redhat.com (Trond Eivind Glomsrød) writes:
> >> There is a security issue here: stuff stored in datadir is not visible
> >> to random other users on the machine (since datadir is mode 700), but
> >> I would not expect sysconfdir to be mode 700.  
> 
> > It could be (the RPMs specify a sysconfdir of /etc/pgsql)
> 
> The usual install procedure would probably leave sysconfdir owned by
> root, if one likes to install in such a way that the binaries are owned
> by root (ie make, su root, make install).  I'd object to a setup that's
> insecure for people who aren't using RPMs.

So make the files unreadable, if so required.

> The real bottom line here, though, is that you haven't shown me any
> positive reason to move the config files out of datadir. 

It conflicts with the FHS - and no, I don't consider configuration
files and data as an identical item. 

-- 
Trond Eivind Glomsrød
Red Hat, Inc.

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Tuesday 15 May 2001 16:12, Tom Lane wrote:
> teg@redhat.com (Trond Eivind Glomsrød) writes:
> >> The real bottom line here, though, is that you haven't shown me any
> >> positive reason to move the config files out of datadir.

> > It conflicts with the FHS -

> AFAIK, the FHS is not designed to support multiple instances of
> unprivileged daemons.

Really?  I've never read that into the FHS.

A '/etc/pgsql' directory can easily accomodate multiple config files, or a 
single config file with multiple sections.  It can just as easily have a 
whole tree of stuff under it.  And it can be owned by a non-privileged user. 
It just cannot be installed into without root's help -- and I know that that 
is the main objection here.

But I have a hard time understanding this all-or-nothing approach.  So what 
if we have a configure-time option to allow a more FHS-compliant approach?  
It won't interfere with the 'traditional' /usr/local/pgsql or /opt/pgsql or 
whatever way of doing things.  Further, it won't uglify the code in the least 
- -- it just allows more choice.  Further, it won't take a hacker long to make 
it work.  It won't even touch 'real' backend code, either -- this is a 
postmaster thingy. 

What's the opposition about?  We have all the configure options already for 
many things that the 'traditional' postgresql user cares nothing about.

But, on a more pragmatic note, I am contemplating the ability for the RPM's 
initscript to allow multiple postmasters -- even up to sane behavior in the 
presence of postmasters that it didn't start -- with multiple datadirs, etc. 

I don't want the RPM's to 'editorialize' any more than anyone else might -- 
but unless a more FHS-compliant approach is at least _allowed_ (NOT 
mandated!), I guess there will need to be some editorializing in the 
intscript as it will have to place its own config file somewhere in order to 
make multiple postmasters happen.

But I don't want to go through that if I don't have to -- or if it's going to 
happen anyway.

And, I know, currently the '-D' postmaster directive does, indirectly, point 
to the location of postgresql.conf..... :-) I will be using that in the 
initscript's logic if another option isn't done.

But, if I may editorialize a little myself, this is just indicative of a 
'Fortress PostgreSQL' attitude that is easy to get into.  'We've always done 
it this way' or  'This is the way PostgreSQL works' are pat answers to 
anything from 'Why can't I more smoothly upgrade?' to 'Why does PostgreSQL 
use non-SQL-standard case-folding?' to 'Why does everything go in 
/usr/local/pgsql?' to 'Do I _really_ have to do an ASCII dump of my 100GB 
database in order to go to the next major version?' to any number of other 
FAQ's.

Just because we've always done it one way does not that one way correct make.

We're one component of a system -- and the PostgreSQL Group has done such a 
good job of being platform agnostic that the platform and systems issues are 
almost second-class citizens.

Well, gotta get off the soap box now, and get to work producing some code, I 
guess. People are going to be expecting multiple postmaster support in the 
RPMset soon. :-)
- --
Lamar Owen
WGCR Internet Radio
1 Peter 4:11
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.4 (GNU/Linux)
Comment: For info see http://www.gnupg.org

iD8DBQE7AZ+15kGGI8vV9eERAu+MAJ9bk8mY8n1qIk8zKqWM1K188/530wCeJnwd
ZZDjAosFhRnTENBWJ+THju4=
=mPC9
-----END PGP SIGNATURE-----

> But, if I may editorialize a little myself, this is just indicative of a 
> 'Fortress PostgreSQL' attitude that is easy to get into.  'We've always done 

I have to admit I like the sound of 'Fortress PostgreSQL'.  :-)

--  Bruce Momjian                        |  http://candle.pha.pa.us pgman@candle.pha.pa.us               |  (610)
853-3000+  If your life is a hard drive,     |  830 Blythe Avenue +  Christ can be your backup.        |  Drexel Hill,
Pennsylvania19026
 

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Tuesday 15 May 2001 17:53, Bruce Momjian wrote:
> > But, if I may editorialize a little myself, this is just indicative of a
> > 'Fortress PostgreSQL' attitude that is easy to get into.  'We've always
> > done

> I have to admit I like the sound of 'Fortress PostgreSQL'.  :-)

I don't moonlight as an English professor for nothing! :-)

I figured that phrase might get attention.......  Just a friendly dig.
- --
Lamar Owen
WGCR Internet Radio
1 Peter 4:11
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.4 (GNU/Linux)
Comment: For info see http://www.gnupg.org

iD8DBQE7AalJ5kGGI8vV9eERAtG4AJwKR/28NtQMWQ5LgfXaegbzq/jO9ACgxs0E
/DH7QEhwqV2jqHZ9wF5TG3c=
=YLQ3
-----END PGP SIGNATURE-----

Lamar Owen <lamar.owen@wgcr.org> writes:
> Just because we've always done it one way does not that one way correct make.

Sure.

> We're one component of a system -- and the PostgreSQL Group has done such a 
> good job of being platform agnostic that the platform and systems issues are 
> almost second-class citizens.

Indeed, that I think is the underlying issue here.  "It's FHS compliant"
cuts no ice with people who don't run FHS-layout systems, and I don't
want to improve FHS compliancy at the price of making life more
difficult for others.  (Likewise for other RPM installation issues, as
you well know ;-))

I do think that the notion of a configure file path search (datadir then
sysconfdir) is reasonable if the security and file protection issues can
be ironed out.  But that will require some thought about separating
security-critical data from not-critical data.  I think we ought to keep
pg_hba.conf and subsidiary files (especially password files!) in datadir
*only*.  I'm not sure about the other config files; up to now no one's
paid any attention to security issues for those files, knowing that they
were all kept in the same place.  We might need to reorganize their
contents.
        regards, tom lane

Lamar Owen writes:

> What's the opposition about?

The /etc directory is for system configuration files.  The
$PGDATA/postgresql.conf file is a database cluster configuration file.

The RPM set only allows one database cluster per system, so it's
appropriate to think of this database cluster as "the" system database
cluster, and of the associated configuration file as "the" system
configuration file.  But since the RPM set creates this situation it is
only fitting that the RPM set resolve this situation.  For example, it
would be trivial to symlink the file /var/lib/pgsql/data/postgresql.conf
to somewhere in /etc.

-- 
Peter Eisentraut   peter_e@gmx.net   http://funkturm.homeip.net/~peter

On Tue, May 15, 2001 at 05:53:36PM -0400, Bruce Momjian wrote:
> > But, if I may editorialize a little myself, this is just indicative of a 
> > 'Fortress PostgreSQL' attitude that is easy to get into.  'We've always
> 
> I have to admit I like the sound of 'Fortress PostgreSQL'.  :-)

Ye Olde PostgreSQL Shoppe
The PostgreSQL of Giza
Our Lady of PostgreSQL, Ascendant
PostgreSQL International Airport
PostgreSQL Galactica
PostgreSQL's Tavern

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Tuesday 15 May 2001 18:23, Tom Lane wrote:
> Lamar Owen <lamar.owen@wgcr.org> writes:
> > We're one component of a system -- and the PostgreSQL Group has done such
> > a good job of being platform agnostic that the platform and systems
> > issues are almost second-class citizens.

> Indeed, that I think is the underlying issue here.  "It's FHS compliant"
> cuts no ice with people who don't run FHS-layout systems, and I don't
> want to improve FHS compliancy at the price of making life more
> difficult for others.  

But that's my point -- it wouldn't, AFAICT, make anyone's life more diffcult 
in the userbase. The existing layout would remain as the default, with the 
FHS behavior a configure-time or run-time option.  Even being configured in 
an FHS compliant mode would not need to force FHS-compliance, just allow it.

But we currently are forcing FHS noncompliance on people. 

>(Likewise for other RPM installation issues, as
> you well know ;-))

As well I know.... ;-)

> But that will require some thought about separating
> security-critical data from not-critical data.  I think we ought to keep
> pg_hba.conf and subsidiary files (especially password files!) in datadir
> *only*.  I'm not sure about the other config files; up to now no one's
> paid any attention to security issues for those files, knowing that they
> were all kept in the same place.  We might need to reorganize their
> contents.

Good points.  However, now that we have breached the issue, GUC in al reality 
doesn't 100% unify configuration.  There's the sticky pg_hba.conf issue, the 
password files issue, etc.

<BRAIN_STORM>

So, what you have are basically two sets of config data:
1.)    Sitewide multiple-postmaster settings
2.)    Per-postmaster (and therefore per-datadir) settings.

There can be overlap here.  While I might want the capability to syslog be 
sitewide, the syslog level and facility might be a per-postmaster thing.  I 
might not want any logging on certain thoroughly tested high-volume databases 
(such as those that back discussion forums on websites). Likewise with TCP 
connections.  address:port and datadir settings are of course a per-datadir 
setting.  

I personally like the config file structure for the web statistics package 
'analog'.  First, there are configured-in defaults set in a header file.  
Then there is the master config file.  But you can then specify a secondary 
config file on the command line that automatically includes the master 
config, but can then override any setting at will.  You can also specify 
where to find the master config.

HOWEVER, while we are somewhat unique amongst databases in that there is 
built-in tcp-wrappers-like functionality, maybe, just maybe, that needs to be 
looked at a little closer.

SO, if postgresql.conf is already sitting in datadir, why not include the 
pg_hba.conf settings in the GUC?  Passwords could be stored similarly to the 
/etc/passwd method in a postgresql.conf section.  If we're going to have 
Unified Configuration, why not go whole-hog on it and really unify the 
configuration?  The default will be to place this file in the datadir, which 
by default is mode 700, so, the default installation will be secure.

Of course, currently postgresql.conf doesn't really have 'sections' except in 
style.  The '.ini' format is well-known and easily parsed, and is common on 
multiple platforms.  But in today's climate, an XML config might be better 
received.  

All that's left to do to satisfy my wish-list is an include directive (which 
may already exist -- the similarity to C headers in that file is striking) 
and a command-line switch to postmaster (or pg_ctl) to grab this file from 
another location, with the default being $PGDATA or the -D setting, if 
specified. With an include directive, a master config setup is easily made 
without unnecessarily complicating matters.  Oh, and tell initdb to not 
overwrite a postgresql.conf that already exists. :-).

Making this work with a single master config and multiple postmasters would 
be easily accomplished by having the ability to 'name' the 'virtual' 
databases and specify the settings for that virtual database in its own named 
section.  You then can start postmaster with pg_ctl and specify a 
- --config=/etc/pgsql/postgresql.conf --name=client1 and get the right datadir 
and the right settings.  Default settings set in the unnamed portion would be 
overridden by the named section's settings.

But, you could just as easily use individual configs that included a master 
(or not) and not use 'named' posmasters.  And the config files could reside 
wherever the admin wanted them to reside.

It's not far from this to making pg_ctl have the ability to start all the 
postmasters in a given config file with one command.  Directives to initdb if 
the datadir doesn't have a database present would simplify installation and 
initial startup, particularly for newbies.

</BRAIN_STORM>

And all of this would be optional usage. You don't use the feature?  The 
existing behavior would be the only reasonable default.

And symlinks are just a Band-Aid patch, and not a solution, as trivial as 
they may be.
- --
Lamar Owen
WGCR Internet Radio
1 Peter 4:11
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.4 (GNU/Linux)
Comment: For info see http://www.gnupg.org

iD8DBQE7Aqck5kGGI8vV9eERAjmfAKDPY2U41TA5rvxEzG/eyo2TfjknmQCeOpmx
1XzcsCRzQ7Eq9p3fagQJSVY=
=SOxC
-----END PGP SIGNATURE-----

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Tuesday 15 May 2001 18:27, Peter Eisentraut wrote:
> Lamar Owen writes:
> > What's the opposition about?

> The /etc directory is for system configuration files.  The
> $PGDATA/postgresql.conf file is a database cluster configuration file.

Which is part of the system.  /etc/mail/aliases is only an email config, but 
is system-wide.  I have multiple bind instances running on my main server -- 
it was relatively easy to tell bind through named.conf where to find the 
particular zone files for the private side (I run NAT here and must maintain 
an inside global DNS as well as an inside local DNS), and it was just as easy 
to tell named to use named.conf.private for the private DNS side.  And all 
those files reside in /etc/named and /etc/named.private.

> The RPM set only allows one database cluster per system, so it's
> appropriate to think of this database cluster as "the" system database
> cluster, and of the associated configuration file as "the" system
> configuration file.  But since the RPM set creates this situation it is
> only fitting that the RPM set resolve this situation.  For example, it
> would be trivial to symlink the file /var/lib/pgsql/data/postgresql.conf
> to somewhere in /etc.

I can resolve the RPM issues.  But, since talk is being made of changing the 
core behavior, I wanted to weigh in on what I'd like to see.  I may not have 
high expectations of what I am likely to actually see happen, but the hope IS 
there.  If the changes are shot down, I can still cope with the issue.

But symlinks aren't the fix, as this is not an RPM-only issue -- there are 
more than just RPM users who might want an FHS-compliant installation with 
the capacity for multiple postmasters.
- --
Lamar Owen
WGCR Internet Radio
1 Peter 4:11
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.4 (GNU/Linux)
Comment: For info see http://www.gnupg.org

iD8DBQE7AqlW5kGGI8vV9eERAqHZAJ4lJW0ndi+0aSSu5GQu12yAPkEDvACg4w9u
VfLdVIODenUU1GL4K4kf9OU=
=OtDH
-----END PGP SIGNATURE-----

Lamar Owen writes:

> I have multiple bind instances running on my main server -- it was
> relatively easy to tell bind through named.conf where to find the
> particular zone files for the private side (I run NAT here and must
> maintain an inside global DNS as well as an inside local DNS), and it
> was just as easy to tell named to use named.conf.private for the
> private DNS side.  And all those files reside in /etc/named and
> /etc/named.private.

Funny, I was going to pull this example, because my zone files are in
/var/named.

> But symlinks aren't the fix, as this is not an RPM-only issue -- there are
> more than just RPM users who might want an FHS-compliant installation with
> the capacity for multiple postmasters.

FHS-compliancy is only going to get you so far.  Where does it stop?
Next thing somebody comes around and claims that BSD hier(7) wants it all
differently.  At some point you're going to have to present usability
arguments.  And I notice that no one besides the RPM maintainer(s) have
ever complained about this, presumably because the current approach is
rather usable.

I don't mind a global configuration file that sets the defaults for or
overrides the local ones, because this adds a possibly useful feature.
But spreading out the local configuration files over the disk does not
help anyone.

-- 
Peter Eisentraut   peter_e@gmx.net   http://funkturm.homeip.net/~peter

Lamar Owen writes:

> SO, if postgresql.conf is already sitting in datadir, why not include the
> pg_hba.conf settings in the GUC?

Because the two have completely different structure.

> But in today's climate, an XML config might be better received.

Not in this lifetime.

> Oh, and tell initdb to not overwrite a postgresql.conf that already
> exists. :-).

Initdb doesn't even start if one already exists.

-- 
Peter Eisentraut   peter_e@gmx.net   http://funkturm.homeip.net/~peter

Lamar Owen writes:

> It's not far from this to making pg_ctl have the ability to start all the
> postmasters in a given config file with one command.

I think there is merit in the idea to let pg_ctl start more than one
postmaster.  But it probably ought to be in a separate config file, since
pg_ctl is a separate program.  Maybe simply have a file with one PGDATA
value per line, and pg_ctl runs in a loop and starts a postmaster for each
directory.  But what if you want to run several postmasters under several
user names (probably a good idea, otherwise, why are you running separate
postmasters)?  Needs more thought.

-- 
Peter Eisentraut   peter_e@gmx.net   http://funkturm.homeip.net/~peter

Lamar Owen writes:

> > Initdb doesn't even start if one already exists.
>
> I know.  Makes it more painful to package a default one. But that's an RPM
> issue, not a generic one.

Initdb puts the "sample" file from share/ in place.  Change that.

-- 
Peter Eisentraut   peter_e@gmx.net   http://funkturm.homeip.net/~peter

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Wednesday 16 May 2001 13:15, Peter Eisentraut wrote:
> Lamar Owen writes:
> > It's not far from this to making pg_ctl have the ability to start all the
> > postmasters in a given config file with one command.

> I think there is merit in the idea to let pg_ctl start more than one
> postmaster.  But it probably ought to be in a separate config file, since
> pg_ctl is a separate program.

Isn't pg_ctl's separate config file in 7.0.x one of the reasons for GUC in 
the first place? :-) Although that particular example is a little 
contrived.....

>  Maybe simply have a file with one PGDATA
> value per line, and pg_ctl runs in a loop and starts a postmaster for each
> directory.  But what if you want to run several postmasters under several
> user names (probably a good idea, otherwise, why are you running separate
> postmasters)?  Needs more thought.

Then you can have separate files.  Flexibility!
- --
Lamar Owen
WGCR Internet Radio
1 Peter 4:11
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.4 (GNU/Linux)
Comment: For info see http://www.gnupg.org

iD8DBQE7ArhH5kGGI8vV9eERAhjQAJ9BDjf1vmWFEPEH9dGN3ZDRuCJJRACgykmY
wKQvcH3Le9bexMbjkkdHFA0=
=N7yV
-----END PGP SIGNATURE-----

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Wednesday 16 May 2001 13:02, Peter Eisentraut wrote:
> Lamar Owen writes:
> > SO, if postgresql.conf is already sitting in datadir, why not include the
> > pg_hba.conf settings in the GUC?

> Because the two have completely different structure.

Currently. :-)  Didn't postmaster.opts.sample (or whatever that pg_ctl 
separate config file for 7.0.x was called) have a different structure from 
the current postgresql.conf? :-)  So the structure needs to change to 
implement -- there is precedent.

> > But in today's climate, an XML config might be better received.

> Not in this lifetime.

You mean not in your lifetime, right? :-)  I'm not a fan of XML either -- 
RealServer configures that way. :-(

> > Oh, and tell initdb to not overwrite a postgresql.conf that already
> > exists. :-).

> Initdb doesn't even start if one already exists.

I know.  Makes it more painful to package a default one. But that's an RPM 
issue, not a generic one.
- --
Lamar Owen
WGCR Internet Radio
1 Peter 4:11
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.4 (GNU/Linux)
Comment: For info see http://www.gnupg.org

iD8DBQE7ArfY5kGGI8vV9eERAqU6AKDTeiLzP82V/8Ej8YLeEssECae2rwCfR5zn
aHQebNNQfL2bqn2q4jtKp1I=
=WWQ8
-----END PGP SIGNATURE-----

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Wednesday 16 May 2001 12:56, Peter Eisentraut wrote:
> Lamar Owen writes:
> > I have multiple bind instances running on my main server -- it was
> > relatively easy to tell bind through named.conf where to find the
> > particular zone files for the private side (I run NAT here and must
> > maintain an inside global DNS as well as an inside local DNS), and it
> > was just as easy to tell named to use named.conf.private for the
> > private DNS side.  And all those files reside in /etc/named and
> > /etc/named.private.

> Funny, I was going to pull this example, because my zone files are in
> /var/named.

Which is arguably a better place to put them, FHS-wise.  But the point is 
this -- I can tell named where to look in a very flexible manner.  The bind 
cache, OTOH, is variable data and should go on the var filesystem....

> differently.  At some point you're going to have to present usability
> arguments.  And I notice that no one besides the RPM maintainer(s) have
> ever complained about this, presumably because the current approach is
> rather usable.

I'm not complaining.  However, I would think Oliver Elphick would like 
similar things for Debian.

As I said before, I can implement an RPM-specific solution.  But if it can 
benefit the general userbase, it shoudn't be an RPM-specific solution.

> I don't mind a global configuration file that sets the defaults for or
> overrides the local ones, because this adds a possibly useful feature.

Good.

> But spreading out the local configuration files over the disk does not
> help anyone.

Flexibility!  The admin should be allowed flexibility in installation, no?  
Of course, there are other directions the flexibility argument could go, but 
I'll not instigate _that_ battle...... :-)
- --
Lamar Owen
WGCR Internet Radio
1 Peter 4:11
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.4 (GNU/Linux)
Comment: For info see http://www.gnupg.org

iD8DBQE7Arno5kGGI8vV9eERAkPSAKDBqXIeeV7D7L4PV6dhp7b3gYq8hACg0jS5
zegguNNxir0at+WBJ9Aexa8=
=TOrY
-----END PGP SIGNATURE-----

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Wednesday 16 May 2001 13:41, Peter Eisentraut wrote:
> Lamar Owen writes:
> > > Initdb doesn't even start if one already exists.

> > I know.  Makes it more painful to package a default one. But that's an
> > RPM issue, not a generic one.

> Initdb puts the "sample" file from share/ in place.  Change that.

I was hoping to leave it more standard (so that an initdb to an alternate 
location would work without forcing the same default config on all), but I 
can deal with that.  Hey, if I must I can always patch initdb (although I do 
NOT want to do that.....).
- --
Lamar Owen
WGCR Internet Radio
1  Peter 4:11
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.4 (GNU/Linux)
Comment: For info see http://www.gnupg.org

iD8DBQE7Ar2K5kGGI8vV9eERApaXAJ4qf0dxbRpH/kyyVkBrmDi8e3I17wCgrGl6
eUul7QmsD0fYoFhNV59IL1M=
=8BSz
-----END PGP SIGNATURE-----

teg@redhat.com (Trond Eivind Glomsrød) writes:
> While on the subject of postgresql conf... shouldn't it be in
> sysconfdir instead of the database directory?

No.  That would (a) not allow different postmasters to have different
config files; (b) not allow a person to create an unprivileged
installation (assuming that sysconfdir is root-owned).
        regards, tom lane

Tom Lane <tgl@sss.pgh.pa.us> writes:

> teg@redhat.com (Trond Eivind Glomsrød) writes:
> >> The real bottom line here, though, is that you haven't shown me any
> >> positive reason to move the config files out of datadir. 
> 
> > It conflicts with the FHS -
> 
> AFAIK, the FHS is not designed to support multiple instances of
> unprivileged daemons.

It's OK to support such files, what I don't like is _requiring_ them.

-- 
Trond Eivind Glomsrød
Red Hat, Inc.

I would like to bring my 2 grains of salt to the discussion.

The advantage of putting configuration files in etc is that the
administrator knows that all his configuration files are there. For backup
or upgrade purpose, it is easy to review the /etc directory and make
modifcation if needed. It is rather a pain to look everywhere where the
configuration files are...

PG could look first in datadir and if nothing is found then go in /etc. This
would allow to run multiple instance of postifix (conf in data dir) or one
instance with a a standard place to put the configuration file...

I would add also, that with the extension mechanism, it would be nice if PG
could look for .so in the standard library path.

Franck Martin
Network and Database Development Officer
SOPAC South Pacific Applied Geoscience Commission
Fiji
E-mail: franck@sopac.org <mailto:franck@sopac.org> 
Web site: http://www.sopac.org/
<http://www.sopac.org/> Support FMaps: http://fmaps.sourceforge.net/
<http://fmaps.sourceforge.net/> 

This e-mail is intended for its addresses only. Do not forward this e-mail
without approval. The views expressed in this e-mail may not be necessarily
the views of SOPAC.

> The advantage of putting configuration files in etc is that the
> administrator knows that all his configuration files are there. For backup
> or upgrade purpose, it is easy to review the /etc directory and make
> modifcation if needed. It is rather a pain to look everywhere where the
> configuration files are...
>
> PG could look first in datadir and if nothing is found then go in
> /etc. This
> would allow to run multiple instance of postifix (conf in data dir) or one
> instance with a a standard place to put the configuration file...

Surely you would want config files in /usr/local/etc - definitely not /etc
...

Chris