Thread: location of the configuration files
The debate on the configuration file sparked a memory of an old patch I submitted in 7.1 days. One of the things I do not like about PostgreSQL is, IMHO, is a backwards configuration process. Rather than specify a data directory, the administrator should specify a database configuration file. Within the configuration file is the location and names of the data directory and other information. Most admins want a central location for this information. One of the things that is frustrating to me, is to have to hunt down where the data directory is so that I can administrate a DB. It can be anywhere, in any directory on any volume. If you had, say, a /usr/local/pgsql/admin, then you could have mydb.conf which could then be checked in to CVS. A standard location for configuration files is a more normal process as the location of the data directory is less so. I just don't think the PG data directory should not contain configuration information. The original patch allowed a user to specify the location of the postgresql.conf file, rather than assuming it lived in $PGDATA Also included in that patch, was the ability to specify the location of the PGDATA directory as well as the names of the pg_hba.conf and other configuration files. It also allowed the user to use PG as it has always worked, The patch was not applied because a better solution was supposedly coming, but that was two major revisions ago. I would still like to see this functionality. Would anyone else?
On Tue, 2003-02-11 at 13:44, mlw wrote: > The debate on the configuration file sparked a memory of an old patch I > submitted in 7.1 days. > > One of the things I do not like about PostgreSQL is, IMHO, is a > backwards configuration process. Rather than specify a data directory, > the administrator should specify a database configuration file. Within > the configuration file is the location and names of the data directory > and other information. Most admins want a central location for this > information. > > One of the things that is frustrating to me, is to have to hunt down > where the data directory is so that I can administrate a DB. It can be > anywhere, in any directory on any volume. If you had, say, a > /usr/local/pgsql/admin, then you could have mydb.conf which could then > be checked in to CVS. A standard location for configuration files is a > more normal process as the location of the data directory is less so. I > just don't think the PG data directory should not contain configuration > information. > > The original patch allowed a user to specify the location of the > postgresql.conf file, rather than assuming it lived in $PGDATA > Also included in that patch, was the ability to specify the location of > the PGDATA directory as well as the names of the pg_hba.conf and other > configuration files. > > It also allowed the user to use PG as it has always worked, The patch > was not applied because a better solution was supposedly coming, but > that was two major revisions ago. I would still like to see this > functionality. Would anyone else? > I'm going to be lazy and ask if you can post what the better solution that was coming was (or a link to the thread). While I'll grant you that the "it's coming" argument is pretty weak after two releases, that fact that it may have been a better solution could still hold up. Robert Treat
Robert Treat wrote: >I'm going to be lazy and ask if you can post what the better solution >that was coming was (or a link to the thread). While I'll grant you that >the "it's coming" argument is pretty weak after two releases, that fact >that it may have been a better solution could still hold up. > >Robert Treat > > AFAIK it wasn't actually done. It was more of a, "we should do something different" argument. At one point it was talked about rewriting the configuration system to allow "include" and other things.
I, personally, also think it makes more sense to pass to the postmaster a configuration file that contains all the rest of the information about the database system, including the disk locations of the various data directories and whatnot. cjs -- Curt Sampson <cjs@cynic.net> +81 90 7737 2974 http://www.netbsd.org Don't you know, in this new Dark Age, we're alllight. --XTC
mlw wrote: > AFAIK it wasn't actually done. It was more of a, "we should do something > different" argument. At one point it was talked about rewriting the > configuration system to allow "include" and other things. That seems like extreme overkill. The PostgreSQL configuration mechanism doesn't seem to me to be anywhere near complicated enough to justify an "include" mechanism. I agree with you: you should be able to specify all of the base configuration information (including the location of the data directories) in one file, and it makes perfect sense to me for the location of the data directory to be a GUC variable. I'd say the only thing the postmaster needs to know prior to startup is the directory containing the postgresql.conf file. An administrator who wishes to set up multiple independent databases can easily do so by using multiple config file directories. When consistency is required, he can easily use symlinks to point to master config files where appropriate. I assume $PGDATA was around long before GUC? -- Kevin Brown kevin@sysexperts.com
Kevin Brown <kevin@sysexperts.com> writes:
> I assume $PGDATA was around long before GUC?
Yes, it was. But I have not yet seen an argument here that justifies
why $SOMECONFIGDIRECTORY/postgresql.conf is better than
$PGDATA/postgresql.conf. The latter keeps all the related files
together. The former seems only to introduce unnecessary complexity.
You can only justify it as simpler if you propose hardwiring a value for
$SOMECONFIGDIRECTORY ... which is a proposal that will not fly with any
of the core developers, because we all run multiple versions of Postgres
on our machines so that we can deal with back-version bug reports,
test installations, etc. It is unlikely to fly with any of the RPM
packagers either, due to the wildly varying ideas out there about the
One True Place where applications should put their config files.
(This point was pretty much why mlw's previous proposal was rejected,
IIRC.)
regards, tom lane
Tom Lane wrote: > Kevin Brown <kevin@sysexperts.com> writes: > > I assume $PGDATA was around long before GUC? > > Yes, it was. But I have not yet seen an argument here that justifies > why $SOMECONFIGDIRECTORY/postgresql.conf is better than > $PGDATA/postgresql.conf. Okay, here's one: most Unix systems store all of the configuration files in a well known directory: /etc. These days it's a hierarchy of directories with /etc as the root of the hierarchy. When an administrator is looking for configuration files, the first place he's going to look is in /etc and its subdirectories. After that, he's forced to look through the startup scripts to figure out where things are located. And if those aren't revealing, then he has to read manpages and hope they're actually useful. :-) And if that doesn't work, then he has to resort to tricks like doing "strings" on the binaries (he doesn't necessarily have access to the sources that the binaries were compiled from, which is all that matters here). > The latter keeps all the related files together. The former seems > only to introduce unnecessary complexity. Well, I'd say it's "unnecessary" only when you already know where the data files are located -- which is true when you're a developer or someone who is already familiar with the installation you're working with. But if you're just getting started and installed it from a package like an RPM file, then you have to look in the package to see where it created the data file areas, or look at the startup scripts, etc. > You can only justify it as simpler if you propose hardwiring a value > for $SOMECONFIGDIRECTORY ... Making things simpler from the standpoint of the code isn't the point. Making things simpler for the DBA and/or Unix sysadmin is. I'd say $SOMECONFIGDIRECTORY should be a hardwired default with a command line override. I doubt you'll get a whole lot of argument from the general user community if you say that the hard wired default should be /etc/postgresql. > which is a proposal that will not fly with any of the core > developers, because we all run multiple versions of Postgres on our > machines so that we can deal with back-version bug reports, test > installations, etc. I absolutely agree that the config directory to use should be something that can be controlled with a command line option. > It is unlikely to fly with any of the RPM packagers either, due to > the wildly varying ideas out there about the One True Place where > applications should put their config files. There seems to be substantial agreement among the distribution maintainers that config files belong somewhere in /etc. At least, I've seen very little disagreement with that idea except from people who believe that each package should have its own, separate directory hierarchy. And the fact that the vast majority of packages put their config files somewhere in /etc supports this. Debian, for instance, actually *does* put the PostgreSQL config files in /etc/postgresql and creates symlinks in the data directory that point to them. This works, but it's a kludge. There are highly practical reasons for putting all the config files under /etc, not the least of which is that it makes backing up files that are *very* likely to change from the default, and which are also highly critical to the operation of the system, very easy. You'll get A LOT more disagreement about where to put data files than config files, as standards go. And in the case of PostgreSQL, where you put your data files is especially important for performance reasons, so it therefore makes even less sense to put the config files in the same location: it means that the config files could literally be anywhere, and any administrator who is unfamiliar with the system will have to dig through startup scripts (or worse!) to figure it out. Oh, here's another reason $SOMECONFIGDIRECTORY is better than $PGDATA: it allows much more appropriate separation of concern by default. Most installations of PostgreSQL start the database from a startup script that's run at boot time. With $PGDATA, changing the target data directory requires changing the startup script, which requires root access to the system -- if it didn't require root access then the entire system is open to the possibility of a world of hurt because the DBA isn't necessarily the same guy as the Unix sysadmin and therefore doesn't necessarily know his way around shell scripts in general, and rc scripts in particular, the way the Unix admin will. The possibility of hurt comes from the fact that the rc script runs at root, at a time that the system is hardest to work with in the event of a failure (many systems haven't even put up any console login prompts and may not have even started any remote login facilities before the PostgreSQL startup script runs). A sufficiently bad screwup on the part of the DBA with that kind of setup will require the Unix sysadmin to go to single user mode or worse to fix it. So unless the Unix sysadmin really trusts the DBA, he's not going to allow the DBA that kind of access. Instead he'll kludge something together so that the DBA can make the appropriate kinds of changes without compromising the system. But every shop will do this a different way. Even Debian, which usually is pretty good about dealing with issues like these, doesn't address this. But it shouldn't even be necessary for a shop to kludge around the problem: with $SOMECONFIGDIRECTORY, the Unix sysadmin can safely give write permissions to the DBA on the config files (and even on the entire directory they reside in), the DBA can point the database at whatever directory he wants the data to reside in, and the Unix admin only has to set up the storage areas, set the permissions, and let the DBA loose on it -- he doesn't have to touch the startup scripts at all, the DBA doesn't have to be much of a shell script wizard, and everyone is (relatively) happy. And, even better, the DBA will be able to do this at most shops he works for because his knowledge will be based on the standard PostgreSQL install. If we want to get wider acceptance amongst the heavy database users (which often *do* separate DBAs from Unix sysadmins), we may want to think about things like this from time to time... -- Kevin Brown kevin@sysexperts.com
On Wed, 2003-02-12 at 08:24, Kevin Brown wrote: > Tom Lane wrote: > > You can only justify it as simpler if you propose hardwiring a value > > for $SOMECONFIGDIRECTORY ... > > Making things simpler from the standpoint of the code isn't the point. > Making things simpler for the DBA and/or Unix sysadmin is. > > I'd say $SOMECONFIGDIRECTORY should be a hardwired default with a > command line override. > > I doubt you'll get a whole lot of argument from the general user > community if you say that the hard wired default should be > /etc/postgresql. > > > which is a proposal that will not fly with any of the core > > developers, because we all run multiple versions of Postgres on our > > machines so that we can deal with back-version bug reports, test > > installations, etc. > > I absolutely agree that the config directory to use should be > something that can be controlled with a command line option. > > > It is unlikely to fly with any of the RPM packagers either, due to > > the wildly varying ideas out there about the One True Place where > > applications should put their config files. > > There seems to be substantial agreement among the distribution > maintainers that config files belong somewhere in /etc. At least, > I've seen very little disagreement with that idea except from people > who believe that each package should have its own, separate directory > hierarchy. And the fact that the vast majority of packages put their > config files somewhere in /etc supports this. > > Debian, for instance, actually *does* put the PostgreSQL config files > in /etc/postgresql and creates symlinks in the data directory that > point to them. This works, but it's a kludge. > Seems like a good compromise would be to make the hard wired default $SOMECONFIGDIRECTORY be $PGDATA; this makes each version of the software more self contained/ less likely to interfere with another installation. (This becomes really handy when doing major upgrades). If you really have a strong desire to change this, you can. As I see it, this change would (should?) need to be something that could be changed in the configure script when building postgresql, as well changeable via a command line option, any other places? Robert Treat
> Okay, here's one: most Unix systems store all of the configuration > files in a well known directory: /etc. These days it's a hierarchy of > directories with /etc as the root of the hierarchy. When an > administrator is looking for configuration files, the first place he's > going to look is in /etc and its subdirectories. After that, he's > forced to look through the startup scripts to figure out where things > are located. And if those aren't revealing, then he has to read > manpages and hope they're actually useful. :-) And if that doesn't > work, then he has to resort to tricks like doing "strings" on the > binaries (he doesn't necessarily have access to the sources that the > binaries were compiled from, which is all that matters here). No goddammit - /usr/local/etc. Why can't the Linux community respect history!!!! It is the ONE TRUE PLACE dammit!!! Chris (btw, there is humour + seriousness in above post...)
> > binaries (he doesn't necessarily have access to the sources that the > > binaries were compiled from, which is all that matters here). > > No goddammit - /usr/local/etc. Why can't the Linux community respect > history!!!! History? It's the only way to make a read-only / (enforced by secure-level) and still be able to change the user applications. I don't mind /usr/X11R6/etc either, but it's not exactly appropriate for PostgreSQL ;) -- Rod Taylor <rbt@rbt.ca> PGP Key: http://www.rbt.ca/rbtpub.asc
"Christopher Kings-Lynne" <chriskl@familyhealth.com.au> wrote: > > Okay, here's one: most Unix systems store all of the configuration > > files in a well known directory: /etc. These days it's a hierarchy of > > directories with /etc as the root of the hierarchy. When an > > administrator is looking for configuration files, the first place he's > > going to look is in /etc and its subdirectories. > No goddammit - /usr/local/etc. Why can't the Linux community respect > history!!!! > > It is the ONE TRUE PLACE dammit!!! Well, to the extent that you're serious, you understand that a lot of people feel that /usr/local should be reserved for stuff that's installed by the local sysadmin, and your vendor/distro isn't supposed to be messing with it. Which means if the the vendor installed Postgresql (say, the Red Hat Database) you'd expect config files to be in /etc. If the postgresql is compiled from source by local admin, you might look somewhere in /usr/local. I've got the vauge feeling that this is all more than a little silly... directory locations floating about depending on who did what, as thought it were such a radical thing to do a ./configure, make & make install. But this is a pretty common feeling among the unix world (more wide spread than just in the Linux world).
On Wednesday 12 February 2003 20:37, Christopher Kings-Lynne wrote: > > Okay, here's one: most Unix systems store all of the configuration > > files in a well known directory: /etc. These days it's a hierarchy of > No [snip] - /usr/local/etc. Why can't the Linux community respect > history!!!! > It is the ONE TRUE PLACE [snip] If PostgreSQL is supported as a part of the base operating system in a Linux distribution, and that distribution wishes to be Linux Standards Base compliant (most do), then PostgreSQL cannot go in /usr/local -- period. IDIC at work. -- Lamar Owen WGCR Internet Radio 1 Peter 4:11
<br /><br /> Tom Lane wrote:<br /><blockquote cite="mid1552.1045034570@sss.pgh.pa.us" type="cite"><pre wrap="">Kevin Brown<a class="moz-txt-link-rfc2396E" href="mailto:kevin@sysexperts.com"><kevin@sysexperts.com></a> writes: </pre><blockquotetype="cite"><pre wrap="">I assume $PGDATA was around long before GUC? </pre></blockquote><pre wrap=""> Yes, it was. But I have not yet seen an argument here that justifies why $SOMECONFIGDIRECTORY/postgresql.conf is better than $PGDATA/postgresql.conf. The latter keeps all the related files together. The former seems only to introduce unnecessary complexity. You can only justify it as simpler if you propose hardwiring a value for $SOMECONFIGDIRECTORY ... which is a proposal that will not fly with any of the core developers, because we all run multiple versions of Postgres on our machines so that we can deal with back-version bug reports, test installations, etc. It is unlikely to fly with any of the RPM packagers either, due to the wildly varying ideas out there about the One True Place where applications should put their config files. (This point was pretty much why mlw's previous proposal was rejected, IIRC.) </pre></blockquote> I wasn't talking about a "default directory" I was talking about configuring a database in a configurationfile.<br /><br /> While I accept that the PostgreSQL group can not be playing catch-up with other databases,this does not preclude the notion accepting common practices and adopting them.<br /><br /> Understand, I reallylike PostgreSQL. I like it better than Oracle, and it is my DB of choice. That being said, I see what other DBs doright. Putting the configuration in the data directory is "wrong," no other database or service under UNIX or Windows doesthis, Period.<br /><br /> Does the PostgreSQL team know better than the rest of the world?<br /><br /> The idea thata, more or less, arbitrary data location determines the database configuration is wrong. It should be obvious to anyadministrator that a configuration file location which controls the server is the "right" way to do it. Regardless ofwhere ever you choose to put the default configuration file, it is EASIER to configure a database by using a file in astandard configuration directory (/etc, /usr/etc, /usr/local/etc, /usr/local/pgsql/conf or what ever). The data directoryshould not contain configuration data as it is typically dependent on where the admin chooses to mount storage.<br/><br /> I am astounded that this point of view is missed by the core group.<br /><br /><br /> Mark.<br />
mlw <pgsql@mohawksoft.com> writes:
> The idea that a, more or less, arbitrary data location determines the
> database configuration is wrong. It should be obvious to any
> administrator that a configuration file location which controls the
> server is the "right" way to do it.
I guess I'm just dense, but I entirely fail to see why this is the One
True Way To Do It. What you seem to be proposing (ignoring
syntactic-sugar issues) is that we replace "postmaster -D
/some/data/dir" by "postmaster -config /some/config/file". I am not
seeing the nature of the improvement. It looks to me like the sysadmin
must now grant the Postgres DBA write access on *two* directories, viz
/some/config/ and /wherever/the/data/directory/is. How is that better
than granting write access on one directory? Given that we can't manage
to standardize the data directory location across multiple Unixen, how
is it that we will be more successful at standardizing a config file
location?
All I see here is an arbitrary break with our past practice. I do not
see any net improvement.
regards, tom lane
<br /><br /> Tom Lane wrote:<br /><blockquote cite="mid10922.1045115467@sss.pgh.pa.us" type="cite"><pre wrap="">mlw <a class="moz-txt-link-rfc2396E"href="mailto:pgsql@mohawksoft.com"><pgsql@mohawksoft.com></a> writes: </pre><blockquotetype="cite"><pre wrap="">The idea that a, more or less, arbitrary data location determines the database configuration is wrong. It should be obvious to any administrator that a configuration file location which controls the server is the "right" way to do it. </pre></blockquote><pre wrap=""> I guess I'm just dense, but I entirely fail to see why this is the One True Way To Do It. What you seem to be proposing (ignoring syntactic-sugar issues) is that we replace "postmaster -D /some/data/dir" by "postmaster -config /some/config/file". I am not seeing the nature of the improvement. It looks to me like the sysadmin must now grant the Postgres DBA write access on *two* directories, viz /some/config/ and /wherever/the/data/directory/is. How is that better than granting write access on one directory? Given that we can't manage to standardize the data directory location across multiple Unixen, how is it that we will be more successful at standardizing a config file location? All I see here is an arbitrary break with our past practice. I do not see any net improvement. </pre></blockquote> There is a pretty well understood convention that a configuration file will be located in some standardlocation depending on your distro. Would you disagree with that?<br /><br /> There is also a convention that mostservers are configured by a configuration file, located in a central location. Look at sendmail, named,, et al. <br /><br/> Here is the test, configure a server, with sendmail, named, apache, and PostgreSQL. Tell me which of these systemsdoesn't configure right.<br />
At 12:31 AM -0500 2/13/03, mlw wrote: >The idea that a, more or less, arbitrary data location determines >the database configuration is wrong. It should be obvious to any >administrator that a configuration file location which controls the >server is the "right" way to do it. Isn't the database data itself a rather significant portion of the 'configuration' of the database? What do you gain by having the postmaster config and the database data live in different locations? -pmb
Peter Bierman wrote: > At 12:31 AM -0500 2/13/03, mlw wrote: > >> The idea that a, more or less, arbitrary data location determines the >> database configuration is wrong. It should be obvious to any >> administrator that a configuration file location which controls the >> server is the "right" way to do it. > > > > Isn't the database data itself a rather significant portion of the > 'configuration' of the database? > > What do you gain by having the postmaster config and the database data > live in different locations? While I don't like to use another product as an example, I think amongst the number of things Oracle does right is that it has a fairly standard way for an admin to find everything. All one needs to do is find the "ORACLE_HOME" directory, and everything can be found from there. If, assume, PostgreSQL worked like every other system. It would have either an entry in /etc or some other directory specified by configure. Somene please tell me how what I'm proposing differs from things like sendmail, named, or anyother standards based UNIX server?
Before I get started, I should note that it may be a good compromise to have the data directory be the same as the config file directory, when neither the config file nor the command line specify something different. So the changes I think may make the most sense are: 1. We add a new GUC variable which specifies where the data is. The data is assumed to reside in the same place the configfiles reside unless the GUC variable is defined (either in postgresql.conf or on the command line, as usual fora GUC variable). Both -D and $PGDATA therefore retain their current semantics unless overridden by the GUC variable,in which case they fall back to the new semantics of specifying only where the config files can be found. 2. We add a configure option that specifies what the hardcoded fallback directory should be when neither -D nor $PGDATAare specified: /etc/postgresql when the option isn't specified to configure. 3. We supply a different default startup script and a different default configuration file (but can make the older versions available in the distribution as well if we wish). The former uses neither $PGDATA nor -D (or uses /etc/postgresqlfor them), and the latter uses the new GUC variable to specify a data directory location (/var/lib/postgresby default?) This combination should work nicely for transitioning and for package builders. It accomplishes all of the goals mentioned in this thread and will cause minimal pain for developers, since they can use their current methods. Sounds like it'll make Tom happy, at least. :-) Tom Lane wrote: > mlw <pgsql@mohawksoft.com> writes: > > The idea that a, more or less, arbitrary data location determines the > > database configuration is wrong. It should be obvious to any > > administrator that a configuration file location which controls the > > server is the "right" way to do it. > > I guess I'm just dense, but I entirely fail to see why this is the One > True Way To Do It. But we're not saying it's the One True Way, just saying that it's a way that has very obvious benefits over the way we're using now, if your job is to manage a system that someone else set up. > What you seem to be proposing (ignoring syntactic-sugar issues) is > that we replace "postmaster -D /some/data/dir" by "postmaster > -config /some/config/file". I am not seeing the nature of the > improvement. The nature of the improvement is that the configuration of a PostgreSQL install will becomes obvious to anyone who looks in the obvious places. Remember, the '-D ...' is optional! The PGDATA environment variable can be used instead, and *is* used in what few installations I've seen. That's not something that shows up on the command line when looking at the process list, which forces the administrator to hunt down the data directory through other means. > It looks to me like the sysadmin must now grant the Postgres DBA > write access on *two* directories, viz /some/config/ and > /wherever/the/data/directory/is. How is that better than granting > write access on one directory? The difference in where you grant write access isn't a benefit to be gained here. The fact that you no longer have to give root privileges to the DBA so that he can change the data directory as needed is the benefit (well, one of them, at least). A standard packaged install can easily set the /etc/postgresql directory up with write permissions for the postgres user by default, so the sysadmin won't even have to touch it if he doesn't want to. A big production database box is usually managed by one or more system administrators and one or more DBAs. Their roles are largely orthogonal. The sysadmins have the responsibility of keeping the boxes up and making sure they don't fall over or crawl to a standstill. The DBAs have the responsibility of maximizing the performance and availability of the database and *that's all*. Giving the DBAs root privileges means giving them the power to screw up the system in ways that they can't recover from and might not even know about. The ways you can take down a system by misconfiguring the database are bad enough. No sane sysadmin is going to give the DBA the power to run an arbitrary script as root at a time during the boot cycle that the system is the most difficult to manage unless he thinks the DBA is *really* good at system administration tasks, too. And that's assuming the sysadmin even *has* the authority to grant the DBA that kind of access. Many organizations keep a tight rein on who can do what in an effort to minimize the damage from screwups. The point is that the DBA isn't likely to have root access to the box. When the DBA lacks that ability, the way we currently do things places greater demand on the sysadmin than is necessary, because root access is required to change the startup scripts, as it should be, and the location of the data, as it should *not* be. > Given that we can't manage to standardize the data directory > location across multiple Unixen, how is it that we will be more > successful at standardizing a config file location? A couple of ways. Firstly, as we mentioned before, just about every other daemon that runs on a Unix system has its configuration file somewhere in the /etc hierarchy. By putting our config files in that same hierarchy we'll be *adhering* to a standard. We don't have to worry about "standardizing" that config file location because it's *already* a standard that we're currently ignoring. Secondly, standards arise as a result of being declared standards and by most people using them. So simply by making /etc/postgresql the default configuration directory, *that* will become the standard place. Most people won't mess with the default install if they don't have to. Right now they almost *have to* mess with the default install, because there is no standard place on a Unix system for high speed, highly reliable disk access. And that means that, right now, there *is* no standard place for our config files -- it's wherever the person who configured the database decided the data should be, and he made that decision based on performance and reliability considerations, not on any standards. > All I see here is an arbitrary break with our past practice. I do not > see any net improvement. That's probably because you're looking at this from the point of view of a developer. From that standpoint there really isn't any net improvement, because *you* still have to specify something on the command line to get your test databases going. As a developer you *always* install and manage your own database installations, so *of course* you'll always know where the config files are. But that's not how it works in the production world. The break we'd be making is *not* arbitrary, and that's much of the point: it's a break towards existing standards, and there are good reasons for doing it, benefits to be had by adhering to those standards. The way we currently handle configuration files is fine for research and development use -- the environment from which PostgreSQL sprang. But now we're talking about getting it used in production environments, and their requirements are very different. Since it is *we* who are not currently adhering to the standard, shouldn't the burden of proof (so to speak) be on those who wish to keep things as they are? -- Kevin Brown kevin@sysexperts.com
> Well, to the extent that you're serious, you understand that > a lot of people feel that /usr/local should be reserved for > stuff that's installed by the local sysadmin, and your > vendor/distro isn't supposed to be messing with it. > > Which means if the the vendor installed Postgresql (say, the > Red Hat Database) you'd expect config files to be in /etc. > If the postgresql is compiled from source by local admin, > you might look somewhere in /usr/local. Indeed. For better or worse, there is a Filesystem Hierarcy Standard, and most of the important Linux distros, BSDs and some legacy Unixen stick to it, so so should we. Configuration files should be in /etc/postgresql/, or at the very least symlinked from there. Martin
On Wed, 12 Feb 2003, J. M. Brenner wrote: > > "Christopher Kings-Lynne" <chriskl@familyhealth.com.au> wrote: > > > > Okay, here's one: most Unix systems store all of the configuration > > > files in a well known directory: /etc. These days it's a hierarchy of > > > directories with /etc as the root of the hierarchy. When an > > > administrator is looking for configuration files, the first place he's > > > going to look is in /etc and its subdirectories. > > > No goddammit - /usr/local/etc. Why can't the Linux community respect > > history!!!! > > > > It is the ONE TRUE PLACE dammit!!! > > Well, to the extent that you're serious, you understand that > a lot of people feel that /usr/local should be reserved for > stuff that's installed by the local sysadmin, and your > vendor/distro isn't supposed to be messing with it. > > Which means if the the vendor installed Postgresql (say, the > Red Hat Database) you'd expect config files to be in /etc. > If the postgresql is compiled from source by local admin, > you might look somewhere in /usr/local. Then why not ~postgres/etc ?? Or substitute ~postgres with the db admin user you (or the distro) decided on at installation time. Gives a common location no matter who installed it or where it was installed. Vince. -- Fast, inexpensive internet service 56k and beyond! http://www.pop4.net/ http://www.meanstreamradio.com http://www.unknown-artists.com Internet radio: It's not file sharing, it's just radio.
On Wed, 12 Feb 2003, Peter Bierman wrote: > What do you gain by having the postmaster config and the database > data live in different locations? You can then standardize a location for the configuration files. Everybody has room in /etc for another 10K of data. Where you have room for something that might potentially be a half terrabyte of data, and is not infrequently several gigabytes or more, is pretty system-depenendent. cjs -- Curt Sampson <cjs@cynic.net> +81 90 7737 2974 http://www.netbsd.org Don't you know, in this new Dark Age, we're alllight. --XTC
In the last exciting episode, cjs@cynic.net (Curt Sampson) wrote: > On Wed, 12 Feb 2003, Peter Bierman wrote: > >> What do you gain by having the postmaster config and the database >> data live in different locations? > > You can then standardize a location for the configuration files. > > Everybody has room in /etc for another 10K of data. Where you have > room for something that might potentially be a half terrabyte of > data, and is not infrequently several gigabytes or more, is pretty > system-depenendent. Ah, but this has two notable problems: 1. It assumes that there is "a location" for "the configuration files for /the single database instance./" If I have a second database instance, that may conflict. 2. It assumes I have write access to /etc If I'm a Plain Old User, as opposed to root, I may only have read-only access to /etc. These conditions have both been known to occur... -- If this was helpful, <http://svcs.affero.net/rm.php?r=cbbrowne> rate me http://www.ntlug.org/~cbbrowne/rdbms.html "Some people, when confronted with a Unix problem, think "I know, I'll use sed." Now they have two problems." -- Jamie Zawinski
> -----Original Message----- > From: Christopher Browne [mailto:cbbrowne@acm.org] > Sent: 13 February 2003 13:33 > To: pgsql-hackers@postgresql.org > Subject: Re: [HACKERS] location of the configuration files > > > In the last exciting episode, cjs@cynic.net (Curt Sampson) wrote: > > On Wed, 12 Feb 2003, Peter Bierman wrote: > > > >> What do you gain by having the postmaster config and the database > >> data live in different locations? > > > > You can then standardize a location for the configuration files. > > > > Everybody has room in /etc for another 10K of data. Where you have > > room for something that might potentially be a half > terrabyte of data, > > and is not infrequently several gigabytes or more, is pretty > > system-depenendent. > > Ah, but this has two notable problems: > > 1. It assumes that there is "a location" for "the configuration files > for /the single database instance./" > > If I have a second database instance, that may conflict. > > 2. It assumes I have write access to /etc > > If I'm a Plain Old User, as opposed to root, I may only have > read-only access to /etc. > > These conditions have both been known to occur... So we can presumably allow the location to be overridden with a configure option? Regards, Dave
<br /><br /> Christopher Browne wrote:<br /><blockquote cite="midm3vfzoz4os.fsf@chvatal.cbbrowne.com" type="cite"><pre wrap="">Inthe last exciting episode, <a class="moz-txt-link-abbreviated" href="mailto:cjs@cynic.net">cjs@cynic.net</a> (CurtSampson) wrote: </pre><blockquote type="cite"><pre wrap="">On Wed, 12 Feb 2003, Peter Bierman wrote: </pre><blockquote type="cite"><pre wrap="">What do you gain by having the postmaster config and the database data live in different locations? </pre></blockquote><pre wrap="">You can then standardize a location for the configurationfiles. Everybody has room in /etc for another 10K of data. Where you have room for something that might potentially be a half terrabyte of data, and is not infrequently several gigabytes or more, is pretty system-depenendent. </pre></blockquote><pre wrap=""> Ah, but this has two notable problems: 1. It assumes that there is "a location" for "the configuration files for /the single database instance./" If I have a second database instance, that may conflict. 2. It assumes I have write access to /etc If I'm a Plain Old User, as opposed to root, I may only have read-only access to /etc. These conditions have both been known to occur... </pre></blockquote> These are not issues at all. You could put the configurationfile anywhere, just as you can for any UNIX service.<br /><br /> postmaster --config=/home/myhome/mydb.conf<br/><br /> I deal with a number of PG databases on a number of sites, and it is a real painin the ass to get to a PG box and hunt around for data directory so as to be able to administer the system. What's reallyannoying is when you have to find the data directory when someone else set up the system.<br /><br /> Configuring postgresqlvia a configuration file which specifies all the data, i.e. data directory, name of other configuration files,etc. is the right way to do it. Even if you have reasons against it, even if you think it is a bad idea, a bad standardis almost always a better solution than an arcane work of perfection.<br /><br /> Personally, however, I think theconfiguration issue is a no-brainer and I am amazed that people are balking. EVERY other service on a UNIX box is configuredin this way, why not do it this way in PostgreSQL? The patch I submitted allowed the configuration to work as itcurrently does, but allowed for the more standard configuration file methodology.<br /><br /> I just don't understand whatthe resistance is, it makes no sense.<br /><br /><br /><br /><br /><br />
On Thu, 13 Feb 2003, Christopher Browne wrote: > 1. It assumes that there is "a location" for "the configuration files > for /the single database instance./" No; it assumes that there's a location for "the default instance." If you have more than one, you could have one default and one elsewhere, or just do what I often do, which is put in an empty config file except for a comment saying "we have several instances of <xxx> on this machine; look in <yyy> for them." > 2. It assumes I have write access to /etc > > If I'm a Plain Old User, as opposed to root, I may only have > read-only access to /etc. Right. It's dependent on the sysadmin to create /etc/postgres/ and make it writeable, or set up proper symlinks, or whatever. Fortunately, the files in /etc are only the defaults, to be used if they're not overridden on the command line. If you're in a situation like #2, you're basically stuck where we are now all the time: you have to just put it somewhere and hope that, if someone else needs to find it, they can. cjs -- Curt Sampson <cjs@cynic.net> +81 90 7737 2974 http://www.netbsd.org Don't you know, in this new Dark Age, we're alllight. --XTC
On Thursday 13 February 2003 08:32, Christopher Browne wrote: > In the last exciting episode, cjs@cynic.net (Curt Sampson) wrote: > > Everybody has room in /etc for another 10K of data. Where you have > > room for something that might potentially be a half terrabyte of > > data, and is not infrequently several gigabytes or more, is pretty > > system-depenendent. > 1. It assumes that there is "a location" for "the configuration files > for /the single database instance./" > If I have a second database instance, that may conflict. If you run multiple servers of any kind, the second and subsequent servers must have a command line switch specifying the location of the config file. This is the way named, sendmail, et al do it. I have run multiple nameds on a single box, by using alternate config file locations. > 2. It assumes I have write access to /etc > If I'm a Plain Old User, as opposed to root, I may only have > read-only access to /etc. So you start postmaster with a config file switch pointing to your config file in your tree. Or you specify the default location with a configure switch at compile time. Or you do it the same way you would run any other typical daemon as a regular user. How does Apache, AOLserver (my favorite), sendmail, jabberd, named, or any other typical daemon do it? For example, AOLserver can easily be installed and run as a plain user (just not on port 80). The command line switch '-t' specifies the tcl configuration script's location. There is no default. The configuration script then specifies pageroot and the like -- and a webserver is very much like running PostgreSQL -- you have configuration, you have logs, and you have the spool (database or pageroot). All can be in different locations at the discretion of the admin. And hardcoding dependencies like this stifles the discretion of the admin. > These conditions have both been known to occur... Just because the situation is known to occur doesn't mean the whole direction of a project should hinge on those corner cases. They should be allowed but not forced. For better or for worse, thanks to Karl DeBisschop, the latest RPMs have the ability to start multiple postmasters with their data trees and configurations in different places. This is all done in the startup script, and required no new functionality from postmaster. I personally think it is for the better; YMMV. -- Lamar Owen WGCR Internet Radio 1 Peter 4:11
On Thu, 13 Feb 2003, Curt Sampson wrote: > On Thu, 13 Feb 2003, Christopher Browne wrote: > > > 1. It assumes that there is "a location" for "the configuration files > > for /the single database instance./" > > No; it assumes that there's a location for "the default instance." If > you have more than one, you could have one default and one elsewhere, or > just do what I often do, which is put in an empty config file except for > a comment saying "we have several instances of <xxx> on this machine; look > in <yyy> for them." > > > 2. It assumes I have write access to /etc > > > > If I'm a Plain Old User, as opposed to root, I may only have > > read-only access to /etc. > > Right. It's dependent on the sysadmin to create /etc/postgres/ and make > it writeable, or set up proper symlinks, or whatever. > > Fortunately, the files in /etc are only the defaults, to be used if > they're not overridden on the command line. If you're in a situation > like #2, you're basically stuck where we are now all the time: you have > to just put it somewhere and hope that, if someone else needs to find > it, they can. It doesn't follow this line of argument directly but it's to do with this thread... Is everyone forgetting that wherever the configuration file is stored and whether or not it needs a command line argument to specify it the database is not going to start up automatically unless at least part of the installation is done as root anyway? As I like to install software as a non root user normally anyway I am happy that the config file lives somewhere not requiring write access by the installer. However, I think having it in an etc directory is a good thing (tm). So, colour me an uncommited, fence sitter :) I'm not talking distribution/package installation here but just plain system administration. Being an untrusting soul I do _not_ want to type make install as root and find things installed outside of where I say I want things placed. That includes configuration files. Doing this as a normal user protects the system from bad software which assumes things about the host system. It also simplifies switching between versions of software, try doing that if your config is /etc/postgresql/postgres.conf. -- Nigel J. Andrews
mlw <pgsql@mohawksoft.com> writes:
> Here is the test, configure a server, with sendmail, named, apache, and
> PostgreSQL. Tell me which of these systems doesn't configure right.
AFAIK, only one of those four is designed to support multiple instances
running on a single machine. This is not unrelated.
regards, tom lane
<br /><br /> Tom Lane wrote:<br /><blockquote cite="mid13146.1045148622@sss.pgh.pa.us" type="cite"><pre wrap="">mlw <a class="moz-txt-link-rfc2396E"href="mailto:pgsql@mohawksoft.com"><pgsql@mohawksoft.com></a> writes: </pre><blockquotetype="cite"><pre wrap="">Here is the test, configure a server, with sendmail, named, apache, and PostgreSQL. Tell me which of these systems doesn't configure right. </pre></blockquote><pre wrap=""> AFAIK, only one of those four is designed to support multiple instances running on a single machine. This is not unrelated. </pre></blockquote> While I will agree with you on sendmail and named, Apache is often run more than once with differentoptions.<br /> Furthermore, I hate to keep bringing it up, Oracle does use the configuration file methodology.<br/><br /> Tom, I just don't understand why this is being resisted so vigorously. What is wrong with startingPostgreSQL as:<br /><br /> postmaster -C /etc/postgresql.conf<br /><br /> UNIX admins would love to have this asa methodology, I don't think you can deny this, can you? I, as a long term PG user, really really want this, because inthe long run, it makes PostgreSQL easier to administer.<br /><br /> If a patch allows PG to function as it does, but alsoallows a configuration file methodology, why not?<br />
On Thu, 13 Feb 2003, mlw wrote: > Tom, I just don't understand why this is being resisted so vigorously. > What is wrong with starting PostgreSQL as: > > postmaster -C /etc/postgresql.conf > > UNIX admins would love to have this as a methodology, I don't think you > can deny this, can you? I, as a long term PG user, really really want > this, because in the long run, it makes PostgreSQL easier to administer. > > If a patch allows PG to function as it does, but also allows a > configuration file methodology, why not? I forgot to say that I don't see why this facility can't be included in addition to the existing scheme. -- Nigel J. Andrews
On Thursday 13 February 2003 10:03, Tom Lane wrote: > mlw <pgsql@mohawksoft.com> writes: > > Here is the test, configure a server, with sendmail, named, apache, and > > PostgreSQL. Tell me which of these systems doesn't configure right. > AFAIK, only one of those four is designed to support multiple instances > running on a single machine. This is not unrelated. One can run many nameds on a single machine by specifying '-c alternate_named.conf' , which then points to a different set of zone files, listens to either a different port or address, etc. I have personally done this, and it worked as if it were designed to do so. Apache can easily have multiple instances by passing the location of httpd.conf on the command line. Everything else comes from that. Although Apache's virtual hosting is typically use instead, it may be necessary for large sites to run multiple instances with degrees of separation at the config file level. I use AOLserver, which in version 3 is designed from the ground up for many (even thousands) of instances on a single box. You access this capability with the '-t' switch (it stands for 'tcl config script' -- previous versions of AOLserver had an 'ini' file accessed with '-i', and version 3 added the tcl config script and deprecated the ini file). In fact, since there is no default, you MUST use -t. The tcl config script specifies all the parameters that instance needs (with the exception of the user and group ID the server should run as, if started as root (for port 80 access) -- but that doesn't effect PostgreSQL since our port is above 1024). Two instances can even share a tcl config script, as long as the virtual server name is specified on the command line, and the tcl config has multiple virtual server sections. I personally only lightly use this feature, running a mere half dozen AOLserver's on one of my production servers. All of which share a single PostgreSQL instance; but that's a different story. AOLserver is an excellent example here, as everything that has a location is configurable. During ./configure, you can specify the prefix and the other standard autoconf type options. This includes the location of the --enable-thread built Tcl 8.4 that you have to have first. I have two versions of AOLserver on that machine, and they coexist very well, because I _can_ be so specific in where everything lies. I run OpenACS on two of those instances, and, due to the size of that install I have those two pageroots on a separate filesystem from the binaries and config script. It was just a single tcl config entry. No biggie. Even sendmail has a -c switch to specify the location of sendmail.cf: so, yes, you can run multiple instances, although it could be argued that it wasn't designed in. Next? -- Lamar Owen WGCR Internet Radio 1 Peter 4:11
<br /><br /> Tom Lane wrote:<br /><blockquote cite="mid13146.1045148622@sss.pgh.pa.us" type="cite"><pre wrap="">mlw <a class="moz-txt-link-rfc2396E"href="mailto:pgsql@mohawksoft.com"><pgsql@mohawksoft.com></a> writes: </pre><blockquotetype="cite"><pre wrap="">Here is the test, configure a server, with sendmail, named, apache, and PostgreSQL. Tell me which of these systems doesn't configure right. </pre></blockquote><pre wrap=""> AFAIK, only one of those four is designed to support multiple instances running on a single machine. This is not unrelated. </pre></blockquote> Also, using an explicit configuration file willalso help you run multiple postgresql's on the same machien in a consistent manner, for instance:<br /><br /> postmaster-C /etc/postgres/common.conf -D /RAID0/postgres -p 5432<br /> postmaster -C /etc/postgres/common.conf -D /RAID1/postgres-p 5433<br /><br /> Please, Tom, tell me why this is such a bad idea?<br /><br /> I will make the patch, Iwill submit it, will you guys put it in?<br /><br /> If not, why?<br /><br /><blockquote cite="mid13146.1045148622@sss.pgh.pa.us"type="cite"><pre wrap=""> </pre></blockquote><br />
On Thu, Feb 13, 2003 at 15:03:09 +0000, "Nigel J. Andrews" <nandrews@investsystems.co.uk> wrote: > > Is everyone forgetting that wherever the configuration file is stored and > whether or not it needs a command line argument to specify it the database is > not going to start up automatically unless at least part of the installation is > done as root anyway? Users can use cron to start there own instance. Your cron script can check if the server is running every (say) 15 minutes and start the server if it isn't.
On Thu, Feb 13, 2003 at 09:23:20 -0500, mlw <pgsql@mohawksoft.com> wrote: > > Personally, however, I think the configuration issue is a no-brainer and > I am amazed that people are balking. EVERY other service on a UNIX box > is configured in this way, why not do it this way in PostgreSQL? The > patch I submitted allowed the configuration to work as it currently > does, but allowed for the more standard configuration file methodology. If you are interested in reading a contrary position, you can read Berstein's arguments for his recommended way to install services at: http://cr.yp.to/unix.html
On Thu, 2003-02-13 at 09:23, mlw wrote: > I deal with a number of PG databases on a number of sites, and it is a > real pain in the ass to get to a PG box and hunt around for data > directory so as to be able to administer the system. What's really > annoying is when you have to find the data directory when someone else > set up the system. > find / -name postgresql.conf -print you now know where all of your configuration files are and where the data for each of those servers is as well. (Not I'm not against the idea...) Robert Treat
<br /><br /> Robert Treat wrote:<br /><blockquote cite="mid1045153944.12930.521.camel@camel" type="cite"><pre wrap="">OnThu, 2003-02-13 at 09:23, mlw wrote: </pre><blockquote type="cite"><pre wrap="">I deal with a number of PG databaseson a number of sites, and it is a real pain in the ass to get to a PG box and hunt around for data directory so as to be able to administer the system. What's really annoying is when you have to find the data directory when someone else set up the system. </pre></blockquote><pre wrap=""> find / -name postgresql.conf -print</pre></blockquote><br /> LOL, That is NOT an option. It can take hours on some systems.Specifically, one of the systems is freedb server, it has over 300,000 files in a directory tree. <br /><br />
<br /><br /> Bruno Wolff III wrote:<br /><blockquote cite="mid20030213163651.GB8483@wolff.to" type="cite"><pre wrap="">OnThu, Feb 13, 2003 at 09:23:20 -0500, mlw <a class="moz-txt-link-rfc2396E" href="mailto:pgsql@mohawksoft.com"><pgsql@mohawksoft.com></a>wrote: </pre><blockquote type="cite"><pre wrap="">Personally,however, I think the configuration issue is a no-brainer and I am amazed that people are balking. EVERY other service on a UNIX box is configured in this way, why not do it this way in PostgreSQL? The patch I submitted allowed the configuration to work as it currently does, but allowed for the more standard configuration file methodology. </pre></blockquote><pre wrap=""> If you are interested in reading a contrary position, you can read Berstein's arguments for his recommended way to install services at: <a class="moz-txt-link-freetext" href="http://cr.yp.to/unix.html">http://cr.yp.to/unix.html</a> </pre></blockquote> Where, specificaly are his arguements against a configuration file methodology?<br /><blockquote cite="mid20030213163651.GB8483@wolff.to"type="cite"><pre wrap=""> </pre></blockquote><br />
On Thu, 13 Feb 2003, mlw wrote: > > > Robert Treat wrote: > > >On Thu, 2003-02-13 at 09:23, mlw wrote: > > > > > >>I deal with a number of PG databases on a number of sites, and it is a > >>real pain in the ass to get to a PG box and hunt around for data > >>directory so as to be able to administer the system. What's really > >>annoying is when you have to find the data directory when someone else > >>set up the system. You realize that the actual code feature doesn't necessarily help this case, right? Putting configuration in /etc and having a configuration file option on the command line are separate concepts. I think the feature is worthwhile, but I have some initial condition functionality questions that may have been answered in the previous patch, but I don't remember at this point. Mostly these have to deal with initial creation. Does the user specify an output location to initdb, do they just specify a data dir as now where the configuration goes but then they need to move it somewhere, does initdb now do nothing relating to configuration file and the user should make one on his own. Related, is the admin expected to have already made (say) /etc/postgresql to stick the config in and set the permissions correctly (since initdb doesn't run as root)?
Stephan Szabo wrote:<br /><blockquote cite="mid20030213084631.C45593-100000@megazone23.bigpanda.com" type="cite"><pre
wrap="">OnThu, 13 Feb 2003, mlw wrote:
</pre><blockquote type="cite"><pre wrap="">
Robert Treat wrote:
</pre><blockquote type="cite"><pre wrap="">On Thu, 2003-02-13 at 09:23, mlw wrote:
</pre><blockquote type="cite"><pre wrap="">I deal with a number of PG databases on a number of sites, and it is a
real pain in the ass to get to a PG box and hunt around for data
directory so as to be able to administer the system. What's really
annoying is when you have to find the data directory when someone else
set up the system. </pre></blockquote></blockquote></blockquote><pre wrap="">
You realize that the actual code feature doesn't necessarily help this
case, right? Putting configuration in /etc and having a configuration file
option on the command line are separate concepts.
I think the feature is worthwhile, but I have some initial condition
functionality questions that may have been answered in the previous patch,
but I don't remember at this point.
Mostly these have to deal with initial creation. Does the user specify an
output location to initdb, do they just specify a data dir as now where
the configuration goes but then they need to move it somewhere, does
initdb now do nothing relating to configuration file and the user should
make one on his own. Related, is the admin expected to have already made
(say) /etc/postgresql to stick the config in and set the permissions
correctly (since initdb doesn't run as root)?</pre></blockquote> My patch only works on the PostgreSQL server code. No
changeshave been made to the initialization scripts.<br /><br /> The patch declares three extra configuration file
parameters:<br/> hbafile= '/etc/postgres/pg_hba.conf'<br /> identfile='/etc/postgres/pg_ident.conf'<br />
datadir='/RAID0/postgres'<br/><br /> The command line option is a capital 'C,' as in:<br /> postmaster -C
/etc/postgresql.conf<br/><br /> I have no problem leaving the default configuration files remaining in the data
directoryas sort of a maintenance / boot strap sort of thing, so I don't see any reason to alter the installation.<br
/><br/><br /> As for this feature helping or not, I think it will. I think it accomplishes two things:<br /> (1)
Separatesconfiguration from data.<br /> (2) Allows an administrator to create a convention across multiple systems
regardlessof the location and mount points of the database storage.<br /> (3) Lastly, it is a familiar methodology to
DBAsnot familiar with PostgreSQL.<br /><br /> Again, I don't see a valid reason for not including the patch. Yes, if
youdon't want to configure PostgreSQL that way, then so be it, but why not add the functionality for those who do?<br
/><br/> I can envision the configuration file methodology of managing a database becoming the "preferred" approach over
timeas it is a more familiar and standard way of configuring servers on UNIX.<br /><br /><br /><blockquote
cite="mid20030213084631.C45593-100000@megazone23.bigpanda.com"type="cite"><pre wrap=""> </pre></blockquote>
On Thu, 2003-02-13 at 13:32, Christopher Browne wrote: > > Everybody has room in /etc for another 10K of data. Where you have > > room for something that might potentially be a half terrabyte of > > data, and is not infrequently several gigabytes or more, is pretty > > system-depenendent. > > Ah, but this has two notable problems: > > 1. It assumes that there is "a location" for "the configuration files > for /the single database instance./" > > If I have a second database instance, that may conflict. I think that moving configuration to [/usr/local]/etc/postgresql implies the need for sub-directories by port, possibly with a default config to be used if there is no port-specific config file. > 2. It assumes I have write access to /etc > > If I'm a Plain Old User, as opposed to root, I may only have > read-only access to /etc. The location should be configurable; I hope we're talking about the default here. For distributions it should be /etc/postgresql; for local builds it should be /usr/local/etc/postgresql, assuming you have root access. If you don't, the -c configfile switch suggested elsewhere in this debate would be needed. -- Oliver Elphick Oliver.Elphick@lfix.co.uk Isle of Wight, UK http://www.lfix.co.uk/oliver GPG: 1024D/3E1D0C1C: CA12 09E0 E8D5 8870 5839 932A 614D 4C34 3E1D 0C1C ======================================== "The earth is the LORD'S, and the fullness thereof; the world, and they thatdwell therein." Psalms 24:1
On Thu, 2003-02-13 at 12:00, Vince Vielhaber wrote: > > Which means if the the vendor installed Postgresql (say, the > > Red Hat Database) you'd expect config files to be in /etc. > > If the postgresql is compiled from source by local admin, > > you might look somewhere in /usr/local. > > Then why not ~postgres/etc ?? Or substitute ~postgres with the > db admin user you (or the distro) decided on at installation time. > Gives a common location no matter who installed it or where it was > installed. Because it doesn't comply with FHS. All projects should remember that they coexist with many others and should do their best to stick to common standards. The default config file location should be set as a parameter to ./configure, which should default to /usr/local/etc/postgresql. Those of us who build for distributions will change it to /etc/postgresql. I suppose if we want to run different postmasters simultaneously, we could have /etc/postgresql/5432/ and so on for each port number being used. Perhaps have a default set in /etc/postgresql/ which can be used if there is no port-specific directory, but a postmaster using those defaults would have to have PGDATA specified on the command line. On the same lines, I have just had a request (as Debian maintainer) to move the location of postmaster.pid to the /var/run hierarchy; firstly, so as to make it easier for the administrator to find, and secondly so as to make it easier to configure SE Linux policy for file access. (SE Linux is the highly secure version produced by the NSA.) I'm not entirely sure why SE Linux has a problem, seeing that postgres needs read-write access to all the files in $PGDATA, but assuming the need is verified, I could do this by moving the pid file from $PGDATA/postmaster.pid to /var/run/postgresql/5432.pid and similarly for other ports. This would also have the benefit of being more FHS compliant What do people think about that? -- Oliver Elphick Oliver.Elphick@lfix.co.uk Isle of Wight, UK http://www.lfix.co.uk/oliver GPG: 1024D/3E1D0C1C: CA12 09E0 E8D5 8870 5839 932A 614D 4C34 3E1D 0C1C ======================================== "The earth is the LORD'S, and the fullness thereof; the world, and they thatdwell therein." Psalms 24:1
On 13 Feb 2003, Oliver Elphick wrote: > On Thu, 2003-02-13 at 12:00, Vince Vielhaber wrote: > > > Which means if the the vendor installed Postgresql (say, the > > > Red Hat Database) you'd expect config files to be in /etc. > > > If the postgresql is compiled from source by local admin, > > > you might look somewhere in /usr/local. > > > > Then why not ~postgres/etc ?? Or substitute ~postgres with the > > db admin user you (or the distro) decided on at installation time. > > Gives a common location no matter who installed it or where it was > > installed. > > Because it doesn't comply with FHS. All projects should remember that > they coexist with many others and should do their best to stick to > common standards. > > The default config file location should be set as a parameter to > ./configure, which should default to /usr/local/etc/postgresql. Those > of us who build for distributions will change it to /etc/postgresql. Seems to me that if FHS allows such a mess, it's reason enough to avoid compliance. Either that or those of you who build for distributions are making an ill advised change. Simply because the distribution makes the decision to add PostgreSQL, or some other package, to it's distribution doesn't make it a requirement to change the location of the config files. Vince. -- Fast, inexpensive internet service 56k and beyond! http://www.pop4.net/ http://www.meanstreamradio.com http://www.unknown-artists.com Internet radio: It's not file sharing, it's just radio.
On Thu, 2003-02-13 at 17:52, Vince Vielhaber wrote: > Seems to me that if FHS allows such a mess, it's reason enough to avoid > compliance. Either that or those of you who build for distributions are > making an ill advised change. Simply because the distribution makes the > decision to add PostgreSQL, or some other package, to it's distribution > doesn't make it a requirement to change the location of the config files. Debian (and FHS) specifically requires that. All configuration files MUST be under /etc; the reason is to make the system administrator's job easier. Part of the raison d'etre of a distribution is to rationalise the idiosyncrasies of individual projects. The locations used by locally-built packages are up to the local administrator, but they really should not be in /etc and are recommended to be under /usr/local. I really don't see why there is such a not-invented-here mentality about this issue. I say again, standards-compliance is the best way. It makes life easier for everyone if standards are followed. Don't we pride ourselves on being closer to the SQL spec than other databases? Any way, if PostgreSQL stays as it is, I will continue to have to ensure that initdb creates symlinks to /etc/postgresql/, as happens now. -- Oliver Elphick Oliver.Elphick@lfix.co.uk Isle of Wight, UK http://www.lfix.co.uk/oliver GPG: 1024D/3E1D0C1C: CA12 09E0 E8D5 8870 5839 932A 614D 4C34 3E1D 0C1C ======================================== "The earth is the LORD'S, and the fullness thereof; the world, and they thatdwell therein." Psalms 24:1
Oliver Elphick wrote: > On Thu, 2003-02-13 at 17:52, Vince Vielhaber wrote: > > Seems to me that if FHS allows such a mess, it's reason enough to avoid > > compliance. Either that or those of you who build for distributions are > > making an ill advised change. Simply because the distribution makes the > > decision to add PostgreSQL, or some other package, to it's distribution > > doesn't make it a requirement to change the location of the config files. > > Debian (and FHS) specifically requires that. All configuration files > MUST be under /etc; the reason is to make the system administrator's job > easier. Part of the raison d'etre of a distribution is to rationalise > the idiosyncrasies of individual projects. The locations used by > locally-built packages are up to the local administrator, but they > really should not be in /etc and are recommended to be under /usr/local. > > I really don't see why there is such a not-invented-here mentality about > this issue. I say again, standards-compliance is the best way. It > makes life easier for everyone if standards are followed. Don't we > pride ourselves on being closer to the SQL spec than other databases? > Any way, if PostgreSQL stays as it is, I will continue to have to ensure > that initdb creates symlinks to /etc/postgresql/, as happens now. It doesn't have anything to do with "not-invented-here", which is a common refrain by people who don't like our decisions, like "Why don't you use mmap()? Oh, it's because I thought of it and you didn't". Does anyone seriously believe that is the motiviation of anyone in this project! I certainly don't. Now, on to this configuration discussion. Seems moving the config file out of $PGDATA requies either:1) we specifiy both the config directory and the data directory onpostmaster start2) we specifythe pgdata directory inside postgresql.conf orother config file Is this accurate? -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001+ If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania19073
<br /><br /> Bruce Momjian wrote:<br /><blockquote cite="mid200302131845.h1DIjRH01532@candle.pha.pa.us" type="cite"><prewrap="">Oliver Elphick wrote: </pre><blockquote type="cite"><pre wrap="">On Thu, 2003-02-13 at 17:52, VinceVielhaber wrote: </pre><blockquote type="cite"><pre wrap="">Seems to me that if FHS allows such a mess, it's reasonenough to avoid compliance. Either that or those of you who build for distributions are making an ill advised change. Simply because the distribution makes the decision to add PostgreSQL, or some other package, to it's distribution doesn't make it a requirement to change the location of the config files. </pre></blockquote><pre wrap="">Debian (andFHS) specifically requires that. All configuration files MUST be under /etc; the reason is to make the system administrator's job easier. Part of the raison d'etre of a distribution is to rationalise the idiosyncrasies of individual projects. The locations used by locally-built packages are up to the local administrator, but they really should not be in /etc and are recommended to be under /usr/local. I really don't see why there is such a not-invented-here mentality about this issue. I say again, standards-compliance is the best way. It makes life easier for everyone if standards are followed. Don't we pride ourselves on being closer to the SQL spec than other databases? Any way, if PostgreSQL stays as it is, I will continue to have to ensure that initdb creates symlinks to /etc/postgresql/, as happens now. </pre></blockquote><pre wrap=""> It doesn't have anything to do with "not-invented-here", which is a common refrain by people who don't like our decisions, like "Why don't you use mmap()? Oh, it's because I thought of it and you didn't". Does anyone seriously believe that is the motiviation of anyone in this project! I certainly don't. Now, on to this configuration discussion. Seems moving the config file out of $PGDATA requies either:1) we specifiy both the config directory and the data directory onpostmaster start2) we specifythe pgdata directory inside postgresql.conf orother config file Is this accurate? </pre></blockquote> The patch that I have adds three settings to postgresql.conf and one command line parameter.<br/><br /> hba_conf = 'filename'<br /> ident_conf='filename'<br /> data_dir='path'<br /><br /> The command linaeparameter is -C, used as:<br /><br /> postmaster -C /usr/local/etc/postgresql.conf<br /><br /> I think this will helpadministrators. <br /><br /> Bruce, can you shed some light as to why this is being so strongly rejected. I just don'tsee any downside. I just don't get it. <br /><br /> I will be resubmitting my patch for the 7.3.2 tree.<br /><br /><br/><blockquote cite="mid200302131845.h1DIjRH01532@candle.pha.pa.us" type="cite"><pre wrap=""> </pre></blockquote><br/>
On Thu, 2003-02-13 at 18:45, Bruce Momjian wrote: > Now, on to this configuration discussion. Seems moving the config file > out of $PGDATA requies either: > > 1) we specifiy both the config directory and the data directory on > postmaster start > > 2) we specify the pgdata directory inside postgresql.conf or > other config file > > Is this accurate? The default start would read the config file from its predefined location, set by ./configure. No command line options would be necessary for the postmaster to run, though they could be provided. The config file should contain the pgdata location; this and any other parameter should be overridden if a different location is specified by a command-line option. I think the config should be able to contain all information that can be specified on the command line (except, of course, the location of the configuration file.) -- Oliver Elphick Oliver.Elphick@lfix.co.uk Isle of Wight, UK http://www.lfix.co.uk/oliver GPG: 1024D/3E1D0C1C: CA12 09E0 E8D5 8870 5839 932A 614D 4C34 3E1D 0C1C ======================================== "The earth is the LORD'S, and the fullness thereof; the world, and they thatdwell therein." Psalms 24:1
On Thu, 2003-02-13 at 18:45, Bruce Momjian wrote: > Oliver Elphick wrote: > > On Thu, 2003-02-13 at 17:52, Vince Vielhaber wrote: > > > Seems to me that if FHS allows such a mess, it's reason enough to avoid > > > compliance. Either that or those of you who build for distributions are > > > making an ill advised change. Simply because the distribution makes the > > > decision to add PostgreSQL, or some other package, to it's distribution > > > doesn't make it a requirement to change the location of the config files. > > ... > > I really don't see why there is such a not-invented-here mentality about > > this issue. I say again, standards-compliance is the best way. It > > makes life easier for everyone if standards are followed. Don't we > > pride ourselves on being closer to the SQL spec than other databases? > > Any way, if PostgreSQL stays as it is, I will continue to have to ensure > > that initdb creates symlinks to /etc/postgresql/, as happens now. > > It doesn't have anything to do with "not-invented-here", which is a > common refrain by people who don't like our decisions, like "Why don't > you use mmap()? Oh, it's because I thought of it and you didn't". Does > anyone seriously believe that is the motiviation of anyone in this > project! I certainly don't. My apologies. I withdraw the comment, which was provoked mostly by Vince's response, quoted above. I agree that it is not characteristic of the project. -- Oliver Elphick Oliver.Elphick@lfix.co.uk Isle of Wight, UK http://www.lfix.co.uk/oliver GPG: 1024D/3E1D0C1C: CA12 09E0 E8D5 8870 5839 932A 614D 4C34 3E1D 0C1C ======================================== "The earth is the LORD'S, and the fullness thereof; the world, and they thatdwell therein." Psalms 24:1
mlw wrote: > >It doesn't have anything to do with "not-invented-here", which is a > >common refrain by people who don't like our decisions, like "Why don't > >you use mmap()? Oh, it's because I thought of it and you didn't". Does > >anyone seriously believe that is the motiviation of anyone in this > >project! I certainly don't. > > > >Now, on to this configuration discussion. Seems moving the config file > >out of $PGDATA requies either: > > > > 1) we specifiy both the config directory and the data directory on > > postmaster start > > > > 2) we specify the pgdata directory inside postgresql.conf or > > other config file > > > >Is this accurate? > > > > > The patch that I have adds three settings to postgresql.conf and one > command line parameter. > > hba_conf = 'filename' > ident_conf='filename' > data_dir='path' > > The command linae parameter is -C, used as: > > postmaster -C /usr/local/etc/postgresql.conf > > I think this will help administrators. > > Bruce, can you shed some light as to why this is being so strongly > rejected. I just don't see any downside. I just don't get it. > > I will be resubmitting my patch for the 7.3.2 tree. Well, in a sense, it trades passing one parameter, PGDATA, for another. I see your point that we should specify configuration first, and let everything pass from there. However, it does add extra configuration parameters, and because you still need to specify/create pgdata, it adds an extra level of abstraction to setting up the server. Also, there is nothing preventing someone from symlinking the configuration files from pgdata to somewhere else. I don't think separate params for each config file is good. At the most, I think we will specify the configuration _directory_ for all the config files, perhaps pgsql/etc, and have pgdata default to ../data, or honor $PGDATA. That might be the cleanest. Of course, that now gives us $PGCONFIG and $PGDATA, and possible intraction if postgresql.conf specifies a different pgdata from $PGDATA. As you can see, it could get messy. And, if you specify pgdata in postgresql.conf, it prevents you from using that file by different postmasters. My best guess would be to not specify pgdata in postgresql.conf, and have a new $PGCONFIG param to specify the configuration directory, but if we do that, $PGDATA/postgresql.conf becomes meaningless, which could also be confusing. Maybe we don't allow those files to exist in $PGDATA if $PGCONFIG is used, _and_ $PGCONFIG is not the same as $PGDATA. See, I am getting myself confused. :-) -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001+ If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania19073
On Thu, 2003-02-13 at 14:06, mlw wrote: > > I will be resubmitting my patch for the 7.3.2 tree. > I'm no core developer, but surely this wont be included in the 7.3.x branch. Any change needs to be made against CVS head. Robert Treat
Robert Treat wrote: > On Thu, 2003-02-13 at 14:06, mlw wrote: > > > > I will be resubmitting my patch for the 7.3.2 tree. > > > > I'm no core developer, but surely this wont be included in the 7.3.x > branch. Any change needs to be made against CVS head. I assume he meant he will repost his 7.3.2-based patch and we will merge it into CVS HEAD if it is accepted. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001+ If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania19073
On Thu, 2003-02-13 at 12:13, mlw wrote: > > My patch only works on the PostgreSQL server code. No changes have been > made to the initialization scripts. > > The patch declares three extra configuration file parameters: > hbafile= '/etc/postgres/pg_hba.conf' > identfile='/etc/postgres/pg_ident.conf' > datadir='/RAID0/postgres' > If we're going to do this, I think we need to account for all of the files in the directory including PG_VERSION, postmaster.opts, postmaster.pid. In the end if we can't build so that we are either fully FHS compliant and/or LSB compliant, we've not done enough work on it. Robert Treat
Bruce Momjian wrote: >Well, in a sense, it trades passing one parameter, PGDATA, for another. >I see your point that we should specify configuration first, and let >everything pass from there. However, it does add extra configuration >parameters, and because you still need to specify/create pgdata, it adds >an extra level of abstraction to setting up the server. > While this is true, it is not uncommon, and it is more or less expected by most UNIX admins. > >Also, there is nothing preventing someone from symlinking the >configuration files from pgdata to somewhere else. > Stop!!! symlinks are not sufficient. When happens when a native Win32 version comes out? there are no symlinks. Also, most of the admins I know don't like to use simlinks as they are not self documenting. Symlinks are "bad." > >I don't think separate params for each config file is good. At the >most, I think we will specify the configuration _directory_ for all the >config files, perhaps pgsql/etc, and have pgdata default to ../data, or >honor $PGDATA. That might be the cleanest. > The problem with that is that you are back to symlinking shared files. Symlinks are a kludge. > >Of course, that now gives us $PGCONFIG and $PGDATA, and possible >intraction if postgresql.conf specifies a different pgdata from $PGDATA. >As you can see, it could get messy. > I don't see it as very messy, for instance: postmaster -C /etc/postgres/postgresql.conf -D /RAID0/postgres -p 5432 postmaster -C /etc/postgres/postgresql.conf -D /RAID1/postgres -p 5433 That looks like a real clean way to run multiple PostgreSQL servers on the same box using the same configuration files. >And, if you specify pgdata in postgresql.conf, it prevents you from >using that file by different postmasters. > Not true, command line parameters, as a rule, override configuration file defaults. > >My best guess would be to not specify pgdata in postgresql.conf, and >have a new $PGCONFIG param to specify the configuration directory, but >if we do that, $PGDATA/postgresql.conf becomes meaningless, which could >also be confusing. Maybe we don't allow those files to exist in $PGDATA >if $PGCONFIG is used, _and_ $PGCONFIG is not the same as $PGDATA. See, >I am getting myself confused. :-) > I think you are making it too complicated. I wouldn't remove the default configration set, it would be useful as a failsafe or maintainence feature.
Robert Treat wrote: > On Thu, 2003-02-13 at 12:13, mlw wrote: > > > > My patch only works on the PostgreSQL server code. No changes have been > > made to the initialization scripts. > > > > The patch declares three extra configuration file parameters: > > hbafile= '/etc/postgres/pg_hba.conf' > > identfile='/etc/postgres/pg_ident.conf' > > datadir='/RAID0/postgres' > > > > If we're going to do this, I think we need to account for all of the > files in the directory including PG_VERSION, postmaster.opts, > postmaster.pid. In the end if we can't build so that we are either fully > FHS compliant and/or LSB compliant, we've not done enough work on it. Woh, how do we move some of those files into /etc or /var/run if we aren't running as root? We certainly don't want to require that. I guess /etc/postgresql will work if that directory is owned by the PostgreSQL superuser, but /var/run will be a problem. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001+ If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania19073
<br /><br /> Robert Treat wrote:<br /><blockquote cite="mid1045164643.12930.534.camel@camel" type="cite"><pre wrap="">OnThu, 2003-02-13 at 12:13, mlw wrote: </pre><blockquote type="cite"><pre wrap="">My patch only works on the PostgreSQLserver code. No changes have been made to the initialization scripts. The patch declares three extra configuration file parameters: hbafile= '/etc/postgres/pg_hba.conf' identfile='/etc/postgres/pg_ident.conf' datadir='/RAID0/postgres' </pre></blockquote><pre wrap=""> If we're going to do this, I think we need to account for all of the files in the directory including PG_VERSION, postmaster.opts, postmaster.pid. In the end if we can't build so that we are either fully FHS compliant and/or LSB compliant, we've not done enough work on it. Robert Treat </pre></blockquote> postmaster.opts, PG_VERSION, and postmaster.pid are not configuration parameters. <br /><br /> PG_VERSIONis VERY important, it is how you know the version of the database.<br /> Postmaster.pid is a postgres writablevalue <br /> AFAIK, postmaster.opts is also a postgres writable value.<br />
On Thu, 2003-02-13 at 19:30, Robert Treat wrote: > If we're going to do this, I think we need to account for all of the > files in the directory including PG_VERSION, postmaster.opts, Not PG_VERSION; that is intimately associated with the data itself and ought to stay in the data directory. -- Oliver Elphick Oliver.Elphick@lfix.co.uk Isle of Wight, UK http://www.lfix.co.uk/oliver GPG: 1024D/3E1D0C1C: CA12 09E0 E8D5 8870 5839 932A 614D 4C34 3E1D 0C1C ======================================== "The earth is the LORD'S, and the fullness thereof; the world, and they thatdwell therein." Psalms 24:1
On Thu, 13 Feb 2003, mlw wrote:
> Stephan Szabo wrote:
> >>>On Thu, 2003-02-13 at 09:23, mlw wrote:
> >>>>I deal with a number of PG databases on a number of sites, and it is a
> >>>>real pain in the ass to get to a PG box and hunt around for data
> >>>>directory so as to be able to administer the system. What's really
> >>>>annoying is when you have to find the data directory when someone else
> >>>>set up the system.
> >>>>
> >>>>
> >
> >You realize that the actual code feature doesn't necessarily help this
> >case, right? Putting configuration in /etc and having a configuration file
> >option on the command line are separate concepts.
Re-read my statement and yours about the case you were mentioning. ;)
Sure, putting the files in /etc lets you find them easily. However, if
you're doing things like finding configuration made by someone else and
said configuration isn't in /etc (which if they wanted to they could do
now with symlinks I believe - yes symlinks aren't a complete solution, but
I think they're reasonable on most of our current ports) then you still
have to search the system for the configuration file, except now it might
not even be postgresql.conf. That's why I said the two issues aren't the
same.
> >I think the feature is worthwhile, but I have some initial condition
> >functionality questions that may have been answered in the previous patch,
> >but I don't remember at this point.
> >
> >Mostly these have to deal with initial creation. Does the user specify an
> >output location to initdb, do they just specify a data dir as now where
> >the configuration goes but then they need to move it somewhere, does
> >initdb now do nothing relating to configuration file and the user should
> >make one on his own. Related, is the admin expected to have already made
> >(say) /etc/postgresql to stick the config in and set the permissions
> >correctly (since initdb doesn't run as root)?
> >
> My patch only works on the PostgreSQL server code. No changes have been
> made to the initialization scripts.
>
> The patch declares three extra configuration file parameters:
> hbafile= '/etc/postgres/pg_hba.conf'
> identfile='/etc/postgres/pg_ident.conf'
> datadir='/RAID0/postgres'
>
> The command line option is a capital 'C,' as in:
> postmaster -C /etc/postgresql.conf
>
> I have no problem leaving the default configuration files remaining in
> the data directory as sort of a maintenance / boot strap sort of thing,
> so I don't see any reason to alter the installation.
>
>
> As for this feature helping or not, I think it will. I think it
> accomplishes two things:
> (1) Separates configuration from data.
> (2) Allows an administrator to create a convention across multiple
> systems regardless of the location and mount points of the database storage.
> (3) Lastly, it is a familiar methodology to DBAs not familiar with
> PostgreSQL.
I agree on all these points ("I think the feature is worthwhile, but...").
I just wonder if we were going to do this, we might as well look at all of
the various things people want and decide what we want to do, for example,
people commenting on default configuration locations through configure,
how does this interact with what we have now, etc. I'd rather have a
month spent arguing out a behavior rather than just adding a new behavior
that we'll need to possibly revisit again in the future. :)
<br /><br /> Bruce Momjian wrote:<br /><blockquote cite="mid200302131943.h1DJhS211683@candle.pha.pa.us"
type="cite"><prewrap="">Robert Treat wrote: </pre><blockquote type="cite"><pre wrap="">On Thu, 2003-02-13 at 12:13, mlw
wrote: </pre><blockquote type="cite"><pre wrap="">My patch only works on the PostgreSQL server code. No changes have
been
made to the initialization scripts.
The patch declares three extra configuration file parameters:
hbafile= '/etc/postgres/pg_hba.conf'
identfile='/etc/postgres/pg_ident.conf'
datadir='/RAID0/postgres'
</pre></blockquote><pre wrap="">If we're going to do this, I think we need to account for all of the
files in the directory including PG_VERSION, postmaster.opts,
postmaster.pid. In the end if we can't build so that we are either fully
FHS compliant and/or LSB compliant, we've not done enough work on it. </pre></blockquote><pre wrap="">
Woh, how do we move some of those files into /etc or /var/run if we
aren't running as root? We certainly don't want to require that. I
guess /etc/postgresql will work if that directory is owned by the
PostgreSQL superuser, but /var/run will be a problem.
</pre></blockquote> I don't think those files need to move. As I said in another post, they are postgres writable and
shouldin the PostgreSQL data directory. However, I suppose, that those also could be configuration parameters? No?<br
/><br/> PG_VERSION obviously should not move.<br /> postmaster.opts gets created when postmaster is run, correct?<br
/><br/> The only issue would be the PID file, and I don't have strong feelings about it, except that using a /var/run
systemwill make running multiple postmasters a pain.<br /><br /><br />
<br /><br /> Stephan Szabo wrote:<br /><blockquote cite="mid20030213114014.S47380-100000@megazone23.bigpanda.com"
type="cite"><prewrap="">On Thu, 13 Feb 2003, mlw wrote:
</pre><blockquote type="cite"><pre wrap="">Stephan Szabo wrote: </pre><blockquote type="cite"><blockquote
type="cite"><blockquotetype="cite"><pre wrap="">On Thu, 2003-02-13 at 09:23, mlw wrote: </pre><blockquote
type="cite"><prewrap="">I deal with a number of PG databases on a number of sites, and it is a
real pain in the ass to get to a PG box and hunt around for data
directory so as to be able to administer the system. What's really
annoying is when you have to find the data directory when someone else
set up the system.
</pre></blockquote></blockquote></blockquote><pre wrap="">You realize that the actual code feature doesn't
necessarilyhelp this
case, right? Putting configuration in /etc and having a configuration file
option on the command line are separate concepts. </pre></blockquote></blockquote><pre wrap="">
Re-read my statement and yours about the case you were mentioning. ;)
Sure, putting the files in /etc lets you find them easily. However, if
you're doing things like finding configuration made by someone else and
said configuration isn't in /etc (which if they wanted to they could do
now with symlinks I believe - yes symlinks aren't a complete solution, but
I think they're reasonable on most of our current ports) then you still
have to search the system for the configuration file, except now it might
not even be postgresql.conf. That's why I said the two issues aren't the
same.
</pre><blockquote type="cite"><blockquote type="cite"><pre wrap="">I think the feature is worthwhile, but I have some
initialcondition
functionality questions that may have been answered in the previous patch,
but I don't remember at this point.
Mostly these have to deal with initial creation. Does the user specify an
output location to initdb, do they just specify a data dir as now where
the configuration goes but then they need to move it somewhere, does
initdb now do nothing relating to configuration file and the user should
make one on his own. Related, is the admin expected to have already made
(say) /etc/postgresql to stick the config in and set the permissions
correctly (since initdb doesn't run as root)?
</pre></blockquote><pre wrap="">My patch only works on the PostgreSQL server code. No changes have been
made to the initialization scripts.
The patch declares three extra configuration file parameters:
hbafile= '/etc/postgres/pg_hba.conf'
identfile='/etc/postgres/pg_ident.conf'
datadir='/RAID0/postgres'
The command line option is a capital 'C,' as in:
postmaster -C /etc/postgresql.conf
I have no problem leaving the default configuration files remaining in
the data directory as sort of a maintenance / boot strap sort of thing,
so I don't see any reason to alter the installation.
As for this feature helping or not, I think it will. I think it
accomplishes two things:
(1) Separates configuration from data.
(2) Allows an administrator to create a convention across multiple
systems regardless of the location and mount points of the database storage.
(3) Lastly, it is a familiar methodology to DBAs not familiar with
PostgreSQL. </pre></blockquote><pre wrap="">
I agree on all these points ("I think the feature is worthwhile, but...").
I just wonder if we were going to do this, we might as well look at all of
the various things people want and decide what we want to do, for example,
people commenting on default configuration locations through configure,
how does this interact with what we have now, etc. I'd rather have a
month spent arguing out a behavior rather than just adding a new behavior
that we'll need to possibly revisit again in the future. :)</pre></blockquote><br /> I have absolutely no problem
debatingand augmenting the feature. None what so ever, I am more pushing to get momentum to actually do it. In 7.1 I
proposedthis, and was told that it wasn't needed because (a) symlinks provide all the functionality you need and (b)
thatthey were going to redesign the configuration system. That was well over a year ago (two?). I am willing to do the
work,but what's the point if the core group isn't even going to use it?<br /><br /> Most of the admins I know don't use
symlinksas they can not carry comments. Without knowing, you can change or delete a file that does not appear to be in
usebut which kills a working server. Symlinks are dangerous in production systems, it is easy to screw them up with scp
whenadministering a cluster of computers.<br /><br /><br /><blockquote
cite="mid20030213114014.S47380-100000@megazone23.bigpanda.com"type="cite"><pre wrap=""> </pre></blockquote>
On 13 Feb 2003, Martin Coxall wrote: > > > Well, to the extent that you're serious, you understand that > > a lot of people feel that /usr/local should be reserved for > > stuff that's installed by the local sysadmin, and your > > vendor/distro isn't supposed to be messing with it. > > > > Which means if the the vendor installed Postgresql (say, the > > Red Hat Database) you'd expect config files to be in /etc. > > If the postgresql is compiled from source by local admin, > > you might look somewhere in /usr/local. > > Indeed. For better or worse, there is a Filesystem Hierarcy Standard, > and most of the important Linux distros, BSDs and some legacy Unixen > stick to it, so so should we. > > Configuration files should be in /etc/postgresql/, or at the very least > symlinked from there. So, how do we handle things like installing three or four versions at the same time. This isn't the same thing as /etc/fstab. While we only would likely need to have one fstab or whatever, with postgresql, it's not unreasonable to want to intall more than one copy or version for various reason. Generally things that live in /etc are owned and operated by the OS. Postgresql, by it's definition is a userspace program, not an OS owned one. I've found having a $PGDATA var where EVERYTHING lives to be a huge advantage when you need to run a half dozen instances of pgsql under different accounts or for different versions on the same box. Now, if we could do it like X, where the base stuff is all in the /etc/X11R6 directory, but your own personal config lives in your home directory, then we're right as rain. but what parts of postgresql would always be common to all flavors that might need to be run at the same time? Not much.
I don't see why we can't keep everyone happy and let the users choose the setup they want. To wit, make the following, probably simple, changes: 1) Have postgresql default to using /etc/postgresql.conf 2) Add a setting in postgresql.conf specifying the data directory 3) Change the meaning of -D to mean "use this config file" 4) In the absence of a specified data directory in postgresql.conf, use the location of the postgresql.conf file as the data directory I see several advantages: 1) Anyone who doesn't want to change doesn't have to - leaving the data directory spec out of postgresql.conf and starting with -D will be essentially identical to how things are now (except it would be -D /foo/bar/postgresql.conf instead of -D /foo/bar/ - even this could be overcome with a bit of bailing wire saying if -D specifies a directory, look for postgresql.conf in that directory). 2) Postgresql will be more "familiar" to those who expect or desire configs to be in /etc. 3) Adding a postgresql.conf line for data location sets the stage for being able to specify directories for all sorts of files (WAL, index, etc.) without the need for symlinks. 4) Multiple config files could be more easily managed for testing/benchmarking/etc. Cheers, Steve On Wednesday 12 February 2003 10:14 pm, Peter Bierman wrote: > At 12:31 AM -0500 2/13/03, mlw wrote: > >The idea that a, more or less, arbitrary data location determines > >the database configuration is wrong. It should be obvious to any > >administrator that a configuration file location which controls the > >server is the "right" way to do it. > > Isn't the database data itself a rather significant portion of the > 'configuration' of the database? > > What do you gain by having the postmaster config and the database > data live in different locations? > > -pmb > > ---------------------------(end of broadcast)--------------------------- > TIP 2: you can get off all lists at once with the unregister command > (send "unregister YourEmailAddressHere" to majordomo@postgresql.org)
mlw writes: > AFAIK it wasn't actually done. It was more of a, "we should do something > different" argument. At one point it was talked about rewriting the > configuration system to allow "include" and other things. The core of the problem was, and continues to be, this: If you move postgresql.conf somewhere else, then someone else will also want to move pg_hba.conf and all the rest. And that opens up a number of security and cumbersome-to-install problems. Just having an option that says, the configuration file is "there", is a first step but not a complete solution. -- Peter Eisentraut peter_e@gmx.net
<br /><br /> Peter Eisentraut wrote:<br /><blockquote cite="midPine.LNX.4.44.0302131909260.2646-100000@peter.localdomain"type="cite"><pre wrap="">mlw writes: </pre><blockquote type="cite"><pre wrap="">AFAIK it wasn't actually done. It was more of a, "we should do something different" argument. At one point it was talked about rewriting the configuration system to allow "include" and other things. </pre></blockquote><pre wrap=""> The core of the problem was, and continues to be, this: If you move postgresql.conf somewhere else, then someone else will also want to move pg_hba.conf and all the rest. And that opens up a number of security and cumbersome-to-install problems. Just having an option that says, the configuration file is "there", is a first step but not a complete solution.</pre></blockquote> The location of pg_hba.conf and pg_ident.conf can be specifiedwithin the postgresql.conf file if desired.<br /><br /> I don't understand the security concerns, what securityissues can there be?<br /><br /><blockquote cite="midPine.LNX.4.44.0302131909260.2646-100000@peter.localdomain" type="cite"><prewrap=""> </pre></blockquote>
On 13 Feb 2003, Oliver Elphick wrote: > On Thu, 2003-02-13 at 18:45, Bruce Momjian wrote: > > Oliver Elphick wrote: > > > On Thu, 2003-02-13 at 17:52, Vince Vielhaber wrote: > > > > Seems to me that if FHS allows such a mess, it's reason enough to avoid > > > > compliance. Either that or those of you who build for distributions are > > > > making an ill advised change. Simply because the distribution makes the > > > > decision to add PostgreSQL, or some other package, to it's distribution > > > > doesn't make it a requirement to change the location of the config files. > > > ... > > > I really don't see why there is such a not-invented-here mentality about > > > this issue. I say again, standards-compliance is the best way. It > > > makes life easier for everyone if standards are followed. Don't we > > > pride ourselves on being closer to the SQL spec than other databases? > > > Any way, if PostgreSQL stays as it is, I will continue to have to ensure > > > that initdb creates symlinks to /etc/postgresql/, as happens now. > > > > It doesn't have anything to do with "not-invented-here", which is a > > common refrain by people who don't like our decisions, like "Why don't > > you use mmap()? Oh, it's because I thought of it and you didn't". Does > > anyone seriously believe that is the motiviation of anyone in this > > project! I certainly don't. > > My apologies. I withdraw the comment, which was provoked mostly by > Vince's response, quoted above. I agree that it is not characteristic > of the project. I certainly wasn't trying to provoke anything. It just seems odd to me that when the distribution installs a package and places it's config files in /etc and later the admin happens to upgrade by the instructions with the package, it's acceptable for the config files to now be in two places and you don't find it confusing. What happens when a new admin comes on and tries to figure out which config file is which? Ever try to figure out where the hell Pine's config really is? Vince. -- Fast, inexpensive internet service 56k and beyond! http://www.pop4.net/ http://www.meanstreamradio.com http://www.unknown-artists.com Internet radio: It's not file sharing, it's just radio.
On Thu, 13 Feb 2003, mlw wrote: > > > Christopher Browne wrote: > > >In the last exciting episode, cjs@cynic.net (Curt Sampson) wrote: > > > >>On Wed, 12 Feb 2003, Peter Bierman wrote: > >> > >>>What do you gain by having the postmaster config and the database > >>>data live in different locations? > >>> > >>You can then standardize a location for the configuration files. > >> > >>Everybody has room in /etc for another 10K of data. Where you have > >>room for something that might potentially be a half terrabyte of > >>data, and is not infrequently several gigabytes or more, is pretty > >>system-depenendent. > > > >Ah, but this has two notable problems: > > > >1. It assumes that there is "a location" for "the configuration files > > for /the single database instance./" > > > > If I have a second database instance, that may conflict. > > > >2. It assumes I have write access to /etc > > > > If I'm a Plain Old User, as opposed to root, I may only have > > read-only access to /etc. > > > >These conditions have both been known to occur... > > > > > These are not issues at all. You could put the configuration file > anywhere, just as you can for any UNIX service. > > postmaster --config=/home/myhome/mydb.conf > > I deal with a number of PG databases on a number of sites, and it is a > real pain in the ass to get to a PG box and hunt around for data > directory so as to be able to administer the system. What's really > annoying is when you have to find the data directory when someone else > set up the system. Really? I would think it's easier to do this: su - pgsuper cd $PGDATA pwd Than to try to figure out what someone entered when they ran ./configure --config=... > Configuring postgresql via a configuration file which specifies all the > data, i.e. data directory, name of other configuration files, etc. is > the right way to do it. Even if you have reasons against it, even if you > think it is a bad idea, a bad standard is almost always a better > solution than an arcane work of perfection. Wrong, I strongly disagree with this sentament. Conformity to standards for simple conformity's sake is as wrong as sticking to the old way because it's what we're all comfy with. > Personally, however, I think the configuration issue is a no-brainer and > I am amazed that people are balking. EVERY other service on a UNIX box > is configured in this way, why not do it this way in PostgreSQL? The > patch I submitted allowed the configuration to work as it currently > does, but allowed for the more standard configuration file methodology. If I do a .tar.gz install of apache, I get /usr/local/apache/conf, which is not the standard way you're listing. If I install openldap from .tar.gz, I get a /usr/local/etc/openldap directory, close, but still not the same. The fact is, it's the packagers that put things into /etc and whatnot, and I can see the postgresql RPMs or debs or whatever having that as the default, but for custom built software, NOTHING that I know of builds from source and uses /etc without a switch to tell it to, just like postgresql can do now. > I just don't understand what the resistance is, it makes no sense. I agree, but from the other side of the fence.
<br /><br /> scott.marlowe wrote:<br /><blockquote cite="midPine.LNX.4.33.0302131358470.23299-100000@css120.ihs.com" type="cite"><prewrap=""></pre><blockquote type="cite"><pre wrap="">These are not issues at all. You could put the configurationfile anywhere, just as you can for any UNIX service. postmaster --config=/home/myhome/mydb.conf I deal with a number of PG databases on a number of sites, and it is a real pain in the ass to get to a PG box and hunt around for data directory so as to be able to administer the system. What's really annoying is when you have to find the data directory when someone else set up the system. </pre></blockquote><pre wrap=""> Really? I would think it's easier to do this: su - pgsuper cd $PGDATA pwd Than to try to figure out what someone entered when they ran ./configure --config=... </pre></blockquote> Why do you think PGDATA would be set for root?<br /><br /><blockquote cite="midPine.LNX.4.33.0302131358470.23299-100000@css120.ihs.com"type="cite"><pre wrap=""> </pre><blockquote type="cite"><prewrap="">Configuring postgresql via a configuration file which specifies all the data, i.e. data directory, name of other configuration files, etc. is the right way to do it. Even if you have reasons against it, even if you think it is a bad idea, a bad standard is almost always a better solution than an arcane work of perfection. </pre></blockquote><pre wrap=""> Wrong, I strongly disagree with this sentament. Conformity to standards for simple conformity's sake is as wrong as sticking to the old way because it's what we're all comfy with. </pre></blockquote> It isn't conformity for conformitys sake. It is following anestablished practice, like driving on the same side of the road or stopping at red lights.<br /><blockquote cite="midPine.LNX.4.33.0302131358470.23299-100000@css120.ihs.com"type="cite"><pre wrap=""> </pre><blockquote type="cite"><pre wrap="">Personally, however, I think the configuration issue is a no-brainer and I am amazed that people are balking. EVERY other service on a UNIX box is configured in this way, why not do it this way in PostgreSQL? The patch I submitted allowed the configuration to work as it currently does, but allowed for the more standard configuration file methodology. </pre></blockquote><pre wrap=""> If I do a .tar.gz install of apache, I get /usr/local/apache/conf, which is not the standard way you're listing. If I install openldap from .tar.gz, I get a /usr/local/etc/openldap directory, close, but still not the same. The fact is, it's the packagers that put things into /etc and whatnot, and I can see the postgresql RPMs or debs or whatever having that as the default, but for custom built software, NOTHING that I know of builds from source and uses /etc without a switch to tell it to, just like postgresql can do now.</pre></blockquote> You are confusing the default location of a file with the ability to use the file.The default I have proposed all along was to use the existing practice of keeping everything in the $PGDATA directory.<br/><br /> The change I wish to make to the code allows this to be changed. Most admins want configuration anddata separate. Most admins do not want to use symlinks because they are dangerous in a production environment.<br /><br/> I would rather have a simpler solution sooner than a perfect solution never.<br /><br />
> All I see here is an arbitrary break with our past practice. I do
> not see any net improvement.
<FreeBSD Port Maintainer>
Well, given that there's a trend to make PostgreSQL more usable, I can
say with absolute certainty, that an FAQ that I get about once a week
is (and granted only from new users) "where is the postgresql.conf? I
don't see it in ${LOCALBASE}/etc/." PostgreSQL is one of a few ports
in an extreme minority that uses a local configuration directive and
it violates the policy of least surprise for sysadmins.
PS LOCALBASE/PREFIX is /usr/local 99.999% of the time
</FreeBSD Port Maintainer>
With my DBA hat on, however, here are a few reasons that I'd like to
see the conf moved out of the data directory:
1) pg_dumpall > foo && rm -rf $PGDATA && initdb
As a DBA I don't have to worry about backing up my config file when doing upgrades since the config file is located
inan external directory.
2) Backing up config files in ${LOCALBASE}/etc is a pretty common practice. Having to make a special case for
postgresql'skind of a PITA.
Suggestions:
1) gmake install installs a default configuration file in ${LOCALBASE}/etc/postgresql.conf.default. Promote that DBAs
should diff postgresql.conf.default with postgresql.conf and make adjustments as they see fit, but gmake install will
_not_,under any circumstances, touch postgresql.conf (by default, it should cp postgresql.conf.default to
postgresql.confthat way things "just work" out of the box).
2) Leave the current functionality in place. Being able to have multiple databases on the same machine is a _really_
nicefeature of PostgreSQL. If you want multiple databases, having the config file in $PGDATA makes some sense
becausewith multiple installations, you want to keep everything together... though it doesn't make much sense if you
haveonly one installation per server... and really, the only reason to have multiple installations is to handle
usernamecollisions (hint hint).
3) In the absence of a PGDATA environment variable (don't want to break backward compatible installations) being set,
thefuture behavior allow for a default location of a config file (if no CLI switch is specified for an explicit
location)that points to a config file. The path would be ${PREFIX}/etc and would provide most admins with a standard
launchingoff point for running/tuning their databases. The config file would have to specify the data directory as
wellas the path to the hba.conf, which should be outside of the datadir as well (speaking of the hba.conf, am I the
onlyone who things that hba.conf should be converted into a system catalog? ::shrug::)
Just some random thoughts from someone who's had to deal with this on
all of the mentioned levels (new users, single installations, multiple
installations, and multiple copies running via daemontools). -sc
PS If there is no huge press for this, I should have the time do do
this in a few weeks if someone doesn't beat me to it.
--
Sean Chittenden
On Thu, 2003-02-13 at 21:21, Vince Vielhaber wrote: > I certainly wasn't trying to provoke anything. It just seems odd to me > that when the distribution installs a package and places it's config files > in /etc and later the admin happens to upgrade by the instructions with > the package, it's acceptable for the config files to now be in two places > and you don't find it confusing. What happens when a new admin comes on > and tries to figure out which config file is which? Ever try to figure > out where the hell Pine's config really is? I've not used pine, and there doesn't seem to be an official Debian package, (it doesn't allow any changes to its source, I believe, which makes it ineligible). But if it were an official package, I know I should look in /etc/pine. If the admin installs a local build of something he has installed as a package, he will presumably take care to separate the two. If his local build is to replace the package, he should purge the installed package, so that there are no traces of it left. Since he is administering a distribution installation, it is certainly his responsibility to understand the difference between local and distributed packages, as well as the different places that each should put their configuration files. (Incidentally, Debian's changes from the upstream configuration are documented in the package.) In the end, though, when we package for a distribution, we expect people to use the packages. If they want to build from source, the packages system lets them do it. Anyone who is building from the upstream source must be presumed to know what he is doing and take responsibility for it. What your comments strongly suggest to me is that projects like PostgreSQL and pine, along with everything else, should comply with FHS; then there will be no confusion because everyone will be following the smae standards. Messes arise when people ignore standards; we have all seen the dreadful examples of MySQL and the Beast, haven't we? -- Oliver Elphick Oliver.Elphick@lfix.co.uk Isle of Wight, UK http://www.lfix.co.uk/oliver GPG: 1024D/3E1D0C1C: CA12 09E0 E8D5 8870 5839 932A 614D 4C34 3E1D 0C1C ======================================== "The earth is the LORD'S, and the fullness thereof; the world, and they thatdwell therein." Psalms 24:1
On Thu, 2003-02-13 at 14:28, Bruce Momjian wrote: > Robert Treat wrote: > > On Thu, 2003-02-13 at 14:06, mlw wrote: > > > > > > I will be resubmitting my patch for the 7.3.2 tree. > > > > > > > I'm no core developer, but surely this wont be included in the 7.3.x > > branch. Any change needs to be made against CVS head. > > I assume he meant he will repost his 7.3.2-based patch and we will merge > it into CVS HEAD if it is accepted. > IIRC he originally wrote the patch for a pre 7.3 version, so it seems like he'd be reworking it for 7.3.x with the above statement. I'm only suggesting he rework it against CVS head if he doesn't have plans to do so already. Course if yall are willing to merge it in for him, none of this really matters does it? :-) Robert Treat
On Thu, 2003-02-13 at 14:43, Bruce Momjian wrote: > Robert Treat wrote: > > On Thu, 2003-02-13 at 12:13, mlw wrote: > > > > > > My patch only works on the PostgreSQL server code. No changes have been > > > made to the initialization scripts. > > > > > > The patch declares three extra configuration file parameters: > > > hbafile= '/etc/postgres/pg_hba.conf' > > > identfile='/etc/postgres/pg_ident.conf' > > > datadir='/RAID0/postgres' > > > > > > > If we're going to do this, I think we need to account for all of the > > files in the directory including PG_VERSION, postmaster.opts, > > postmaster.pid. In the end if we can't build so that we are either fully > > FHS compliant and/or LSB compliant, we've not done enough work on it. > > Woh, how do we move some of those files into /etc or /var/run if we > aren't running as root? We certainly don't want to require that. I > guess /etc/postgresql will work if that directory is owned by the > PostgreSQL superuser, but /var/run will be a problem. > Seems like some are saying one of the problems with the current system is it doesn't follow FHS or LSB. If those are valid reasons to change the system, it seems like a change which doesn't actually address those concerns would not be acceptable. (Unless those really aren't valid concerns...) Robert Treat
On Thu, 2003-02-13 at 14:51, mlw wrote: > > > Robert Treat wrote: > > > On Thu, 2003-02-13 at 12:13, mlw wrote: > > > > My patch only works on the PostgreSQL server code. No changes have been > > made to the initialization scripts. > > > > The patch declares three extra configuration file parameters: > > hbafile= '/etc/postgres/pg_hba.conf' > > identfile='/etc/postgres/pg_ident.conf' > > datadir='/RAID0/postgres' > > > If we're going to do this, I think we need to account for all of the > > files in the directory including PG_VERSION, postmaster.opts, > > postmaster.pid. In the end if we can't build so that we are either fully > > FHS compliant and/or LSB compliant, we've not done enough work on it. > > > postmaster.opts, PG_VERSION, and postmaster.pid are not configuration > parameters. > So? I'm not saying they all have to be moved, just they all need to be accounted for. > PG_VERSION is VERY important, it is how you know the version of the > database. > Postmaster.pid is a postgres writable value > AFAIK, postmaster.opts is also a postgres writable value. > IIRC the postmaster.pid file should be in /var/run according to FHS, I'm not sure about postmaster.opts though... Again, if we're going to make a change, let's make sure we think it through. Robert Treat
Oliver Elphick wrote: > What your comments strongly suggest to me is that projects like > PostgreSQL and pine, along with everything else, should comply with FHS; > then there will be no confusion because everyone will be following the > smae standards. Messes arise when people ignore standards; we have all > seen the dreadful examples of MySQL and the Beast, haven't we? Can the FHS handle installing PostgreSQL as non-root? -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001+ If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania19073
Robert Treat wrote: > IIRC the postmaster.pid file should be in /var/run according to FHS, I'm > not sure about postmaster.opts though... > > Again, if we're going to make a change, let's make sure we think it > through. Can non-root write to /var/run? -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001+ If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania19073
On Thu, 2003-02-13 at 15:08, mlw wrote: > Stephan Szabo wrote: > > Re-read my statement and yours about the case you were mentioning. ;) > > Sure, putting the files in /etc lets you find them easily. However, if > > you're doing things like finding configuration made by someone else and > > said configuration isn't in /etc (which if they wanted to they could do > > now with symlinks I believe - yes symlinks aren't a complete solution, > but > > I think they're reasonable on most of our current ports) then you still > > have to search the system for the configuration file, except now it > might > > not even be postgresql.conf. That's why I said the two issues aren't the > > same. > Actually, I'd almost go so far as to say it will make it worse. In the current system, if you can figure out where $PGDATA is, you've found everything you need for that installation. In the new system, there's no telling where people will put things, and it certainly won't be any easier to find it. THinking on the above Stephan, you'd almost have to require that the config file be called postgresql.conf in order to run, anything else leads to real scary scenario's. > On Thu, 13 Feb 2003, mlw wrote: > > I have absolutely no problem debating and augmenting the feature. None > what so ever, I am more pushing to get momentum to actually do it. Stick with it, I think most of us here can see the value in the option, but there are valid concerns that it be implemented correctly. Personally I think a postgresql installation is much more like an apache installation, which generally contains all of the files (data and config) under /usr/local/apache. Maybe someone can dig more to see if that system is more appropriate a comparison than something like bind. Robert Treat
<br /><br /> Bruce Momjian wrote:<br /><blockquote cite="mid200302132255.h1DMtGB26234@candle.pha.pa.us" type="cite"><prewrap="">Robert Treat wrote: </pre><blockquote type="cite"><pre wrap="">IIRC the postmaster.pid file shouldbe in /var/run according to FHS, I'm not sure about postmaster.opts though... Again, if we're going to make a change, let's make sure we think it through. </pre></blockquote><pre wrap=""> Can non-root write to /var/run? </pre></blockquote> Shouldn't be able too<br />
On 13 Feb 2003, Oliver Elphick wrote: > What your comments strongly suggest to me is that projects like > PostgreSQL and pine, along with everything else, should comply with FHS; > then there will be no confusion because everyone will be following the > smae standards. Messes arise when people ignore standards; we have all > seen the dreadful examples of MySQL and the Beast, haven't we? Actually FHS says the opposite. If the distribution installs PostgreSQL then the config files belong in /etc/postgresql. If the admin does then they belong in /usr/local/etc/postgresql. FHS is out of their tree. If PostgreSQL or any other package is not critical to the basic operation of the operating system, it's config files shouldn't be polluting /etc. Vince. -- Fast, inexpensive internet service 56k and beyond! http://www.pop4.net/ http://www.meanstreamradio.com http://www.unknown-artists.com Internet radio: It's not file sharing, it's just radio.
<br /><br /> Robert Treat wrote:<br /><blockquote cite="mid1045176269.12930.576.camel@camel" type="cite"><pre wrap="">OnThu, 2003-02-13 at 14:51, mlw wrote: </pre><blockquote type="cite"><pre wrap=""> Robert Treat wrote: On Thu, 2003-02-13 at 12:13, mlw wrote: My patch only works on the PostgreSQL server code. No changes have been made to the initialization scripts. The patch declares three extra configuration file parameters: hbafile= '/etc/postgres/pg_hba.conf' identfile='/etc/postgres/pg_ident.conf' datadir='/RAID0/postgres' If we're going to do this, I think we need to account for all of the files in the directory including PG_VERSION, postmaster.opts, postmaster.pid. In the end if we can't build so that we are either fully FHS compliant and/or LSB compliant, we've not done enough work on it. postmaster.opts, PG_VERSION, and postmaster.pid are not configuration parameters. </pre></blockquote><pre wrap=""> So? I'm not saying they all have to be moved, just they all need to be accounted for. </pre></blockquote> OK, what was the point?<br /><br /><blockquote cite="mid1045176269.12930.576.camel@camel"type="cite"><blockquote type="cite"><pre wrap="">PG_VERSION is VERY important,it is how you know the version of the database. Postmaster.pid is a postgres writable value AFAIK, postmaster.opts is also a postgres writable value. </pre></blockquote><pre wrap=""> IIRC the postmaster.pid file should be in /var/run according to FHS, I'm not sure about postmaster.opts though... Again, if we're going to make a change, let's make sure we think it through.</pre></blockquote> I'm not a big fan of the "/var/run" directory convention, especially when we expect multipleinstances of the server to be able to run concurrently. I suppose it can be a parameter in both the configurationfile and command line.<br /><br />
On Thursday 13 February 2003 17:53, Bruce Momjian wrote: > Oliver Elphick wrote: > > What your comments strongly suggest to me is that projects like > > PostgreSQL and pine, along with everything else, should comply with FHS; > > then there will be no confusion because everyone will be following the > > smae standards. Messes arise when people ignore standards; we have all > > seen the dreadful examples of MySQL and the Beast, haven't we? > Can the FHS handle installing PostgreSQL as non-root? Once again, no one is trying to make an FHS install the default 'let's force everyone to think our way or no way' coercion. We just want the option. For those who wish to do non-root installs, nothing would need to change. You can still put it into /usr/local/pgsql (assuming you have permissions to put it there) or your home directory, or wherever. I deal with RPMs; Oliver deals with .deb's. Neither can be installed as non-root. The daemon can of course run as non-root (and it does, which is exactly correct); but the installation of the files is done as root _always_ in an RPM or deb environment. So I really don't care about non-root installs; sorry. I wonder what percentage of our users are not the administrator of the machine on which they are running PostgreSQL? I dispute the statement made earlier in the thread (not by Bruce) that PostgreSQL is by definition not an OS service. This is false, and needs to be realized by this community. PostgreSQL is becoming an essential OS core service in many cases: virtually all Linux distributions (the lion's share of our current distribution) include PostgreSQL as a core service. Many of our new users see PostgreSQL as 'SQL server' in the Red Hat installation menu. Now, on a Win32 server, what is PostgreSQL going to be considered? It is probably going to run as a service, right? So you need to be Administrator there to perform the install, right? This isn't the same environment, Bruce, that you got into back when it was still Postgres95. We are in the big leagues OS-wise, and we need to act like it. Assuming that we are a 'userspace' program (which is a misnomer anyway, as _anything_ non-kernel is 'userspace') is not going to cut it anymore. So we need to fit in to an OS environment, whether it is FreeBSD, OS/X, Win32, Solaris, or Linux. In FreeBSD, as the ports maintainer excellently posted, PostgreSQL should live in LOCALBASE. We should make that easy. In Win32, configuration might be better stored in the system registry (Argh! Did I actually say THAT! Yuck!) -- we should make even that easy. In OS/X we should use the OS/X paradigm (whatever that is). And we should make it easy to make PostgreSQL LSB-compliant for our very large Linux user community. We should be adaptable to the accepted administration paradigm on whatever system we are running -- this should be a minimum. These concerns vastly outweigh the occasional non-root install from source, in my mind at least. I am not opposed to that way even being the default; after all, leaving the default the same as now agrees with the principle of least surprise (although we really don't ascribe to that; witness the 7.2-7.3 migration fiasco -- 7.3 should have been 8.0 to warn people of the major changes going on in client connections). But I do advocate _allowing_ the configuration options Mark has enumerated -- although I really wish we could use the lowercase c instead, for consistency with other OS services. -- Lamar Owen WGCR Internet Radio 1 Peter 4:11
On Thursday 13 February 2003 18:07, Vince Vielhaber wrote: > Actually FHS says the opposite. If the distribution installs PostgreSQL > then the config files belong in /etc/postgresql. If the admin does then > they belong in /usr/local/etc/postgresql. FHS is out of their tree. If > PostgreSQL or any other package is not critical to the basic operation of > the operating system, it's config files shouldn't be polluting /etc. PostgreSQL is as critical as PHP, Apache, or whatever other package is being backended by PostgreSQL. If the package is provided by the distributor, consider it part of the OS. If it isn't, well, it isn't. This is so that local admin installed (from source -- not from binary package) files don't get clobbered by the next operating system binary upgrade. In that context the FHS (LSB) mandate makes lots of sense. -- Lamar Owen WGCR Internet Radio 1 Peter 4:11
On Thu, 13 Feb 2003, Lamar Owen wrote: > On Thursday 13 February 2003 18:07, Vince Vielhaber wrote: > > Actually FHS says the opposite. If the distribution installs PostgreSQL > > then the config files belong in /etc/postgresql. If the admin does then > > they belong in /usr/local/etc/postgresql. FHS is out of their tree. If > > PostgreSQL or any other package is not critical to the basic operation of > > the operating system, it's config files shouldn't be polluting /etc. > > PostgreSQL is as critical as PHP, Apache, or whatever other package is being > backended by PostgreSQL. If the package is provided by the distributor, > consider it part of the OS. If it isn't, well, it isn't. You completely miss my point, but lately you've been real good at that. Can the system boot without PHP, Apache, PostgreSQL, Mysql and/or Pine? Can the root user log in without PHP, Apache, PostgreSQL, Mysql and/or Pine? Can any user log in without PHP, Apache, PostgreSQL, Mysql and/or Pine? Note, I'm not even including an MTA here. I said BASIC OPERATION. If a package is not critical as I just outlined, it shouldn't matter who installed it. After the last go around with you Lamar, this will be my last response to you on this. Vince. -- Fast, inexpensive internet service 56k and beyond! http://www.pop4.net/ http://www.meanstreamradio.com http://www.unknown-artists.com Internet radio: It's not file sharing, it's just radio.
On Thursday 13 February 2003 18:41, Vince Vielhaber wrote: > On Thu, 13 Feb 2003, Lamar Owen wrote: > > PostgreSQL is as critical as PHP, Apache, or whatever other package is > > being backended by PostgreSQL. If the package is provided by the > > distributor, consider it part of the OS. If it isn't, well, it isn't. > You completely miss my point, but lately you've been real good at that. No, Vince, I understand your point. But understand mine: it does matter who installed it. > Note, I'm not even including an MTA here. I said BASIC OPERATION. > If a package is not critical as I just outlined, it shouldn't matter who > installed it. 'Critical' is in the eye of the admin of the system in question. For my servers, if, for instance, sshd doesn't come up, then there's a major problem, as they are all headless. If the webserver doesn't come up, I have other problems, as OpenACS is mission-critical here. So what's critical is a question for the individual sysadmin. So, to continue your point, what is 'critical' to the 'basic operation' of the system shouldn't pollute /etc. So, let's eliminate the /etc/mail, /etc/samba, /etc/xinetd.d, /etc/X11, /etc/httpd, and the other subtrees foung in at least Red Hat 8. While we're at it, many other files in /etc need to go: named.conf for one. It depends on what you consider 'critical'. PostgreSQL is at least as critical on my systems as some of the other things that already 'pollute' /etc. > After the last go around with you Lamar, this will be my last response > to you on this. Aw Vince, I don't know what your problem is with conflicting opinions. But that's your choice. And Open Source is about _choice_. You are free to admin your systems your way, and I'm free to do so my way. And all's well. -- Lamar Owen WGCR Internet Radio 1 Peter 4:11
On Thu, Feb 13, 2003 at 05:59:17PM -0500, Robert Treat wrote:
> On Thu, 2003-02-13 at 15:08, mlw wrote:
> > Stephan Szabo wrote:
> >
> > On Thu, 13 Feb 2003, mlw wrote:
> >
> > I have absolutely no problem debating and augmenting the feature. None
> > what so ever, I am more pushing to get momentum to actually do it.
>
> Stick with it, I think most of us here can see the value in the option,
> but there are valid concerns that it be implemented correctly.
> Personally I think a postgresql installation is much more like an apache
> installation, which generally contains all of the files (data and
> config) under /usr/local/apache. Maybe someone can dig more to see if
> that system is more appropriate a comparison than something like bind.
I think you are making a pretty uninformed, if not just plain wrong
generalization. I've run exactly one system with apache configuration
files in /usr/local/apache, and even then, the data was not there.
A quick straw poll of the people I know who actually do run real systems
also mentioned that they use packaging systems like encap or rpm to manage
upgrades, and would almost never put datafiles into /usr/local.
RedHat (7.3 at least)'s default httpd datafiles go in /var/www/html and
config goes in /etc/httpd
One OpenBSD user I talked to puts his in /home/www and config files in
/etc/httpd. The defaults are /var/www and /var/www/conf
Another user reports:
On systems that I set up I have /web/{apache|httpd}/ and put all
the config info there.
And /web/sites/name/ holds site data.
What does this mean?
People will put things in different places, and there are typically
very good reasons for this. This is ESPECIALLY true when one wants to
have configuration files, at least the base ones in a common place such
as /etc or /usr/local/etc in order to make backup of configuration easy
and clean, while leaving data somewhere else for performance or magnitude
of partition reasons. It just makes sense to ME to have postgresql.conf
reside in /etc, yet put my data in /var/data/postgresql, yet retain the
option to put my data in /raid/data/postgresql at a later date, when the
new hardware comes in.
Yes, symlinks are an option on most systems. No, they are not a good
one on most systems.
What _I_ would like to see:
o. a default postgresql.conf location of $PREFIX/data/postgresql.conf
o. a default PGDATA location of whatever directory postgresql.conf is in
(this should maintain backward compatibility)
o. a ./configure - time option to override the location of the postgresql.conf
o. a run-time option to override the location of the postgresql.conf
o. options in postgresql.conf to specify the location of PGDATA and PID files.
($PREFIX is already settable at ./configure - time)
This would allow:o. Config files in /usr/local/pgsql/data, /etc, /usr/local/etc, ~postgresql, or /dev/.hidden-node,
whicheveryou prefer, so long as you either knowthe compile-time default, or are willing to specify it at startup.
o. Datafiles to be in /usr/local/pgsql/data, /var/data, /raid0, /nfs/bigmountor whichever you prefer, so long as you
eitherknow the compile-time default,or are willing to specify it in a config file that you specify at startup.
Does it add complexity to the system? Sure -- a very little bit, IMHO, especially
compared to the BTREE-folding that I see being bantered about.
Is it some work? Sure -- a very little bit, and it seems that it has already
been done.
However, this seems, to me, to be a very small addition that has some real-world
(and yes, we need to start paying attention to the real world) advantages.
And finally, don't go telling me that I'm wrong to put my data and config files
where I am. You can offer advice, but I'm probably going to ignore it because
I like where they are and don't need to explain why.
--
Adam Haberlach | "Because manholes are round."
adam@mediariffic.com |
http://mediariffic.com |
Lamar Owen wrote:
> This isn't the same environment, Bruce, that you got into back when it was
> still Postgres95. We are in the big leagues OS-wise, and we need to act like
> it. Assuming that we are a 'userspace' program (which is a misnomer anyway,
> as _anything_ non-kernel is 'userspace') is not going to cut it anymore.
So you are saying this isn't my grandma's database anymore. :-)
Anyway, I think I have _a_ proposal that we can use to work toward a
goal.
First, a few conclusions:
We can't use /var/run because we need the postmaster to createthose, and it isn't root.
Right now, the fact that we mix the config stuff with thedata isn't ideal. Someone mentioned:
pg_dumpall > foo && rm -rf $PGDATA && initdb
discards all the config files.
So, I propose we change a few things. The good news is that this is
something only administrators deal with; client apps don't deal with
it.
OK, first, we keep postmaster.pid and postmaster.opts in /data. We
can't put them in /var/run, and /data seems like the best spot for them.
That leaves postgresql.conf, pg_hba.conf, and pg_ident.conf. I
recommend moving them all, by default, into pgsql/etc. I recommend we
add these to postgresql.conf:
data_dir = ../datapg_hba_dir = ./pg_ident_dir = ./
Those paths are relative to postgresql.conf.
We then add a PGCONFIG variable and postmaster -C flag to point to the
config _directory_. That way, if folks want to move all of this into
/etc, then easily do that. This also pulls those files out of /data so
they are easier to back up.
We can also firm up stuff in 7.5 by removing PGDATA and -D, and perhaps
removing the other duplicate postmaster flags that have postgresql.conf
entries.
-- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610)
359-1001+ If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square,
Pennsylvania19073
On Thursday 13 February 2003 20:09, Bruce Momjian wrote: > Lamar Owen wrote: > > This isn't the same environment, Bruce, that you got into back when it > > was still Postgres95. > So you are saying this isn't my grandma's database anymore. :-) I actually thought of saying it that way, too. :-) > Anyway, I think I have _a_ proposal that we can use to work toward a > goal. > First, a few conclusions: > We can't use /var/run because we need the postmaster to create > those, and it isn't root. It isn't without precedent to have a directory under /var/run. Maybe /var/run/postgresql. Under this one could have a uniquely named pid file. I say uniquely named so that multiple postmasters could run. Naming those files could be fun. /var/run/postgresql would be owned by the postmaster run user. This of course requires root to install -- but would be completely optional. > pg_dumpall > foo && rm -rf $PGDATA && initdb > discards all the config files. Yes, this is a big deal. It makes it more difficult to properly restore. While it's not impossible to do so now, of course, it just could be a little easier. > So, I propose we change a few things. > OK, first, we keep postmaster.pid and postmaster.opts in /data. We > can't put them in /var/run, and /data seems like the best spot for them. Can we make that configurable? The default in pgdata is fine; just having the option is good. > That leaves postgresql.conf, pg_hba.conf, and pg_ident.conf. I > recommend moving them all, by default, into pgsql/etc. I recommend we > add these to postgresql.conf: > data_dir = ../data > pg_hba_dir = ./ > pg_ident_dir = ./ > Those paths are relative to postgresql.conf. And these are all just defaults, easily changed. Good. > We then add a PGCONFIG variable and postmaster -C flag to point to the > config _directory_. That way, if folks want to move all of this into > /etc, then easily do that. This also pulls those files out of /data so > they are easier to back up. Yes. I'm thinking along the lines of this sort of structure: /etc |---postgresql |----- name of postmaster one (unique ID of some kind) |----- name of postmastertwo . . Not difficult. > We can also firm up stuff in 7.5 by removing PGDATA and -D, and perhaps > removing the other duplicate postmaster flags that have postgresql.conf > entries. Now I really _like_ this idea. By removing it to 7.5, and therefore deprecating it in 7.4, this brings best practice into effect. However, at the same time, I wouldn't be opposed to leaving them in place, either, for backwards compatibility. -- Lamar Owen WGCR Internet Radio 1 Peter 4:11
Bruce Momjian wrote: > I don't think separate params for each config file is good. At the > most, I think we will specify the configuration _directory_ for all the > config files, perhaps pgsql/etc, and have pgdata default to ../data, or > honor $PGDATA. That might be the cleanest. > > Of course, that now gives us $PGCONFIG and $PGDATA, and possible > intraction if postgresql.conf specifies a different pgdata from $PGDATA. > As you can see, it could get messy. Uh...why are we having to mess with environment variables at all? It's one thing for shell scripts to make use of them, but another thing entirely for an executable like the postmaster to do the same. Seems logical to me to eliminate the use of $PGDATA in the postmaster entirely. It usually gets started from a shell script, so let the shell script pass the appropriate parameter telling the postmaster where to find the data, or the config files, or whatever. > And, if you specify pgdata in postgresql.conf, it prevents you from > using that file by different postmasters. Not at all. Don't GUC variables that are specified on the command line override the ones in the configuration file? > My best guess would be to not specify pgdata in postgresql.conf, and > have a new $PGCONFIG param to specify the configuration directory, but > if we do that, $PGDATA/postgresql.conf becomes meaningless, which could > also be confusing. Maybe we don't allow those files to exist in $PGDATA > if $PGCONFIG is used, _and_ $PGCONFIG is not the same as $PGDATA. See, > I am getting myself confused. :-) I think the solution is real simple: 1. Eliminate the use of $PGDATA in the postmaster. It causes far more headaches than it's worth. Instead, require that-D be passed on the command line. It's fine if the postmaster *sets* $PGDATA in order to minimize any changes thatneed to be made elsewhere, but the postmaster should not use it until it sets it. The postmaster right now readsall the config files (including postgresql.conf) from the directory specified by the -D option. Keep it that way. 2. Add a GUC variable that specifies where the data is. If this variable is not defined either on the command line orin the config file, then assume that the data is in the same place as the config file. Obviously files like PG_VERSIONare associated with the data and not with the config, so they get treated appropriately. The above addresses *everyone's* concerns that I've seen thus far, I think. Thoughts? -- Kevin Brown kevin@sysexperts.com
Lamar Owen wrote: > > First, a few conclusions: > > > We can't use /var/run because we need the postmaster to create > > those, and it isn't root. > > It isn't without precedent to have a directory under /var/run. Maybe > /var/run/postgresql. Under this one could have a uniquely named pid file. I > say uniquely named so that multiple postmasters could run. Naming those > files could be fun. /var/run/postgresql would be owned by the postmaster run > user. This of course requires root to install -- but would be completely > optional. But how do you handle the default then, where you have postmaster.pid in /data? Do we rename it to postmaster.pid.5432 so it can sit in /var/run/postgresql alone with other backends? Another issue is that pg_ctl looks at that file, so moving it around is going to be tricky. Also, this brings up a new issue that pg_ctl all of a sudden can't just look at $PGDATA but must instead grope through postgresql.conf to find the data directory location. That could be interesting. Of course, it can still supply the /data path on the command line, but if we use only $PGCONFIG, we would need to have it find /data automatically from postgresql.conf. > > OK, first, we keep postmaster.pid and postmaster.opts in /data. We > > can't put them in /var/run, and /data seems like the best spot for them. > > Can we make that configurable? The default in pgdata is fine; just having the > option is good. Basically, I am saying that unless someone wants to use this configurability, it is going to cause code confusion so it is best avoided. ` > Yes. I'm thinking along the lines of this sort of structure: > /etc > |---postgresql > |----- name of postmaster one (unique ID of some kind) > |----- name of postmaster two > . > . > > Not difficult. Yes, that would work easily. > > We can also firm up stuff in 7.5 by removing PGDATA and -D, and perhaps > > removing the other duplicate postmaster flags that have postgresql.conf > > entries. > > Now I really _like_ this idea. By removing it to 7.5, and therefore > deprecating it in 7.4, this brings best practice into effect. > > However, at the same time, I wouldn't be opposed to leaving them in place, > either, for backwards compatibility. The problem is that we would be having too many ways to specify the /data directory. I am now wondering if we even want pg_hba_dir and pg_ident_dir. Seems we can assume they are in the same directory as postgresql.conf. That leaves only data_dir as new for postgresql.conf. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001+ If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania19073
Vince Vielhaber wrote: > On Thu, 13 Feb 2003, Lamar Owen wrote: > > > On Thursday 13 February 2003 18:07, Vince Vielhaber wrote: > > > Actually FHS says the opposite. If the distribution installs PostgreSQL > > > then the config files belong in /etc/postgresql. If the admin does then > > > they belong in /usr/local/etc/postgresql. FHS is out of their tree. If > > > PostgreSQL or any other package is not critical to the basic operation of > > > the operating system, it's config files shouldn't be polluting /etc. > > > > PostgreSQL is as critical as PHP, Apache, or whatever other package is being > > backended by PostgreSQL. If the package is provided by the distributor, > > consider it part of the OS. If it isn't, well, it isn't. > > You completely miss my point, but lately you've been real good at that. > > Can the system boot without PHP, Apache, PostgreSQL, Mysql and/or > Pine? Yep. > Can the root user log in without PHP, Apache, PostgreSQL, Mysql > and/or Pine? Hopefully. > Can any user log in without PHP, Apache, PostgreSQL, Mysql and/or > Pine? That depends, doesn't it? There exist PAM modules that allow authentication against a database, for instance. If you're using them and the database doesn't come up, the users can't log in. So suddenly the database config files belong in /etc? The mission of the box is what counts. If the mission of the box is to be a web server then I'm probably not going to care whether non-root users can log into it: that simply doesn't factor into the mission profile. The web server process is going to be as critical to the mission of the box as almost anything else on it, as will anything the web server process depends on -- which may well include a database. > Note, I'm not even including an MTA here. I said BASIC OPERATION. So by your reasoning sendmail.cf doesn't belong in /etc?? I dare say that's news to most of us. Where, then, *does* it belong? > If a package is not critical as I just outlined, it shouldn't matter > who installed it. Oh, it matters a great deal, because people upgrade their OS installs from time to time. Many OS distributions come with a lot of packages that aren't "critical" as you define them but which nevertheless will cause much pain and suffering for the sysadmin if they install themselves over what the sysadmin has previously built by hand. The purpose for differentiating between a package that was compiled and installed from the source by the sysadmin and a prebuilt package that was provided to the sysadmin by the vendor is to keep them from stepping on each other -- if the sysadmin went to the trouble of compiling and installing a package from the source instead of using a prebuilt version from the vendor, then he probably did so for a very good reason, and is going to be *really* annoyed if an OS upgrade blows away his work. There are some good reasons for putting all the config files in /etc, one of them being that it gives you *one* directory full of config files to worry about backing up instead of many. If you've got other ideas I'm certainly interested in hearing the reasoning behind them. But from the point of view of maintaining a widely deployed package like PostgreSQL, the conventions the distributions and sysadmins use matter a great deal, whether or not you happen to agree with those conventions. -- Kevin Brown kevin@sysexperts.com
> On 13 Feb 2003, Oliver Elphick wrote: > > > What your comments strongly suggest to me is that projects like > > PostgreSQL and pine, along with everything else, should comply with FHS; > > then there will be no confusion because everyone will be following the > > smae standards. Messes arise when people ignore standards; we have all > > seen the dreadful examples of MySQL and the Beast, haven't we? > > Actually FHS says the opposite. If the distribution installs PostgreSQL > then the config files belong in /etc/postgresql. If the admin does then > they belong in /usr/local/etc/postgresql. FHS is out of their tree. If > PostgreSQL or any other package is not critical to the basic operation of > the operating system, it's config files shouldn't be polluting /etc. I suspect you may be conflating BSD usage with Linux usage here... The point isn't of being "critical to basic operation of the operating system;" it is of whether or not the software is being "package-managed" or not. One of the operating principles in FHS is that "/usr/local" is an area that the distribution should never "pollute." And so, a "package-managed" PostgreSQL installation should never touch that area. Looking at FHS, for a moment: http://www.pathname.com/fhs/2.2/ 3.7.1 Purpose /etc contains configuration files and directories that are specific to the current system. 3.7.4 Indicates that "Host-specific configuration files for add-on application software packages must be installed within the directory /etc/opt/<package>, where <package> is the name of the subtree in /opt where the static data from that package is stored." 3.12 indicates: /opt is reserved for the installation of add-on application software packages. A package to be installed in /opt must locate its static files in a separate /opt/<package> directory tree, where <package> is a name that describes the software package. Then comes 5.1, on /var /var contains variable data files. This includes spool directories and files, administrative and logging data, and transient and temporary files. It would make most sense, based on FHS, for PostgreSQL information to assortedly reside in: - /etc/opt/postgresql or /etc/postgresql, for static config information; - Binaries could assortedly live in /usr/bin or /opt/postgresql; - Logs should live in /var/log or /var/log/postgresql; - Data could assortedly live in /var/lib/postgresql, /var/opt/postgresql; - PIDs should live in /var/lock or /var/lock/postgresql. None of these choices should come as any spectacular shock to anyone; there are an assortment of sets of bigotry out there surrounding the Proper Purposes of /opt and /usr/local, and there's probably enough wriggle room there to avoid overly enraging anyone that (for instance) felt calling a directory "/opt" would make someone deserving of carpet bombing by B-52s. Interestingly, the Debian install of PostgreSQL somewhat resembles this, with, assortedly: /etc/postgresql /etc/postgresql/postgresql.conf /etc/postgresql/postmaster.conf /etc/postgresql/pg_hba.conf /etc/postgresql/pg_ident.conf /etc/init.d/postgresql /usr/share/doc/postgresql /usr/share/man/man1/pg_ctl.1.gz /usr/lib/postgresql /usr/lib/postgresql/bin/postgres /usr/lib/postgresql/bin/enable_lang /usr/lib/postgresql/bin/initdb /usr/lib/postgresql/bin/initlocation /usr/lib/postgresql/bin/ipcclean /usr/lib/postgresql/bin/pg_ctl /usr/lib/postgresql/bin/pg_dumpall /var/run/postgresql (This is obviously incomplete; this just gives the flavor that there are files in a reasonably rational but diverse assortment of places.) Note that the server software hides in /usr/lib/postgresql/bin; it's not stuff you should normally run from the command line, so, quel surprise, it is stashed somewhere that's unlikely to be in your $PATH. Stashing _everything_ in /var/lib/postgres would seem a tad surprising. Mind you, if I need to manage 4 instances on one box, I might very well install several instances some place custom, say /opt/postgres, or similar, and in that case, it's probably preferable for everything to reside clearly underneath that, and for my custom backup scripts to work in that area. But if I'm managing 4 instances on one box, it should be quite evident that I'm going well beyond what any packaging system is likely to be prepared to handle. Again, quel surprise. -- (reverse (concatenate 'string "gro.gultn@" "enworbbc")) http://www3.sympatico.ca/cbbrowne/linuxxian.html "Of _course_ it's the murder weapon. Who would frame someone with a fake?"
On Thu, 13 Feb 2003, scott.marlowe wrote: > If I do a .tar.gz install of apache, I get /usr/local/apache/conf, which > is not the standard way you're listing. I'm going to stay out of this argument from now on, but this struck a sore point. /usr is designed to be a filesystem that can be shared. Is the stuff in /usr/local/apache/conf really supposed to be shared amongst all machines of that architecture on your site that run apache? cjs -- Curt Sampson <cjs@cynic.net> +81 90 7737 2974 http://www.netbsd.org Don't you know, in this new Dark Age, we're alllight. --XTC
On Thursday 13 February 2003 21:13, Bruce Momjian wrote: > Lamar Owen wrote: > > It isn't without precedent to have a directory under /var/run. Maybe > > /var/run/postgresql. Under this one could have a uniquely named pid > > file. > But how do you handle the default then, where you have postmaster.pid in > /data? Do we rename it to postmaster.pid.5432 so it can sit in > /var/run/postgresql alone with other backends? Well, you can have the default as 'postmaster.pid' if it wasn't named. But more thought is needed. I'll have to admit; the wisdom of AOLserver having a full-fledged tcl config script is beginning to look better and better. > Another issue is that pg_ctl looks at that file, so moving it around is > going to be tricky. pg_ctl could be interesting. > I am now wondering if we even want pg_hba_dir and pg_ident_dir. Seems > we can assume they are in the same directory as postgresql.conf. That > leaves only data_dir as new for postgresql.conf. Ok, if we're going this far already, tell me exactly why we have three config files. Why not really Unify things and fulfil the full promise of Grand Unified Configuration by rolling hba and ident into postgresql.conf. Is there a compelling reason not to do so? The structure of that configuration data would have to change, for sure. Although I seem to remember this being suggested once before, but my mind draws a blank trying to recall it. Just a suggestion; maybe not even a good one, but something that crossed my mind. -- Lamar Owen WGCR Internet Radio 1 Peter 4:11
Oliver Elphick <olly@lfix.co.uk> writes:
> I'm not entirely sure why SE Linux has a problem, seeing that postgres
> needs read-write access to all the files in $PGDATA, but assuming the
> need is verified, I could do this by moving the pid file from
> $PGDATA/postmaster.pid to /var/run/postgresql/5432.pid and similarly for
> other ports. This would also have the benefit of being more FHS
> compliant What do people think about that?
No chance at all. Breaking the connection between the data directory
and the postmaster.pid file means we don't have an interlock against
starting two postmasters in the same data directory.
I do not see the argument for moving the pid file anyway. Surely no
one's going to tell us that the postmaster shouldn't have write access
to the data directory?
regards, tom lane
On Thursday 13 February 2003 21:49, Tom Lane wrote: > Oliver Elphick <olly@lfix.co.uk> writes: > > need is verified, I could do this by moving the pid file from > > $PGDATA/postmaster.pid to /var/run/postgresql/5432.pid and similarly for > > other ports. This would also have the benefit of being more FHS > > compliant What do people think about that? > No chance at all. Breaking the connection between the data directory > and the postmaster.pid file means we don't have an interlock against > starting two postmasters in the same data directory. It's not a pid file in the /var/run sense, really. It's an interlock for PGDATA. So it might be argued that postmaster.pid is misnamed. -- Lamar Owen WGCR Internet Radio 1 Peter 4:11
Lamar Owen wrote: > > I am now wondering if we even want pg_hba_dir and pg_ident_dir. Seems > > we can assume they are in the same directory as postgresql.conf. That > > leaves only data_dir as new for postgresql.conf. > > Ok, if we're going this far already, tell me exactly why we have three config > files. Why not really Unify things and fulfil the full promise of Grand > Unified Configuration by rolling hba and ident into postgresql.conf. Is > there a compelling reason not to do so? The structure of that configuration > data would have to change, for sure. Although I seem to remember this being > suggested once before, but my mind draws a blank trying to recall it. Just a > suggestion; maybe not even a good one, but something that crossed my mind. postgresql.conf is var=val, while the others are column-based. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001+ If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania19073
Lamar Owen <lamar.owen@wgcr.org> writes:
> Ok, if we're going this far already, tell me exactly why we have three config
> files. Why not really Unify things and fulfil the full promise of Grand
> Unified Configuration by rolling hba and ident into postgresql.conf. Is
> there a compelling reason not to do so?
Lack of backwards compatibility; unnecessary complexity. Unifying those
files would create a big headache in terms of having to unify their
syntax. And there are some basic semantic differences too. For
instance, order matters in pg_hba.conf, but not in postgresql.conf.
Another reason not to do it is that there are differences in the
security requirements of these files. postgresql.conf probably doesn't
contain anything that needs to be hidden from prying eyes, but I'd be
inclined to want to keep the other two mode 600.
---
Okay, I've been laying low all day, but here are my thoughts on the
discussion:
I do see the value in being able to (as opposed to being forced to,
please) keep hand-edited config files in a separate location from
the machine-processed data files. We have already gone some distance
in that direction over the past few releases --- there's much less in
the top $PGDATA directory than there once was. It makes sense to let
people keep hand-edited files away from what initdb will overwrite.
I would favor a setup that allows a -C *directory* (not file) to be
specified as a postmaster parameter separately from the -D directory;
then the hand-editable config files would be sought in -C not -D. In
the absence of -C the config files should be sought in -D, same as they
ever were (thus simplifying life for people like me who run many
postmasters and don't give a darn about FHS ;-)).
I don't see any great value in a separate postgresql.conf parameter for
each secondary config file; that just means clutter to me, especially
if we add more such files in future. I am also distinctly not in favor
of eliminating the PGDATA environment variable; that reads to me as
"we are going to force you to do it our way rather than the way you've
always done it, even if you like the old way".
To make the RPM packagers happy, I guess that the default -C directory
has to be settable via configure. We do not currently have a default
-D directory, and I didn't hear anyone arguing in favor of adding one.
So that leaves the following possible combinations that the postmaster
might see at startup, for which I propose the following behaviors:
1. No -C switch, no -D switch, no PGDATA found in environment: seek
postgresql.conf in the default -C directory established at configure
time. Use the 'datadir' specified therein as -D. Fail if postgresql.conf
doesn't define a datadir value.
2. No -C switch, no -D switch, PGDATA found in environment: use $PGDATA
as both -C and -D. (Minor detail: if the postgresql.conf in the $PGDATA
directory specifies a different directory as datadir, do we follow that
or raise an error? I'd be inclined to say "follow it" but maybe there
is an argument for erroring out.)
(In all the following cases, any environment PGDATA value is ignored.)
3. No -C switch, -D switch on command line: use -D value as both -C and -D,
proceed as in case 2.
4. -C switch, no -D switch on command line: seek postgresql.conf in
-C directory, use the datadir it specifies.
5. -C and -D on command line: seek postgresql.conf in -C directory,
use -D as datadir overriding what is in postgresql.conf (this is just
the usual rule that command line switches override postgresql.conf).
Cases 2 and 3 are backwards-compatible with our historical behavior,
so that anyone who likes the historical behavior will not be unhappy.
Cases 1 and 4 I think will make mlw and our packagers happy. Case 5
is just the logical conclusion for that combination.
In all cases, pg_hba.conf and pg_ident.conf would be sought in the
same directory as postgresql.conf. The other stuff in the toplevel
$PGDATA directory should stay where it is, IMHO.
I would venture that the configure-time-default for -C should be
${prefixdir}/etc if configure is not told differently, while the
packagers would probably set it to /etc/postgresql/ (ie, the
config files should live in a subdirectory that can be owned by
postgres user). I'm not wedded to that though.
Another interesting question is whether the installed-by-default
postgresql.conf should specify a datadir value, and if so what.
If initdb installs it, it can and probably should insert the actual
datadir location the user gave to initdb into the file. But should
initdb install any config files at all anymore? I'm leaning to the
thought that initdb should store default config files into $PGDATA
same as it ever did, and then it's up to the user (or package install
scripts) to move them to the desired -C directory if appropriate.
Or I suppose we could add a -C parameter to initdb to tell it where to
put 'em.
Comments?
regards, tom lane
Tom Lane wrote: > I don't see any great value in a separate postgresql.conf parameter for > each secondary config file; that just means clutter to me, especially > if we add more such files in future. I am also distinctly not in favor > of eliminating the PGDATA environment variable; that reads to me as > "we are going to force you to do it our way rather than the way you've > always done it, even if you like the old way". The scripts needn't ignore PGDATA at all. Only postmaster. Since the vast majority of people start the postmaster from a script, this winds up being a minor issue, except for the fact that without PGDATA administrators will be able to count on looking at the output of 'ps' to determine where the postmaster is looking for either the config file or the data directory. In other words, they'll have somewhere to start from without having to poke through scripts that might not even have been used (what happens when a user defines PGDATA and starts a postmaster? The administrator will have to go to more extreme lengths, like using lsof, to figure out where the data directory is. Not all systems have such tools). > Comments? I agree with your assessment for the most part, except for PGDATA. There's no good reason I can think of for the postmaster to look at it. It's fine if it sets it for processes it forks to inherit, but it shouldn't pay attention to it on startup. Some people might complain, but there's little difference in doing a "postmaster -D $PGDATA" and just "postmaster", and people who are starting things by hand hopefully aren't so inflexible as to demand that PGDATA remain treated as-is. People who really care can create a simple little 'pm.sh' script that simply does a "postmaster -D $PGDATA", which will save them typing even over just doing a "postmaster" from the command line. -- Kevin Brown kevin@sysexperts.com
On Thu, 2003-02-13 at 23:06, mlw wrote: > > Bruce Momjian wrote: > > Can non-root write to /var/run? > > > > > Shouldn't be able too But it should be able to write under /var/run/postgresql, which the distribution will set up with the correct permissions. -- Oliver Elphick Oliver.Elphick@lfix.co.uk Isle of Wight, UK http://www.lfix.co.uk/oliver GPG: 1024D/3E1D0C1C: CA12 09E0 E8D5 8870 5839 932A 614D 4C34 3E1D 0C1C ======================================== "God be merciful unto us, and bless us; and cause his face to shine uponus." Psalms 67:1
On Thu, 2003-02-13 at 22:53, Bruce Momjian wrote: > Oliver Elphick wrote: > > What your comments strongly suggest to me is that projects like > > PostgreSQL and pine, along with everything else, should comply with FHS; > > then there will be no confusion because everyone will be following the > > smae standards. Messes arise when people ignore standards; we have all > > seen the dreadful examples of MySQL and the Beast, haven't we? > > Can the FHS handle installing PostgreSQL as non-root? Certainly. It is only necessary to set permissions correctly in /etc/postgresql and /var/run/postgresql. -- Oliver Elphick Oliver.Elphick@lfix.co.uk Isle of Wight, UK http://www.lfix.co.uk/oliver GPG: 1024D/3E1D0C1C: CA12 09E0 E8D5 8870 5839 932A 614D 4C34 3E1D 0C1C ======================================== "God be merciful unto us, and bless us; and cause his face to shine uponus." Psalms 67:1
On Fri, 2003-02-14 at 02:49, Tom Lane wrote: > Oliver Elphick <olly@lfix.co.uk> writes: > > I'm not entirely sure why SE Linux has a problem, seeing that postgres > > needs read-write access to all the files in $PGDATA, but assuming the > > need is verified, I could do this by moving the pid file from > > $PGDATA/postmaster.pid to /var/run/postgresql/5432.pid and similarly for > > other ports. This would also have the benefit of being more FHS > > compliant What do people think about that? > > No chance at all. Breaking the connection between the data directory > and the postmaster.pid file means we don't have an interlock against > starting two postmasters in the same data directory. Yes; that would take a lot of effort to get round. Not worth it, I think. > I do not see the argument for moving the pid file anyway. Surely no > one's going to tell us that the postmaster shouldn't have write access > to the data directory? I'm waiting for a response on that one; I don't understand it either. -- Oliver Elphick Oliver.Elphick@lfix.co.uk Isle of Wight, UK http://www.lfix.co.uk/oliver GPG: 1024D/3E1D0C1C: CA12 09E0 E8D5 8870 5839 932A 614D 4C34 3E1D 0C1C ======================================== "God be merciful unto us, and bless us; and cause his face to shine uponus." Psalms 67:1
Kevin Brown <kevin@sysexperts.com> writes:
> I agree with your assessment for the most part, except for PGDATA.
> There's no good reason I can think of for the postmaster to look at
> it.
The other side of that coin is, what's the good reason to remove it?
There's a long way between "I don't want my setup to depend on PGDATA"
and "I don't think your setup should be allowed to depend on PGDATA".
If you don't want to use it, then don't use it. Why do you need to
tell me how I'm allowed to run my installation?
> ... people who are starting things by hand hopefully aren't so
> inflexible as to demand that PGDATA remain treated as-is.
Yes, I could reconfigure my scripts to not depend on this. You have
not given me an adequate argument why I should have to.
regards, tom lane
On Fri, 2003-02-14 at 02:45, cbbrowne@cbbrowne.com wrote: > 3.7.1 Purpose > /etc contains configuration files and directories that are specific to the > current system. > > 3.7.4 Indicates that > > "Host-specific configuration files for add-on application software packages > must be installed within the directory /etc/opt/<package>, where <package> is > the name of the subtree in /opt where the static data from that package is > stored." > > 3.12 indicates: /opt is reserved for the installation of add-on application > software packages. > > A package to be installed in /opt must locate its static files in a separate > /opt/<package> directory tree, where <package> is a name that describes the > software package. ... > It would make most sense, based on FHS, for PostgreSQL information to > assortedly reside in: > > - /etc/opt/postgresql or /etc/postgresql, for static config information; I feel that /opt (and therefore /etc/opt) are intended for the use of vendors; so commercial packages designed to fit in with FHS should use those. I don't think they are for locally built stuff. No matter; it illustrates the main point, which is that these things should be easily configurable. -- Oliver Elphick Oliver.Elphick@lfix.co.uk Isle of Wight, UK http://www.lfix.co.uk/oliver GPG: 1024D/3E1D0C1C: CA12 09E0 E8D5 8870 5839 932A 614D 4C34 3E1D 0C1C ======================================== "God be merciful unto us, and bless us; and cause his face to shine uponus." Psalms 67:1
Apache explicitly supports a number of different layouts for files out of the box (and provides support for you to roll your own very easily). From the manual: The second, and more flexible way to configure the install path locations for Apache is using the config.layout file. Using this method, it is possible to separately specify the location for each type of file within the Apache installation. The config.layout file contains several example configurations, and you can also create your own custom configuration following the examples. The different layouts in this file are grouped into <Layout FOO>...</Layout> sections and referred to by name as in FOO. --enable-layout=LAYOUT Use the named layout in the config.layoutfile to specify the installation paths. Maybe pg could benefit from something similar? cheers andrew ----- Original Message ----- From: "scott.marlowe" <scott.marlowe@ihs.com> Sent: Thursday, February 13, 2003 4:07 PM [snip] > If I do a .tar.gz install of apache, I get /usr/local/apache/conf, which > is not the standard way you're listing. If I install openldap from > .tar.gz, I get a /usr/local/etc/openldap directory, close, but still not > the same. The fact is, it's the packagers that put things into /etc and > whatnot, and I can see the postgresql RPMs or debs or whatever having that > as the default, but for custom built software, NOTHING that I know of > builds from source and uses /etc without a switch to tell it to, just like > postgresql can do now.
Tom Lane wrote: > Kevin Brown <kevin@sysexperts.com> writes: > > I agree with your assessment for the most part, except for PGDATA. > > There's no good reason I can think of for the postmaster to look at > > it. > > The other side of that coin is, what's the good reason to remove it? > There's a long way between "I don't want my setup to depend on PGDATA" > and "I don't think your setup should be allowed to depend on PGDATA". > If you don't want to use it, then don't use it. Why do you need to > tell me how I'm allowed to run my installation? I'm not talking about getting rid of ALL dependency on PGDATA in our entire distribution, only postmaster's. Recall that the main purpose of making any of these changes at all is to make life easier for the guys who have to manage the systems that will be running PostgreSQL. Agreed? So: imagine you're the newly-hired DBA and your boss points you to the system and says "administrate the database on that". You go over to the computer and start looking around. You do a "ps" and see a postmaster process running. You know that it's the process that is listening for connections. The "ps" listing only says "/usr/bin/postmaster". No arguments to clue you in, nothing. Where do you look to figure out where the data is? How do you figure out what port it's listening on? Well, we're already agreed on how to deal with that question: you look in /etc/postgresql, and because this is a relatively new install (and the PostgreSQL maintainers, who are very wise and benevolent, made that the default location for configs :-), it has a postgresql.conf file with a line that says "data_directory = /var/lib/pgsql". It doesn't mention a port to listen to so you know that it's listening on port 5432. As a DBA, you're all set. Now let's repeat that scenario, except that instead of seeing one postmaster process, you see five. And they all say "/usr/bin/postmaster" in the "ps" listing. No arguments to clue you in or anything, as before. You might be able to figure out where one of them is going by looking at /etc/postgresql, but what about the rest? Now you're stuck unless you want to do a "find" (time consuming and I/O intensive -- a good way to slow the production database down a bit), or you're knowledgeable enough to use 'lsof' or black magic like digging into kernel memory to figure out where the config files and data directories are, or you have enough knowledge to pore through the startup scripts and understand what they're doing. Lest you think that this is an unlikely scenario, keep in mind that most startup scripts, including pg_ctl, currently start the postmaster without arguments and rely on PGDATA, so a shop that hasn't already been bitten by this *will* be. Right now shops that wish to avoid the trap I described have to go to *extra* lengths: they have to make exactly the same kinds of changes to the scripts that I'm talking about us making (putting an explicit '-D "$PGDATA"' where none now exists) or they have to resort to tricks like renaming the postmaster executable and creating a shell script in its place that will invoke the (renamed) postmaster with '-D "$PGDATA"'. It's not so bad if only a few shops have to make those changes. But what if it's thousands? Yeah, the distribution guys can patch the scripts to do this, but why should they have to? They, and the shops that run PostgreSQL, are our customers. All of that is made possible because the postmaster can use an inherited PGDATA for the location of the config files and (if the config files don't say differently in our new scheme) the data directory, and pg_ctl takes advantage of that fact (as do most startup scripts that I've seen, that don't just invoke pg_ctl). I'm not arguing that we should remove the use of PGDATA *everywhere*, only in postmaster (and then, only postmaster's use of an *inherited* PGDATA. It should still set PGDATA so its children can use it). It means changing pg_ctl and the startup scripts we ship. The earlier we make these changes, the less overall pain there will be in the long run. > > ... people who are starting things by hand hopefully aren't so > > inflexible as to demand that PGDATA remain treated as-is. > > Yes, I could reconfigure my scripts to not depend on this. You have > not given me an adequate argument why I should have to. [By this I'm assuming you're referring to the scripts you use for testing, and not the ones that ship with the distribution] I'm not arguing that you should get rid of all the references to PGDATA in your scripts or anything crazy like that. The changes I'm talking about are minor: where you see "postmaster" without any "-D" arguments, you simply add '-D "$PGDATA"' to it, before any other arguments that you might also be passing. That's it. Nothing else should be needed. The reason for removing postmaster's use of an inherited PGDATA is the same as the reason for making the other changes we already agree should be made: to make things easier for the guys in the field who have to manage production systems. -- Kevin Brown kevin@sysexperts.com
Kevin Brown wrote: > Tom Lane wrote: > > Kevin Brown <kevin@sysexperts.com> writes: > > > I agree with your assessment for the most part, except for PGDATA. > > > There's no good reason I can think of for the postmaster to look at > > > it. > > > > The other side of that coin is, what's the good reason to remove it? > > There's a long way between "I don't want my setup to depend on PGDATA" > > and "I don't think your setup should be allowed to depend on PGDATA". > > If you don't want to use it, then don't use it. Why do you need to > > tell me how I'm allowed to run my installation? > > I'm not talking about getting rid of ALL dependency on PGDATA in our > entire distribution, only postmaster's. > > Recall that the main purpose of making any of these changes at all is > to make life easier for the guys who have to manage the systems that > will be running PostgreSQL. Agreed? > > So: imagine you're the newly-hired DBA and your boss points you to the > system and says "administrate the database on that". You go over to > the computer and start looking around. > > You do a "ps" and see a postmaster process running. You know that > it's the process that is listening for connections. The "ps" listing > only says "/usr/bin/postmaster". No arguments to clue you in, > nothing. Where do you look to figure out where the data is? How do > you figure out what port it's listening on? If you want ps to display the data dir, you should use -D. Remember, it is mostly important for multiple postmaster, so if you are doing that, just use -D, but don't prevent single-postmaster folks from using PGDATA. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001+ If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania19073
In reference to determining what port postgres or any program is listening on
On my Redhat Linux machines
netstat --inet -nlp
when run as root
produces a nice list of all programs listening on the network with IP and port
number the process is listening on, the name of the process and the pid.
The environment used to start each of these postmasters can be found at
cat /proc/${POSTMASTER-PID}/environ | tr "\000" "\n"
I'm not arguing one way or the other on your issue, just hope these tips make
the "black magic" a little easier to use.
On Friday 14 February 2003 04:58 am, Kevin Brown wrote:
> Now let's repeat that scenario, except that instead of seeing one
> postmaster process, you see five. And they all say
> "/usr/bin/postmaster" in the "ps" listing. No arguments to clue you
> in or anything, as before. You might be able to figure out where one
> of them is going by looking at /etc/postgresql, but what about the
> rest? Now you're stuck unless you want to do a "find" (time consuming
> and I/O intensive -- a good way to slow the production database down a
> bit), or you're knowledgeable enough to use 'lsof' or black magic like
> digging into kernel memory to figure out where the config files and
> data directories are, or you have enough knowledge to pore through the
> startup scripts and understand what they're doing.
OK, here is an updated proposal. I think we have decided:
Moving postmaster.pid and postmaster.opts isn't worth it.
We don't want per-file GUC variables, but assume it is inthe same config directory as postgresql.conf. I don'tsee any
validreason they would want to put them somewheredifferent than postgresql.conf.
So, we add data_dir to postgresql.conf, and add -C/PGCONFIG to
postmaster.
Regarding Tom's idea of replacing data_dir with a full path during
initdb, I think we are better having it be relative to the config
directory, that way if they move pgsql/, the system still works.
However, if the config directory is in a different lead directory path,
we should replace it with the full path, e.g. /usr/local/pgsql/data and
/usr/local/pgsql/etc use relative paths, ../data, while /etc/postgresql
and /usr/local/pgsql/data get an absolute path.
My idea is to introduce the above capabilities in 7.4, and keep the
config files in /data. This will allow package folks to move the config
files in 7.4.
I also think we should start telling people to use PGCONFIG rather than
PGDATA. Then, in 7.5, we move the default config file location to
pgsql/etc, and tell folks to point there rather than /data. I think
there is major value to getting those config files out of the initdb
creation area and for backups.
I am now wondering if we should add PGCONFIG and move them out of data
all in the same release. Not sure if delaying the split is valuable.
---------------------------------------------------------------------------
Tom Lane wrote:
> Lamar Owen <lamar.owen@wgcr.org> writes:
> > Ok, if we're going this far already, tell me exactly why we have three config
> > files. Why not really Unify things and fulfil the full promise of Grand
> > Unified Configuration by rolling hba and ident into postgresql.conf. Is
> > there a compelling reason not to do so?
>
> Lack of backwards compatibility; unnecessary complexity. Unifying those
> files would create a big headache in terms of having to unify their
> syntax. And there are some basic semantic differences too. For
> instance, order matters in pg_hba.conf, but not in postgresql.conf.
>
> Another reason not to do it is that there are differences in the
> security requirements of these files. postgresql.conf probably doesn't
> contain anything that needs to be hidden from prying eyes, but I'd be
> inclined to want to keep the other two mode 600.
>
> ---
>
> Okay, I've been laying low all day, but here are my thoughts on the
> discussion:
>
> I do see the value in being able to (as opposed to being forced to,
> please) keep hand-edited config files in a separate location from
> the machine-processed data files. We have already gone some distance
> in that direction over the past few releases --- there's much less in
> the top $PGDATA directory than there once was. It makes sense to let
> people keep hand-edited files away from what initdb will overwrite.
>
> I would favor a setup that allows a -C *directory* (not file) to be
> specified as a postmaster parameter separately from the -D directory;
> then the hand-editable config files would be sought in -C not -D. In
> the absence of -C the config files should be sought in -D, same as they
> ever were (thus simplifying life for people like me who run many
> postmasters and don't give a darn about FHS ;-)).
>
> I don't see any great value in a separate postgresql.conf parameter for
> each secondary config file; that just means clutter to me, especially
> if we add more such files in future. I am also distinctly not in favor
> of eliminating the PGDATA environment variable; that reads to me as
> "we are going to force you to do it our way rather than the way you've
> always done it, even if you like the old way".
>
> To make the RPM packagers happy, I guess that the default -C directory
> has to be settable via configure. We do not currently have a default
> -D directory, and I didn't hear anyone arguing in favor of adding one.
> So that leaves the following possible combinations that the postmaster
> might see at startup, for which I propose the following behaviors:
>
> 1. No -C switch, no -D switch, no PGDATA found in environment: seek
> postgresql.conf in the default -C directory established at configure
> time. Use the 'datadir' specified therein as -D. Fail if postgresql.conf
> doesn't define a datadir value.
>
> 2. No -C switch, no -D switch, PGDATA found in environment: use $PGDATA
> as both -C and -D. (Minor detail: if the postgresql.conf in the $PGDATA
> directory specifies a different directory as datadir, do we follow that
> or raise an error? I'd be inclined to say "follow it" but maybe there
> is an argument for erroring out.)
>
> (In all the following cases, any environment PGDATA value is ignored.)
>
> 3. No -C switch, -D switch on command line: use -D value as both -C and -D,
> proceed as in case 2.
>
> 4. -C switch, no -D switch on command line: seek postgresql.conf in
> -C directory, use the datadir it specifies.
>
> 5. -C and -D on command line: seek postgresql.conf in -C directory,
> use -D as datadir overriding what is in postgresql.conf (this is just
> the usual rule that command line switches override postgresql.conf).
>
> Cases 2 and 3 are backwards-compatible with our historical behavior,
> so that anyone who likes the historical behavior will not be unhappy.
> Cases 1 and 4 I think will make mlw and our packagers happy. Case 5
> is just the logical conclusion for that combination.
>
> In all cases, pg_hba.conf and pg_ident.conf would be sought in the
> same directory as postgresql.conf. The other stuff in the toplevel
> $PGDATA directory should stay where it is, IMHO.
>
> I would venture that the configure-time-default for -C should be
> ${prefixdir}/etc if configure is not told differently, while the
> packagers would probably set it to /etc/postgresql/ (ie, the
> config files should live in a subdirectory that can be owned by
> postgres user). I'm not wedded to that though.
>
> Another interesting question is whether the installed-by-default
> postgresql.conf should specify a datadir value, and if so what.
> If initdb installs it, it can and probably should insert the actual
> datadir location the user gave to initdb into the file. But should
> initdb install any config files at all anymore? I'm leaning to the
> thought that initdb should store default config files into $PGDATA
> same as it ever did, and then it's up to the user (or package install
> scripts) to move them to the desired -C directory if appropriate.
> Or I suppose we could add a -C parameter to initdb to tell it where to
> put 'em.
>
> Comments?
>
> regards, tom lane
>
> ---------------------------(end of broadcast)---------------------------
> TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org
>
-- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610)
359-1001+ If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square,
Pennsylvania19073
On Fri, 2003-02-14 at 12:17, Bruce Momjian wrote: > If you want ps to display the data dir, you should use -D. Remember, it > is mostly important for multiple postmaster, so if you are doing that, > just use -D, but don't prevent single-postmaster folks from using > PGDATA. Could not the ps line be rewritten to show this, as the backend's ps lines are rewritten? -- Oliver Elphick Oliver.Elphick@lfix.co.uk Isle of Wight, UK http://www.lfix.co.uk/oliver GPG: 1024D/3E1D0C1C: CA12 09E0 E8D5 8870 5839 932A 614D 4C34 3E1D 0C1C ======================================== "God be merciful unto us, and bless us; and cause his face to shine uponus." Psalms 67:1
I am not sure if it is a good idea to be mucking with it. For backend, we do the entire thing, so it is clear we modified something. --------------------------------------------------------------------------- Oliver Elphick wrote: > On Fri, 2003-02-14 at 12:17, Bruce Momjian wrote: > > If you want ps to display the data dir, you should use -D. Remember, it > > is mostly important for multiple postmaster, so if you are doing that, > > just use -D, but don't prevent single-postmaster folks from using > > PGDATA. > > Could not the ps line be rewritten to show this, as the backend's ps > lines are rewritten? > > -- > Oliver Elphick Oliver.Elphick@lfix.co.uk > Isle of Wight, UK http://www.lfix.co.uk/oliver > GPG: 1024D/3E1D0C1C: CA12 09E0 E8D5 8870 5839 932A 614D 4C34 3E1D 0C1C > ======================================== > "God be merciful unto us, and bless us; and cause his > face to shine upon us." Psalms 67:1 > > -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001+ If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania19073
Bruce Momjian wrote:
> If you want ps to display the data dir, you should use -D. Remember, it
> is mostly important for multiple postmaster, so if you are doing that,
> just use -D, but don't prevent single-postmaster folks from using
> PGDATA.
Perhaps the best compromise would be to change pg_ctl so that it uses
-D explicitly when invoking postmaster. That's an easy change.
Could you describe how you and other developers use PGDATA? I'm quite
interested in knowing why there seems to be so much resistance to
removing the "potential_DataDir = getenv("PGDATA");" line from
postmaster.c.
--
Kevin Brown kevin@sysexperts.com
Bruce Momjian wrote:
> If you want ps to display the data dir, you should use -D. Remember, it
> is mostly important for multiple postmaster, so if you are doing that,
> just use -D, but don't prevent single-postmaster folks from using
> PGDATA.
Perhaps another reasonable approach would be to put an #ifdef/#endif
around the "potential_DataDir = getenv("PGDATA");" line in postmater.c
and create a configure option to enable it. That way you guys get the
behavior you want for testing but production builds could disable it
if that's viewed as desirable. You'd want to make the error message
that's produced when no data directory is specified depend on the same
#ifdef variable, of course.
Then the group would get to fight it out over whether the configure
default should be "enable" or "disable". :-)
--
Kevin Brown kevin@sysexperts.com
Kevin Brown wrote:
> Bruce Momjian wrote:
> > If you want ps to display the data dir, you should use -D. Remember, it
> > is mostly important for multiple postmaster, so if you are doing that,
> > just use -D, but don't prevent single-postmaster folks from using
> > PGDATA.
>
> Perhaps the best compromise would be to change pg_ctl so that it uses
> -D explicitly when invoking postmaster. That's an easy change.
>
> Could you describe how you and other developers use PGDATA? I'm quite
> interested in knowing why there seems to be so much resistance to
> removing the "potential_DataDir = getenv("PGDATA");" line from
> postmaster.c.
I just set PGDATA in my login and I don't have to deal with it again.
-- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610)
359-1001+ If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square,
Pennsylvania19073
> > If you are interested in reading a contrary position, you can read > > Berstein's arguments for his recommended way to install services at: > > http://cr.yp.to/unix.html But since DJB is a class-A monomaniac, he may not be the best person to listen to. /var/qmail/control for qmail configuration files? Yeah, good one, DJB. -- Martin Coxall <coxall@cream.org>
> Generally things that live in /etc are owned and operated by the OS. > Postgresql, by it's definition is a userspace program, not an OS owned > one. Partially true. The FHS specifies that the /etc top layer is for system-own3d stuff, but the subdirectories off it are explicitly used for user space programs and, well, everything. (/etc/apache, /etc/postgres, /etc/tomcat3, /etc/tomcat4...) Martin Coxall
On Thu, 2003-02-13 at 20:28, Steve Crawford wrote: > I don't see why we can't keep everyone happy and let the users choose the > setup they want. To wit, make the following, probably simple, changes: > > 1) Have postgresql default to using /etc/postgresql.conf /etc/postgres/postgresql.conf, if we want to be proper FHS-bitches. > 2) Add a setting in postgresql.conf specifying the data directory > 3) Change the meaning of -D to mean "use this config file" > 4) In the absence of a specified data directory in postgresql.conf, use the > location of the postgresql.conf file as the data directory Shouldn't it in that case default to, say /var/lib/postgres? -- Martin Coxall <coxall@cream.org>
Bruce Momjian wrote: > I just set PGDATA in my login and I don't have to deal with it > again. Hmm...you don't use pg_ctl to start/stop/whatever the database? You invoke the postmaster directly (I can easily see that you would, just asking if you do)? -- Kevin Brown kevin@sysexperts.com
Kevin Brown wrote: > Bruce Momjian wrote: > > I just set PGDATA in my login and I don't have to deal with it > > again. > > Hmm...you don't use pg_ctl to start/stop/whatever the database? You > invoke the postmaster directly (I can easily see that you would, just > asking if you do)? I can use either to start/stop it. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001+ If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania19073
Bruce Momjian wrote: > I just set PGDATA in my login and I don't have to deal with it > again. Duh....I just realized a reason you guys might care about this so much. It's because you want to be able to start the postmaster from within a debugger (or profiler, or whatever), and you don't want to have to mess with command line options from there, right? Sounds like fixing pg_ctl to use -D explicitly when invoking the postmaster is the right change to make here, since that's probably how the majority of the production shops are going to be starting the database anyway. Takes care of the majority of the "visibility" problem and leaves PGDATA intact. Thoughts? -- Kevin Brown kevin@sysexperts.com
On 14 Feb 2003, Martin Coxall wrote: > > > > If you are interested in reading a contrary position, you can read > > > Berstein's arguments for his recommended way to install services at: > > > http://cr.yp.to/unix.html > > But since DJB is a class-A monomaniac, he may not be the best person to > listen to. /var/qmail/control for qmail configuration files? Yeah, good > one, DJB. I'm guessing that rather than reading it the above mentioned link you chose to waste our time with this instead. Good one, MC. Vince. -- Fast, inexpensive internet service 56k and beyond! http://www.pop4.net/ http://www.meanstreamradio.com http://www.unknown-artists.com Internet radio: It's not file sharing, it's just radio.
Give it up. As long as we have -D, we will allow PGDATA. If you don't want to use it, don't use it. --------------------------------------------------------------------------- Kevin Brown wrote: > Bruce Momjian wrote: > > I just set PGDATA in my login and I don't have to deal with it > > again. > > Duh....I just realized a reason you guys might care about this so > much. > > It's because you want to be able to start the postmaster from within a > debugger (or profiler, or whatever), and you don't want to have to > mess with command line options from there, right? > > > Sounds like fixing pg_ctl to use -D explicitly when invoking the > postmaster is the right change to make here, since that's probably how > the majority of the production shops are going to be starting the > database anyway. Takes care of the majority of the "visibility" > problem and leaves PGDATA intact. Thoughts? > > > > -- > Kevin Brown kevin@sysexperts.com > > ---------------------------(end of broadcast)--------------------------- > TIP 2: you can get off all lists at once with the unregister command > (send "unregister YourEmailAddressHere" to majordomo@postgresql.org) > -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001+ If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania19073
My point is that folks with multiple postmasters may not want to use PGDATA, but for folks who have single postmasters, it makes things easier and less error-prone. --------------------------------------------------------------------------- Bruce Momjian wrote: > > Give it up. As long as we have -D, we will allow PGDATA. If you don't > want to use it, don't use it. > > --------------------------------------------------------------------------- > > Kevin Brown wrote: > > Bruce Momjian wrote: > > > I just set PGDATA in my login and I don't have to deal with it > > > again. > > > > Duh....I just realized a reason you guys might care about this so > > much. > > > > It's because you want to be able to start the postmaster from within a > > debugger (or profiler, or whatever), and you don't want to have to > > mess with command line options from there, right? > > > > > > Sounds like fixing pg_ctl to use -D explicitly when invoking the > > postmaster is the right change to make here, since that's probably how > > the majority of the production shops are going to be starting the > > database anyway. Takes care of the majority of the "visibility" > > problem and leaves PGDATA intact. Thoughts? > > > > > > > > -- > > Kevin Brown kevin@sysexperts.com > > > > ---------------------------(end of broadcast)--------------------------- > > TIP 2: you can get off all lists at once with the unregister command > > (send "unregister YourEmailAddressHere" to majordomo@postgresql.org) > > > > -- > Bruce Momjian | http://candle.pha.pa.us > pgman@candle.pha.pa.us | (610) 359-1001 > + If your life is a hard drive, | 13 Roberts Road > + Christ can be your backup. | Newtown Square, Pennsylvania 19073 > > ---------------------------(end of broadcast)--------------------------- > TIP 5: Have you checked our extensive FAQ? > > http://www.postgresql.org/users-lounge/docs/faq.html > -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001+ If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania19073
On Fri, 2003-02-14 at 14:21, Vince Vielhaber wrote: > On 14 Feb 2003, Martin Coxall wrote: > > > > > > > If you are interested in reading a contrary position, you can read > > > > Berstein's arguments for his recommended way to install services at: > > > > http://cr.yp.to/unix.html > > > > But since DJB is a class-A monomaniac, he may not be the best person to > > listen to. /var/qmail/control for qmail configuration files? Yeah, good > > one, DJB. > > I'm guessing that rather than reading it the above mentioned link you > chose to waste our time with this instead. Good one, MC. Yeah, I've read it several times, and have often linked to it as an example of why one should be wary of DJB's software. It seems to me that since DJB doesn't follow his own advice regarding the filesystem hierarchy (see both qmail and djbdns), it'd be odd for him to expect anyone else to. *Especially* seing as he's a bit mental. ("I'm not going to take this any more. I demand cross-platform compatibility!") -- Martin Coxall <coxall@cream.org>
Bruce Momjian wrote: > OK, here is an updated proposal. I think we have decided: > > Moving postmaster.pid and postmaster.opts isn't worth it. > > We don't want per-file GUC variables, but assume it is in > the same config directory as postgresql.conf. I don't > see any valid reason they would want to put them somewhere > different than postgresql.conf. > > So, we add data_dir to postgresql.conf, and add -C/PGCONFIG to > postmaster. Agreed. One additional thing: when pg_ctl invokes the postmaster, it should explicitly specify -C on the postmaster command line, and if it doesn't find a data_dir in $PGCONFIG/postgresql.conf then it should explicitly specify a -D as well. Pg_ctl is going to have to be modified to take a -C argument anyway, so we may as well go all the way to do the right thing here. This way, people who start the database using the standard tools we supply will know exactly what's going on when they get a "ps" listing. -- Kevin Brown kevin@sysexperts.com
Bruce Momjian wrote: > > Give it up. As long as we have -D, we will allow PGDATA. If you don't > want to use it, don't use it. Agreed. I'm not sure I see how this diminishes the argument for fixing pg_ctl so that it passes an explicit -D option to the postmaster when invoking it... -- Kevin Brown kevin@sysexperts.com
Kevin Brown wrote: > Bruce Momjian wrote: > > OK, here is an updated proposal. I think we have decided: > > > > Moving postmaster.pid and postmaster.opts isn't worth it. > > > > We don't want per-file GUC variables, but assume it is in > > the same config directory as postgresql.conf. I don't > > see any valid reason they would want to put them somewhere > > different than postgresql.conf. > > > > So, we add data_dir to postgresql.conf, and add -C/PGCONFIG to > > postmaster. > > Agreed. One additional thing: when pg_ctl invokes the postmaster, it > should explicitly specify -C on the postmaster command line, and if it > doesn't find a data_dir in $PGCONFIG/postgresql.conf then it should > explicitly specify a -D as well. Pg_ctl is going to have to be > modified to take a -C argument anyway, so we may as well go all the > way to do the right thing here. > > This way, people who start the database using the standard tools we > supply will know exactly what's going on when they get a "ps" listing. No. If you want ps to display, don't use environment variables. Many don't care --- especially those with only one postmaster. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001+ If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania19073
Bruce Momjian wrote: > > This way, people who start the database using the standard tools we > > supply will know exactly what's going on when they get a "ps" listing. > > No. If you want ps to display, don't use environment variables. Many > don't care --- especially those with only one postmaster. You know that the code in pg_ctl doesn't send an explicit -D to the postmaster even if pg_ctl itself is invoked with a -D argument, right? The only way to make pg_ctl do that is by using the "-o" option. A typical vendor-supplied install is going to invoke pg_ctl to do the dirty work. That's why I'm focusing on pg_ctl. I completely understand your need for keeping PGDATA in postmaster. I don't understand why pg_ctl *shouldn't* be changed to invoke postmaster with an explicit -D option. It might be desirable for ps to not show any arguments to postmaster in some circumstances (I have no idea what those would be), but why in the world would you want that to be the *default*? Why would we want the default behavior to make things harder on administrators and not easier? -- Kevin Brown kevin@sysexperts.com
Kevin Brown <kevin@sysexperts.com> writes:
> Tom Lane wrote:
>> The other side of that coin is, what's the good reason to remove it?
>> There's a long way between "I don't want my setup to depend on PGDATA"
>> and "I don't think your setup should be allowed to depend on PGDATA".
>> If you don't want to use it, then don't use it. Why do you need to
>> tell me how I'm allowed to run my installation?
> I'm not talking about getting rid of ALL dependency on PGDATA in our
> entire distribution, only postmaster's.
We're obviously talking past each other. You are arguing that under
circumstances X, Y, or Z, depending on a PGDATA setting is a bad idea.
You are then drawing the conclusion that I should not be allowed to
depend on PGDATA, whether or not I care about X, Y, or Z.
I am happy to design an arrangement that allows you not to depend on
PGDATA if you don't want to. But I don't see why you need to break
my configuration procedures in order to fix yours. As I outlined last
night, it's possible to do what you want without breaking backwards
compatibility for those that like PGDATA.
regards, tom lane
On 14 Feb 2003, Martin Coxall wrote: > On Fri, 2003-02-14 at 14:21, Vince Vielhaber wrote: > > On 14 Feb 2003, Martin Coxall wrote: > > > > > > > > > > If you are interested in reading a contrary position, you can read > > > > > Berstein's arguments for his recommended way to install services at: > > > > > http://cr.yp.to/unix.html > > > > > > But since DJB is a class-A monomaniac, he may not be the best person to > > > listen to. /var/qmail/control for qmail configuration files? Yeah, good > > > one, DJB. > > > > I'm guessing that rather than reading it the above mentioned link you > > chose to waste our time with this instead. Good one, MC. > > Yeah, I've read it several times, and have often linked to it as an > example of why one should be wary of DJB's software. It seems to me that > since DJB doesn't follow his own advice regarding the filesystem > hierarchy (see both qmail and djbdns), it'd be odd for him to expect > anyone else to. *Especially* seing as he's a bit mental. ("I'm not going > to take this any more. I demand cross-platform compatibility!") I seriously doubt your ability to judge anyone's mental stability. I can also see that you prefer cross-platform INcompatibility. Your position and mindset are now crystal clear. Vince. -- Fast, inexpensive internet service 56k and beyond! http://www.pop4.net/ http://www.meanstreamradio.com http://www.unknown-artists.com Internet radio: It's not file sharing, it's just radio.
Tom Lane wrote: > I am happy to design an arrangement that allows you not to depend on > PGDATA if you don't want to. But I don't see why you need to break > my configuration procedures in order to fix yours. As I outlined last > night, it's possible to do what you want without breaking backwards > compatibility for those that like PGDATA. Yes, I agree. I hadn't really thought of all the possible benefits of PGDATA. Sorry. :-( Would you agree that it would be a beneficial change to have pg_ctl pass explicit arguments to postmaster? It would go a long way towards eliminating most of the situations I described. A warning in the documentation about the consequences of using PGDATA might not be a bad idea, either... -- Kevin Brown kevin@sysexperts.com
Bruce Momjian <pgman@candle.pha.pa.us> writes:
> So, we add data_dir to postgresql.conf, and add -C/PGCONFIG to
> postmaster.
Wait one second. You are blithely throwing in a PGCONFIG variable
without any detailed proposal of exactly how it will work. Does
that override a PGDATA environment variable? How do they interact?
Also, please note Kevin Brown's nearby arguments against using PGDATA
at all, which surely apply with equal force to a PGCONFIG variable.
Now, I don't buy that Kevin's arguments are enough reason to break
backwards compatibility by removing PGDATA --- but I think they are
enough reason not to introduce a new environment variable. PGCONFIG
wouldn't offer any backwards-compatibility value, and that tilts the
scales against it.
> Regarding Tom's idea of replacing data_dir with a full path during
> initdb, I think we are better having it be relative to the config
> directory, that way if they move pgsql/, the system still works.
Good thought, but you're assuming that initdb knows where the config
files will eventually live. If we do that, then moving the config
files breaks the installation. I think it will be fairly common to
let initdb drop its proposed config files into $PGDATA, and then
manually place them where they should go (or even more likely,
manually merge them with a prior version). Probably better to force
datadir to be an absolute path in the config file. (In fact, on safety
grounds I'd argue in favor of rejecting a datadir value taken from the
config file that wasn't absolute.)
> I also think we should start telling people to use PGCONFIG rather than
> PGDATA. Then, in 7.5, we move the default config file location to
> pgsql/etc, and tell folks to point there rather than /data.
I agree with none of this. This is not improvement, this is only change
for the sake of change. The packagers will do what they want to do
(and are already doing, mostly) regardless.
regards, tom lane
Oliver Elphick <olly@lfix.co.uk> writes:
> On Fri, 2003-02-14 at 12:17, Bruce Momjian wrote:
>> If you want ps to display the data dir, you should use -D. Remember, it
>> is mostly important for multiple postmaster, so if you are doing that,
>> just use -D, but don't prevent single-postmaster folks from using
>> PGDATA.
> Could not the ps line be rewritten to show this, as the backend's ps
> lines are rewritten?
I for one would rather it didn't do that. I already set my postmaster
command lines the way I want 'em, and I don't want the code overriding
that. (I prefer to use explicit -p arguments to distinguish the various
postmasters I have running --- shorter and easier to read than explicit
-D would be. At least for me.)
regards, tom lane
Kevin Brown <kevin@sysexperts.com> writes:
> I'm quite interested in knowing why there seems to be so much resistance to
> removing the "potential_DataDir = getenv("PGDATA");" line from
> postmaster.c.
Backwards compatibility. Also, you still haven't explained why
"I don't want to use PGDATA" should translate to "no one should
be allowed to use PGDATA". If you don't set PGDATA, what problem
is there for you in that line being there?
regards, tom lane
Martin Coxall <coxall@cream.org> writes:
> Partially true. The FHS specifies that the /etc top layer is for system-own3d
> stuff, but the subdirectories off it are explicitly used for user space programs
> and, well, everything. (/etc/apache, /etc/postgres, /etc/tomcat3,
> /etc/tomcat4...)
FHS or no FHS, I would think that the preferred arrangement would be to
keep Postgres' config files in a postgres-owned subdirectory, not
directly in /etc. That way you need not be root to edit them. (My idea
of an editor, Emacs, always wants to write a backup file, so I dislike
having to edit files that live in directories I can't write.)
Here's a pretty topic for a flamewar: should it be /etc/postgres/ or
/etc/postgresql/ ?
regards, tom lane
On Thu, 2003-02-13 at 19:22, Adam Haberlach wrote: > On Thu, Feb 13, 2003 at 05:59:17PM -0500, Robert Treat wrote: > > On Thu, 2003-02-13 at 15:08, mlw wrote: > > > Stephan Szabo wrote: > > > > > > On Thu, 13 Feb 2003, mlw wrote: > > Personally I think a postgresql installation is much more like an apache > > installation, which generally contains all of the files (data and > > config) under /usr/local/apache. Maybe someone can dig more to see if > > that system is more appropriate a comparison than something like bind. > > I think you are making a pretty uninformed, if not just plain wrong > generalization. I've run exactly one system with apache configuration > files in /usr/local/apache, and even then, the data was not there. Uh... the last time I built apache from source, it stuck everything under /usr/local/apache. It uses a conf directory for the config files, and htdocs for the "data" files... That is it's default configuration. <snip stories of all the different ways people run apache> You know, this is why I actually suggested looking closer at apache. By default, everything is crammed in one directory, but if you want to, you can configure it "six different ways to sunday". That seems to be a big plus IMO > > What does this mean? > > People will put things in different places, and there are typically > very good reasons for this. This is ESPECIALLY true when one wants to > have configuration files, at least the base ones in a common place such > as /etc or /usr/local/etc in order to make backup of configuration easy > and clean, while leaving data somewhere else for performance or magnitude > of partition reasons. It just makes sense to ME to have postgresql.conf > reside in /etc, yet put my data in /var/data/postgresql, yet retain the > option to put my data in /raid/data/postgresql at a later date, when the > new hardware comes in. Is anyone arguing against this? I'm certainly not. But maybe my needs are more varied than yours. On my local development box, I run multiple versions of apache, compiled with different versions of php. It really helps to keep all of apache's stuff centralized, and using things like rpms actually overly complicates this. Now sure, that's a development machine, but on the phppgadmin demo server, which is essentially a production system, I run three different versions of postgresql. In fact, I need to upgrade two of those (to 7.2.4 and 7.3.2), I shudder to think about doing that if postgresql forced me to use the /etc/ directory for all of my config files. Now sure, this probably isn't typical use, but I would say that when it comes time to upgrade major versions, unless you running an operation where you can have large amounts of downtime, postgresql needs to have the ability to have multiple versions install that don't conflict with each other, and it needs to do this easily. The upgrade process is hard enough already. <snip> > However, this seems, to me, to be a very small addition that has some real-world > (and yes, we need to start paying attention to the real world) advantages. > > And finally, don't go telling me that I'm wrong to put my data and config files > where I am. You can offer advice, but I'm probably going to ignore it because > I like where they are and don't need to explain why. > Have I wronged you in some former life? I've very little concern for where you put your data files, and have no idea why you'd think I'd criticize your setup. Robert Treat
Bruce Momjian <pgman@candle.pha.pa.us> writes:
> My point is that folks with multiple postmasters may not want to use
> PGDATA, but for folks who have single postmasters, it makes things
> easier and less error-prone.
Actually, for multi postmasters too. I have little shell-environment
config files that switch my entire world view between different source
trees and installation trees, for example this one sets me up to mess
with the 7.2 branch:
STDPATH=${STDPATH:-$PATH}
STDMANPATH=${STDMANPATH:-$MANPATH}
PGSRCROOT=$HOME/REL7_2/pgsql
PGINSTROOT=$HOME/version72
PATH=$PGINSTROOT/bin:/opt/perl5.6.1/bin:$STDPATH
MANPATH=$PGINSTROOT/man:$STDMANPATH
PGLIB=$PGINSTROOT/lib
PGDATA=$PGINSTROOT/data
PMOPTIONS="-p 5472 -i -F"
PMLOGFILE=server72.log
export PGSRCROOT PGINSTROOT PATH MANPATH PGLIB PGDATA STDPATH STDMANPATH
export PMOPTIONS PMLOGFILE
After sourcing one of these, I can use pg_ctl as well as a half dozen
other convenient little scripts that do things like remake and reinstall
the backend:
#!/bin/sh
pg_ctl -w stop
cd $PGSRCROOT/src/backend
make install-bin
startpg
or this one that fires up gdb on a crashed backend:
#!/bin/sh
# Usage: gdbcore
cd $HOME
CORES=`find $PGDATA/base -name core -type f -print`
if [ x"$CORES" != x"" ]
then ls -l $CORES
fi
if [ `echo "$CORES" | wc -w` -eq 1 ]
then exec gdb $PGINSTROOT/bin/postgres "$CORES"
else exec gdb $PGINSTROOT/bin/postgres
fi
This is vastly less error-prone than keeping track of the various
related elements in my head.
Now, it's certainly true that I could still make this work if I had
to explicitly say -D $PGDATA to the postmaster. But that would clutter
my ps display. I am happy with -p as the ps indicator of which
postmaster is which; I don't want more stuff in there.
regards, tom lane
On Friday 14 Feb 2003 9:05 pm, you wrote: > Martin Coxall <coxall@cream.org> writes: > Here's a pretty topic for a flamewar: should it be /etc/postgres/ or > /etc/postgresql/ ? I vote for /etc/pgsql. Keeping in line of unix philosophy of cryptic and short names. Who wants a descriptive names anyway..:-) Seriously, the traffic on last three days ahd very high noise ratio. Especially the whole discussion of PGDATA stuff fails to register as significant IMO. Right now, I can do things the way I want to do and I guess it is pretty much same with everyone else. Is it last topic left to improve? Keep it simple and on tpoic guys. This is hackers. Keep it low volume otherwise, two years down the lines, archives will be unsearchable.. Shridhar
On Thu, 13 Feb 2003, mlw wrote: > > > scott.marlowe wrote: > > >>These are not issues at all. You could put the configuration file > >>anywhere, just as you can for any UNIX service. > >> > >>postmaster --config=/home/myhome/mydb.conf > >> > >>I deal with a number of PG databases on a number of sites, and it is a > >>real pain in the ass to get to a PG box and hunt around for data > >>directory so as to be able to administer the system. What's really > >>annoying is when you have to find the data directory when someone else > >>set up the system. > >> > >> > > > >Really? I would think it's easier to do this: > > > >su - pgsuper > >cd $PGDATA > >pwd > > > >Than to try to figure out what someone entered when they ran ./configure > >--config=... > > > > > Why do you think PGDATA would be set for root? Did you not notice the "su - pgsuper" line above? You know, the one where you become the account that runs that instance of the database. Again, I ask you, isn't that easier than trying to find out what someone typed when they typed ./configure --config=? > >>Configuring postgresql via a configuration file which specifies all the > >>data, i.e. data directory, name of other configuration files, etc. is > >>the right way to do it. Even if you have reasons against it, even if you > >>think it is a bad idea, a bad standard is almost always a better > >>solution than an arcane work of perfection. > >> > >> > > > >Wrong, I strongly disagree with this sentament. Conformity to standards > >for simple conformity's sake is as wrong as sticking to the old way > >because it's what we're all comfy with. > > > It isn't conformity for conformitys sake. It is following an established > practice, like driving on the same side of the road or stopping at red > lights. But this isn't the same thing at all. Apache, when built from a tar ball, goes into /usr/local/apache/ and ALL it's configuration files are there. When installed as a package, my OS manufacturer decides where that goes. Those are the two "standard" ways of doing things. I like that postgresql installs into the /usr/local/pgsql directory from a tar ball. I like the fact that it uses $PGDATA to tell it where the cluster is, so that all my scripts, like pg_ctl, just know where it is without a -D switch each time.
On Fri, 2003-02-14 at 15:35, Tom Lane wrote: > Here's a pretty topic for a flamewar: should it be /etc/postgres/ or > /etc/postgresql/ ? It should be configurable! Debian uses /etc/postgresql, if you want to stick to what quite a lot of people are familiar with. -- Oliver Elphick Oliver.Elphick@lfix.co.uk Isle of Wight, UK http://www.lfix.co.uk/oliver GPG: 1024D/3E1D0C1C: CA12 09E0 E8D5 8870 5839 932A 614D 4C34 3E1D 0C1C ======================================== "God be merciful unto us, and bless us; and cause his face to shine uponus." Psalms 67:1
> I seriously doubt your ability to judge anyone's mental stability. > I can also see that you prefer cross-platform INcompatibility. Your > position and mindset are now crystal clear. Come now- don't take it personally. All I said is, as someone who wrestles daily with QMail, we should prefer the FHS over DJB's way of doing things and that DJB is a little, ahem, egocentric at times. Neither of these things was meant as a mortal insult to you personally, and if I offended you I apologise. Anyway, it looks like it's all been agreed over there, anyway. -- Martin Coxall <coxall@cream.org>
On Fri, 14 Feb 2003, Curt Sampson wrote: > On Thu, 13 Feb 2003, scott.marlowe wrote: > > > If I do a .tar.gz install of apache, I get /usr/local/apache/conf, which > > is not the standard way you're listing. > > I'm going to stay out of this argument from now on, but this struck a sore > point. > > /usr is designed to be a filesystem that can be shared. Is the stuff in > /usr/local/apache/conf really supposed to be shared amongst all machines > of that architecture on your site that run apache? Interesting. I've always viewed usr EXCEPT for local this way. In fact, on most of my boxes I create a seperate mount point for /usr/local so it's easier to backup and maintain, and it doesn't fill up the /usr directory. Asking for everything in a directory with the name local in it to be shared is kind of counter intuitive to me.
Tom Lane wrote: > Bruce Momjian <pgman@candle.pha.pa.us> writes: > > So, we add data_dir to postgresql.conf, and add -C/PGCONFIG to > > postmaster. > > Wait one second. You are blithely throwing in a PGCONFIG variable > without any detailed proposal of exactly how it will work. Does > that override a PGDATA environment variable? How do they interact? I am just throwing out ideas. I don't think we are near interaction issues yet. I think the big question is whether we want the default to install the configs in a separate directory, pgsql/etc, or just allow the specification of a separate location. Advantages of pgsql/etc are initdb-safe, and easier backups. I do think PGCONFIG would be helpful for the same reason that PGDATA is. However, there is clearly a problem of how does data_dir interact with PGDATA. The big question is whether PGDATA is still our driving config variable, and PGCONFIG/-C is just an additional option, or whether we are moving in a direction where PGCONFIG/-C is going to be the driving value, and data_dir is going to be read as part of that. > Also, please note Kevin Brown's nearby arguments against using PGDATA > at all, which surely apply with equal force to a PGCONFIG variable. > Now, I don't buy that Kevin's arguments are enough reason to break > backwards compatibility by removing PGDATA --- but I think they are > enough reason not to introduce a new environment variable. PGCONFIG > wouldn't offer any backwards-compatibility value, and that tilts the > scales against it. Weren't you just showing how you set environment variables to easily configure stuff. If you use a separate configure dir, isn't PGCONFIG part of that? > > Regarding Tom's idea of replacing data_dir with a full path during > > initdb, I think we are better having it be relative to the config > > directory, that way if they move pgsql/, the system still works. > > Good thought, but you're assuming that initdb knows where the config > files will eventually live. If we do that, then moving the config > files breaks the installation. I think it will be fairly common to > let initdb drop its proposed config files into $PGDATA, and then > manually place them where they should go (or even more likely, > manually merge them with a prior version). Probably better to force > datadir to be an absolute path in the config file. (In fact, on safety > grounds I'd argue in favor of rejecting a datadir value taken from the > config file that wasn't absolute.) Maybe. Not sure. > > I also think we should start telling people to use PGCONFIG rather than > > PGDATA. Then, in 7.5, we move the default config file location to > > pgsql/etc, and tell folks to point there rather than /data. > > I agree with none of this. This is not improvement, this is only change > for the sake of change. The packagers will do what they want to do > (and are already doing, mostly) regardless. Well, it is a step forward in terms of initdb-safe and easier backups. Several people said they liked that. I thought you were one of them. This is back to the big question, who drives things in the default install, config file or pgdata. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001+ If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania19073
Bruce Momjian wrote: > The big question is whether PGDATA is still our driving config variable, > and PGCONFIG/-C is just an additional option, or whether we are moving > in a direction where PGCONFIG/-C is going to be the driving value, and > data_dir is going to be read as part of that. I'm actually leaning towards PGCONFIG + PGDATA. Yeah, it may be a surprise given my previous arguments, but I can't help but think that the advantages you get with PGDATA will also exist for PGCONFIG. My previous arguments for removing PGDATA from postmaster can be dealt with by fixing pg_ctl to use explicit command line directives when invoking postmaster -- no changes to postmaster needed. PGCONFIG would be no different in that regard. Sorry if I seem a big gung-ho on the administrator point of view, but as a system administrator myself I understand and feel their pain. :-) -- Kevin Brown kevin@sysexperts.com
On Thu, Feb 13, 2003 at 11:53:26 -0500, mlw <pgsql@mohawksoft.com> wrote: > > Where, specificaly are his arguements against a configuration file > methodology? I don't think he is argueing against a configuration methodology, but rather against the methodology being used in Unix distributions. In particular he doesn't file the Linux File Standard because it puts the same software in different places depending on whether the vendor or using installed it.
"scott.marlowe" <scott.marlowe@ihs.com> writes:
> But this isn't the same thing at all. Apache, when built from a tar ball,
> goes into /usr/local/apache/ and ALL it's configuration files are there.
Two comments:
1) Even in that case the config files go into /usr/local/apache/conf and the other kinds of files like data logs and
cachefiles, all go in other subdirectories.
2) What you describe is only true if you configure with the default "--with-layout=Apache". The naming should perhaps
bea clue that this isn't a conventional layout. If you configure with --with-layout=GNU you get the conventional Unix
layoutin /usr/local, If you use --with-layout=RedHat you get the conventional layout in /usr directly which is mainly
usefulfor distribution packagers.
Putting stuff in a subdirectory like /usr/local/apache or /usr/local/pgsql is
unfortunately a widespread practice. It does have some advantages over the
conventional layout in /usr/local/{etc,bin,...} directly. But the major
disadvantage is that users can't run programs without adding dozens of entries
to their paths, can't compile programs without dozens of -L and -I lines, etc.
GNU autoconf script makes it pretty easy to configure packages to work either
though, and /usr/local is the purview of the local admin. As long as it's easy
to configure postgres to install "properly" with --prefix=/usr/local it won't
be any more of an offender than lots of other packages like apache, kde, etc.
Though I'll mention, please make it $prefix/etc not $prefix/conf. No need to
be gratuitously non-standard on an arbitrary name, and no need to pollute
/usr/local with multiple redundant directories.
--
greg
On Fri, Feb 14, 2003 at 02:58:49 -0800, Kevin Brown <kevin@sysexperts.com> wrote: > > Lest you think that this is an unlikely scenario, keep in mind that > most startup scripts, including pg_ctl, currently start the postmaster > without arguments and rely on PGDATA, so a shop that hasn't already > been bitten by this *will* be. Right now shops that wish to avoid the > trap I described have to go to *extra* lengths: they have to make > exactly the same kinds of changes to the scripts that I'm talking > about us making (putting an explicit '-D "$PGDATA"' where none now > exists) or they have to resort to tricks like renaming the postmaster > executable and creating a shell script in its place that will invoke > the (renamed) postmaster with '-D "$PGDATA"'. On at least some systems ps will dump process' environment and could be easily used to check PGDATA.
Kevin Brown wrote:
> Bruce Momjian wrote:
> > The big question is whether PGDATA is still our driving config variable,
> > and PGCONFIG/-C is just an additional option, or whether we are moving
> > in a direction where PGCONFIG/-C is going to be the driving value, and
> > data_dir is going to be read as part of that.
>
> I'm actually leaning towards PGCONFIG + PGDATA.
>
> Yeah, it may be a surprise given my previous arguments, but I can't
> help but think that the advantages you get with PGDATA will also exist
> for PGCONFIG.
>
> My previous arguments for removing PGDATA from postmaster can be dealt
> with by fixing pg_ctl to use explicit command line directives when
> invoking postmaster -- no changes to postmaster needed. PGCONFIG
> would be no different in that regard.
I see your point --- pg_ctl does a PGDATA trick when passed -D:
-D) shift # pass environment into new postmaster PGDATA="$1" export
PGDATA
It should pass -D just like it was given.
> Sorry if I seem a big gung-ho on the administrator point of view, but
> as a system administrator myself I understand and feel their pain.
Making things easy for sysadmins is an important feature of PostgreSQL.
-- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610)
359-1001+ If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square,
Pennsylvania19073
Bruce Momjian wrote: > I see your point --- pg_ctl does a PGDATA trick when passed -D: > > -D) > shift > # pass environment into new postmaster > PGDATA="$1" > export PGDATA > > It should pass -D just like it was given. Yes, exactly. Now, the more interesting question in my mind is: if pg_ctl isn't passed -D but inherits PGDATA, should it nonetheless pass -D explicitly to the postmaster? We can make it do that, and it would have the benefit of making transparent what would otherwise be opaque. I'm inclined to answer "yes" to that question, but only because someone who *really* doesn't want the postmaster to show up with a -D argument in "ps" can start the postmaster directly without using pg_ctl at all. Tom made a good argument for sometimes wanting to keep the ps output clean, but it's not clear to me that it should necessarily apply to pg_ctl. But you guys might have a different perspective on that. :-) -- Kevin Brown kevin@sysexperts.com
On Friday 14 February 2003 6:07 am, Martin Coxall wrote: > On Thu, 2003-02-13 at 20:28, Steve Crawford wrote: > > I don't see why we can't keep everyone happy and let the users choose the > > setup they want. To wit, make the following, probably simple, changes: > > > > 1) Have postgresql default to using /etc/postgresql.conf > > /etc/postgres/postgresql.conf, if we want to be proper FHS-bitches. > > > 2) Add a setting in postgresql.conf specifying the data directory > > 3) Change the meaning of -D to mean "use this config file" > > 4) In the absence of a specified data directory in postgresql.conf, use > > the location of the postgresql.conf file as the data directory > > Shouldn't it in that case default to, say /var/lib/postgres? Idea 4 was just a way to preserve current behaviour for those who desire. Moving postgresql.conf requires adding the data directory info into postgresql.conf or specifying it in some other way. If, in the absence of any specification in postgresql.conf, postgres just looks in the same directory as postgresql.conf then it will be almost identical to the current setup. Cheers, Steve
Tom Lane writes:
> I would favor a setup that allows a -C *directory* (not file) to be
> specified as a postmaster parameter separately from the -D directory;
A directory is not going to satisfy people.
> I don't see any great value in a separate postgresql.conf parameter for
> each secondary config file; that just means clutter to me,
Not to other people.
> 1. No -C switch, no -D switch, no PGDATA found in environment: seek
> postgresql.conf in the default -C directory established at configure
> time. Use the 'datadir' specified therein as -D. Fail if postgresql.conf
> doesn't define a datadir value.
OK.
> 2. No -C switch, no -D switch, PGDATA found in environment: use $PGDATA
> as both -C and -D.
This behavior would be pretty inconsistent. But maybe it's the best we
can do.
> 3. No -C switch, -D switch on command line: use -D value as both -C and -D,
> proceed as in case 2.
Same as above.
> 4. -C switch, no -D switch on command line: seek postgresql.conf in
> -C directory, use the datadir it specifies.
OK.
> 5. -C and -D on command line: seek postgresql.conf in -C directory,
> use -D as datadir overriding what is in postgresql.conf (this is just
> the usual rule that command line switches override postgresql.conf).
But that usual rule seems to be in conflict with cases 2 and 3 above.
(The usual rule is that a command-line option overrides a postgresql.conf
parameter. The rule in 3, for example is, that a command-line option (the
same one!) overrides where postgresql.conf is in the first place.)
> I would venture that the configure-time-default for -C should be
> ${prefixdir}/etc if configure is not told differently,
Yeah, we already have that as --sysconfdir.
--
Peter Eisentraut peter_e@gmx.net
Bruce Momjian <pgman@candle.pha.pa.us> writes:
> I think the big question is whether we want the default to install the
> configs in a separate directory, pgsql/etc, or just allow the
> specification of a separate location. Advantages of pgsql/etc are
> initdb-safe, and easier backups.
I don't see why we don't just let initdb install suggested config files
into the new $PGDATA directory, same as it ever did. Then (as long as
we don't use relative paths in the config files) people can move them
somewhere else if they like, or not if they prefer not to. Adding more
mechanism than that just adds complexity without buying much (except the
possibility of initdb overwriting your old config files, which is
exactly what I thought we wanted to avoid).
> The big question is whether PGDATA is still our driving config variable,
> and PGCONFIG/-C is just an additional option, or whether we are moving
> in a direction where PGCONFIG/-C is going to be the driving value, and
> data_dir is going to be read as part of that.
I thought the idea was to allow both approaches. We are not moving in
the direction of one or the other, we are giving people a choice of how
they want to drive it.
> Weren't you just showing how you set environment variables to easily
> configure stuff. If you use a separate configure dir, isn't PGCONFIG
> part of that?
I'm just pointing out that there's no backward-compatibility argument
for PGCONFIG. It should only be put in if the people who want to use
the -C-is-driver approach want it. Kevin clearly doesn't ;-).
regards, tom lane
Peter Eisentraut wrote: > Tom Lane writes: > > > I would favor a setup that allows a -C *directory* (not file) to be > > specified as a postmaster parameter separately from the -D directory; > > A directory is not going to satisfy people. Who is asking to put postgresql.conf, pg_hba.conf, and pg_ident.conf in different directories? I haven't heard anyone ask for that. > > I don't see any great value in a separate postgresql.conf parameter for > > each secondary config file; that just means clutter to me, > > Not to other people. > > > 1. No -C switch, no -D switch, no PGDATA found in environment: seek > > postgresql.conf in the default -C directory established at configure > > time. Use the 'datadir' specified therein as -D. Fail if postgresql.conf > > doesn't define a datadir value. > > OK. > > > 2. No -C switch, no -D switch, PGDATA found in environment: use $PGDATA > > as both -C and -D. > > This behavior would be pretty inconsistent. But maybe it's the best we > can do. What happens if postgresql.conf then defines data_dir? Seems we ignore it. This brings up the same issue of whether -C/PGCONFIG is a inferior option to -D/PGDATA, and whether we keep the config files in /data by default. > > 3. No -C switch, -D switch on command line: use -D value as both -C and -D, > > proceed as in case 2. > > Same as above. > > > 4. -C switch, no -D switch on command line: seek postgresql.conf in > > -C directory, use the datadir it specifies. > > OK. Here we are saying the -C doesn't override postgresql.conf as the proper PGDATA value. Is that what we want? We had the question above over how a data_dir in postgresql.conf is handled. > > > 5. -C and -D on command line: seek postgresql.conf in -C directory, > > use -D as datadir overriding what is in postgresql.conf (this is just > > the usual rule that command line switches override postgresql.conf). > > But that usual rule seems to be in conflict with cases 2 and 3 above. > (The usual rule is that a command-line option overrides a postgresql.conf > parameter. The rule in 3, for example is, that a command-line option (the > same one!) overrides where postgresql.conf is in the first place.) Yes, the big question seems to be if we are defaulting -C to be the same as -D, whether that is an actual specification of -D that should override postgresql.conf. This is part of the reason I don't like the -D assumes -C and stuff like that. I think we need to move the config files to pgsql/etc, for backup and initdb safety, and move toward having PGCONFIG/-C as the driving parameter. I think having both function equally and defaulting if the other is not specified is going to breed confusion. I am willing to make thing a little difficult for backward compatibility to do this, and I think because it is only administrators, they will welcome the improvement. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001+ If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania19073
Peter Eisentraut <peter_e@gmx.net> writes:
> Tom Lane writes:
>> I would favor a setup that allows a -C *directory* (not file) to be
>> specified as a postmaster parameter separately from the -D directory;
> A directory is not going to satisfy people.
Why not? Who won't it satisfy, and what's their objection?
AFAICS, you can either set -C to /etc if you want your PG config files
loose in /etc, or you can set it to /etc/postgresql/ if you want them
in a privately-owned directory. Which other arrangements are needed?
regards, tom lane
Tom Lane wrote: > Bruce Momjian <pgman@candle.pha.pa.us> writes: > > I think the big question is whether we want the default to install the > > configs in a separate directory, pgsql/etc, or just allow the > > specification of a separate location. Advantages of pgsql/etc are > > initdb-safe, and easier backups. > > I don't see why we don't just let initdb install suggested config files > into the new $PGDATA directory, same as it ever did. Then (as long as > we don't use relative paths in the config files) people can move them > somewhere else if they like, or not if they prefer not to. Adding more > mechanism than that just adds complexity without buying much (except the > possibility of initdb overwriting your old config files, which is > exactly what I thought we wanted to avoid). > > > The big question is whether PGDATA is still our driving config variable, > > and PGCONFIG/-C is just an additional option, or whether we are moving > > in a direction where PGCONFIG/-C is going to be the driving value, and > > data_dir is going to be read as part of that. > > I thought the idea was to allow both approaches. We are not moving in > the direction of one or the other, we are giving people a choice of how > they want to drive it. That's where I am unsure. Is the initdb-safe and backup advantages enough to start to migrate those out to data/? I need to hear comments on that. One new idea is to move the config files into data/etc. That makes it clear which are config files, and makes backup a little easier. It would make -C more logical because you are not moving a clear directory. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001+ If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania19073
Kevin Brown wrote: > Bruce Momjian wrote: > > The big question is whether PGDATA is still our driving config variable, > > and PGCONFIG/-C is just an additional option, or whether we are moving > > in a direction where PGCONFIG/-C is going to be the driving value, and > > data_dir is going to be read as part of that. > > I'm actually leaning towards PGCONFIG + PGDATA. > > Yeah, it may be a surprise given my previous arguments, but I can't > help but think that the advantages you get with PGDATA will also exist > for PGCONFIG. > > My previous arguments for removing PGDATA from postmaster can be dealt > with by fixing pg_ctl to use explicit command line directives when > invoking postmaster -- no changes to postmaster needed. PGCONFIG > would be no different in that regard. The following patch propogates pg_ctl -D to the postmaster as a -D flag. I see no other pg_ctl flags that make sense to propogate. Applied. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania 19073 Index: src/bin/pg_ctl/pg_ctl.sh =================================================================== RCS file: /cvsroot/pgsql-server/src/bin/pg_ctl/pg_ctl.sh,v retrieving revision 1.30 diff -c -c -r1.30 pg_ctl.sh *** src/bin/pg_ctl/pg_ctl.sh 18 Oct 2002 22:05:35 -0000 1.30 --- src/bin/pg_ctl/pg_ctl.sh 14 Feb 2003 22:04:56 -0000 *************** *** 115,120 **** --- 115,122 ---- logfile= silence_echo= shutdown_mode=smart + PGDATAOPTS="" + POSTOPTS="" while [ "$#" -gt 0 ] do *************** *** 129,135 **** ;; -D) shift ! # pass environment into new postmaster PGDATA="$1" export PGDATA ;; --- 131,138 ---- ;; -D) shift ! # we need to do this so -D datadir shows in ps display ! PGDATAOPTS="-D $1" PGDATA="$1" export PGDATA ;; *************** *** 333,344 **** fi if [ -n "$logfile" ]; then ! "$po_path" ${1+"$@"} </dev/null >>$logfile 2>&1 & else # when starting without log file, redirect stderr to stdout, so # pg_ctl can be invoked with >$logfile and still have pg_ctl's # stderr on the terminal. ! "$po_path" ${1+"$@"} </dev/null 2>&1 & fi # if had an old lockfile, check to see if we were able to start --- 336,347 ---- fi if [ -n "$logfile" ]; then ! "$po_path" ${1+"$@"} ${PGDATAOPTS+$PGDATAOPTS} </dev/null >>$logfile 2>&1 & else # when starting without log file, redirect stderr to stdout, so # pg_ctl can be invoked with >$logfile and still have pg_ctl's # stderr on the terminal. ! "$po_path" ${1+"$@"} ${PGDATAOPTS+$PGDATAOPTS} </dev/null 2>&1 & fi # if had an old lockfile, check to see if we were able to start
I don't want to over-engineer this. Propogating -D into postmaster makes sense, but grabbing PGDATA doesn't to me. --------------------------------------------------------------------------- Kevin Brown wrote: > Bruce Momjian wrote: > > I see your point --- pg_ctl does a PGDATA trick when passed -D: > > > > -D) > > shift > > # pass environment into new postmaster > > PGDATA="$1" > > export PGDATA > > > > It should pass -D just like it was given. > > Yes, exactly. > > Now, the more interesting question in my mind is: if pg_ctl isn't > passed -D but inherits PGDATA, should it nonetheless pass -D > explicitly to the postmaster? We can make it do that, and it would > have the benefit of making transparent what would otherwise be opaque. > > I'm inclined to answer "yes" to that question, but only because > someone who *really* doesn't want the postmaster to show up with a -D > argument in "ps" can start the postmaster directly without using > pg_ctl at all. Tom made a good argument for sometimes wanting to keep > the ps output clean, but it's not clear to me that it should > necessarily apply to pg_ctl. > > But you guys might have a different perspective on that. :-) > > > > -- > Kevin Brown kevin@sysexperts.com > > ---------------------------(end of broadcast)--------------------------- > TIP 4: Don't 'kill -9' the postmaster > -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001+ If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania19073
On Fri, 14 Feb 2003, scott.marlowe wrote: > Asking for everything in a directory with the name local in it to be > shared is kind of counter intuitive to me. Not really. If you install a particular program that doesn't come with the OS on one machine on your site, why would you not want to install it separately on all of the others? Typically, I want my favourite non-OS utilities on all machines, not just one. (Even if I don't use them on all machines.) Thus /usr/local is for site-local stuff. cjs -- Curt Sampson <cjs@cynic.net> +81 90 7737 2974 http://www.netbsd.org Don't you know, in this new Dark Age, we're alllight. --XTC
Robert Treat wrote: >Seems like some are saying one of the problems with the current system >is it doesn't follow FHS or LSB. If those are valid reasons to change >the system, it seems like a change which doesn't actually address those >concerns would not be acceptable. (Unless those really aren't valid >concerns...) > > > I did not start this thread to make PostgreSQL FHS compatible, someone else brought that up. As I said somewhere else, I'm an old fashioned UNIX guy, capability without policy. The patch that I submitted for 7.3.2 will allow the user to configure PostgreSQL with a configuration file outside the $PGDATA directory. That's all I care about. If someone wants to get on the FHS bandwagon, that's fine. PostgreSQL should allow that ability but should not require it.
Martin Coxall wrote: >On Thu, 2003-02-13 at 20:28, Steve Crawford wrote: > > >>I don't see why we can't keep everyone happy and let the users choose the >>setup they want. To wit, make the following, probably simple, changes: >> >>1) Have postgresql default to using /etc/postgresql.conf >> >> > >/etc/postgres/postgresql.conf, if we want to be proper FHS-bitches. > > > >>2) Add a setting in postgresql.conf specifying the data directory >>3) Change the meaning of -D to mean "use this config file" >>4) In the absence of a specified data directory in postgresql.conf, use the >>location of the postgresql.conf file as the data directory >> >> > >Shouldn't it in that case default to, say /var/lib/postgres? > I would really like to push back this whole discussion to adding the ability the flexibility to configure PostgreSQ as opposed to determining a specific configuration strategy. Adding the ability is easy. Let the distros determine their strategy. Trying to enforce one way over another will make this continue on forever and will never be solved. > >
mlw <pgsql@mohawksoft.com> writes:
> I would really like to push back this whole discussion to adding the
> ability the flexibility to configure PostgreSQ as opposed to determining
> a specific configuration strategy.
> Adding the ability is easy. Let the distros determine their strategy.
> Trying to enforce one way over another will make this continue on
> forever and will never be solved.
I agree that we shouldn't be in the business of dictating choices.
But it is important to examine what the plausible choices are, so that
we can be sure the solution we provide will accommodate all of them.
So I don't think this part of the thread has been useless.
regards, tom lane
On Friday 14 February 2003 15:10, Tom Lane wrote: > I don't see why we don't just let initdb install suggested config files > into the new $PGDATA directory, same as it ever did. Ok, let me take another tack. Just exactly why does initdb need to drop any config files anywhere? We provide templates; initdb can initialize the data structure. If we can by default (as part of make install) put the config file templates in $SYSCONFDIR (as set by ./configure), then why does initdb need to retouch them? I say that having configured PostgreSQL like this: (this is for 7.2.4, not 7.3.x) --enable-locale --with-CXX --prefix=/usr --disable-rpath --with-perl --enable-multibyte --with-tcl --with-odbc --enable-syslog --with-python --with-openssl --with-pam --with-krb5=/usr/kerberos --enable-nls --sysconfdir=/etc/pgsql --mandir=/usr/share/man --docdir=/usr/share/doc --includedir=/usr/include --datadir=/usr/share/pgsql So, in my case, it would be preferable to me for initdb to not make a default postgresql.conf, pg_hba.conf, and pg_ident.conf. The make install process should populate sysconfdir (/etc/pgsql here) with those files. Why does initdb even need to be involved now (I know the historical reason)? Comments? -- Lamar Owen WGCR Internet Radio 1 Peter 4:11
On Friday 14 February 2003 11:02, "Shridhar Daithankar wrote: > Especially the whole discussion of PGDATA stuff fails to register as > significant IMO. Right now, I can do things the way I want to do and I > guess it is pretty much same with everyone else. Is it last topic left to > improve? If it weren't significant to a few, then there wouldn't be the traffic. If there's too much traffic, well, there are alternatives. > Keep it simple and on tpoic guys. This is hackers. Keep it low volume > otherwise, two years down the lines, archives will be unsearchable.. The system configuration of PostgreSQL is on topic for -hackers. IMNSHO. -- Lamar Owen WGCR Internet Radio 1 Peter 4:11
Lamar Owen <lamar.owen@wgcr.org> writes:
> Just exactly why does initdb need to drop any config files anywhere?
Because we'd like it to edit the correct datadir into the config file,
to take just the most obvious example. There has also been a great deal
of discussion recently about other things initdb might automatically put
into the config file after looking at the system environment. That's
not happened yet, but we'd really be restricting ourselves to say that
initdb can never customize the config files.
regards, tom lane
On Saturday 15 February 2003 20:19, Tom Lane wrote: > Lamar Owen <lamar.owen@wgcr.org> writes: > > Just exactly why does initdb need to drop any config files anywhere? > Because we'd like it to edit the correct datadir into the config file, > to take just the most obvious example. Shouldn't we be consistent and have initdb use the datadir set in the config file, which could be supplied by a ./configure switch? > There has also been a great deal > of discussion recently about other things initdb might automatically put > into the config file after looking at the system environment. That's > not happened yet, but we'd really be restricting ourselves to say that > initdb can never customize the config files. Customize != writing the original. I'm looking at a packager point of view here. The initdb is done well after the package is made, and installed. It would be ideal from this point of view to have things fully configured pre-initdb (and thus pre-packaging). But I understand that this might not be ideal from a multipostmaster point of view. Surely these two can be reconciled. -- Lamar Owen WGCR Internet Radio 1 Peter 4:11
Lamar Owen <lamar.owen@wgcr.org> writes:
> On Saturday 15 February 2003 20:19, Tom Lane wrote:
>> Lamar Owen <lamar.owen@wgcr.org> writes:
> Just exactly why does initdb need to drop any config files anywhere?
>> Because we'd like it to edit the correct datadir into the config file,
>> to take just the most obvious example.
> Shouldn't we be consistent and have initdb use the datadir set in the config
> file, which could be supplied by a ./configure switch?
That'd mean there is no way to perform an initdb into a nonstandard
location without first hand-preparing a config file. I don't much care
for that.
> I'm looking at a packager point of view here. The initdb is done well after
> the package is made, and installed. It would be ideal from this point of
> view to have things fully configured pre-initdb (and thus pre-packaging).
This point of view means that no post-configure knowledge can be
applied. We might as well forget the separate initdb step altogether
and have "make install" do it.
I realize that from a packager's point of view, the separate initdb step
is not very useful. But it is from my point of view.
regards, tom lane
On Saturday 15 February 2003 09:48 am, mlw wrote: > Robert Treat wrote: > >Seems like some are saying one of the problems with the current system > >is it doesn't follow FHS or LSB. If those are valid reasons to change > >the system, it seems like a change which doesn't actually address those > >concerns would not be acceptable. (Unless those really aren't valid > >concerns...) > > I did not start this thread to make PostgreSQL FHS compatible, someone > else brought that up. > > As I said somewhere else, I'm an old fashioned UNIX guy, capability > without policy. The patch that I submitted for 7.3.2 will allow the user > to configure PostgreSQL with a configuration file outside the $PGDATA > directory. That's all I care about. If someone wants to get on the FHS > bandwagon, that's fine. PostgreSQL should allow that ability but should > not require it. If we're going to go through the trouble to change the way things work, we might as well try to get something that will allow instalation to match real desired configurations out there, like FHS and LSB, or how Oliver and Lamar want for packaging without symlinks. If the goal is just to get something that you like, apply the patch locally and be done with it. Robert Treat
On Saturday 15 February 2003 21:06, Tom Lane wrote: > Lamar Owen <lamar.owen@wgcr.org> writes: > > Shouldn't we be consistent and have initdb use the datadir set in the > > config file, which could be supplied by a ./configure switch? > That'd mean there is no way to perform an initdb into a nonstandard > location without first hand-preparing a config file. I don't much care > for that. Six of one and half-dozen of another. And that's my real point. We do things quite differently from many other standard services, even those which are built from the ground up for multiple instances. Making things more consistent for admins, even if it's not what we are used to or what we might like (because it's familiar) should at least be thought about. I'm not advocating changing just for the sake of change; but getting a new fresh look at our current setup can't hurt. > > I'm looking at a packager point of view here. The initdb is done well > > after the package is made, and installed. It would be ideal from this > > point of view to have things fully configured pre-initdb (and thus > > pre-packaging). > This point of view means that no post-configure knowledge can be > applied. We might as well forget the separate initdb step altogether > and have "make install" do it. I wouldn't complain. Although that isn't conducive to the multiple instance case. The necessary post-configure knowledge would be in the instance postgresql.conf file. One place for it. But this is hypothetical; fishing around the waters here at this point. Realize that my own packages apply an initdb automatically if an install isn't found the first time the system initscript is started. It is virtually automatic. With the multiple postmaster support, creating a couple of files and a symlink (for the initscript), and starting the new initscript symlink does all the dirty work. But it could be easier. > I realize that from a packager's point of view, the separate initdb step > is not very useful. But it is from my point of view. Would you mind elucidating which point of view is yours? General idea of who else might have the same point of view, and why you find the initdb in its current form to be more useful than alternatives. Again, I'm fishing for knowledge -- if nothing else it gives me an answer to those users who send me nastygrams about the way things are right now. -- Lamar Owen WGCR Internet Radio 1 Peter 4:11
Lamar Owen <lamar.owen@wgcr.org> writes:
>> I realize that from a packager's point of view, the separate initdb step
>> is not very useful. But it is from my point of view.
> Would you mind elucidating which point of view is yours?
Primarily, one that wants to have multiple postmasters running, of the
same or different versions; including test and temporary installations
that mustn't conflict with the existing primary installation on a machine.
Currently, I don't need to do anything more than set PGDATA or say -D
to initdb in order to set up the data directory wherever I like. I also
don't need to worry about whether I'm selecting the wrong config file.
You're talking about making manual installations significantly more
difficult (and error-prone, I think) in order to simplify automated
installs. Now you've acknowledged that your script can do what
you want it to, and in fact already does. Why is it good to make my
life more difficult to make a script's easier? Cycles are cheap.
I like to think that my time is worth something.
Nor will I buy an argument that only a few developers have need for test
installations. Ordinary users will want to do that anytime they are
doing preliminary tests on a new PG version before migrating their
production database to it. To the extent that you make manual selection
of a nonstandard datadir location more difficult and error-prone, you
are hurting them too.
regards, tom lane
On Sunday 16 February 2003 00:16, Tom Lane wrote: > Lamar Owen <lamar.owen@wgcr.org> writes: > > Would you mind elucidating which point of view is yours? > Primarily, one that wants to have multiple postmasters running, of the > same or different versions; including test and temporary installations > that mustn't conflict with the existing primary installation on a machine. Well, due to our upgrading difficulty, having multiple versions running has its advantages. > You're talking about making manual installations significantly more > difficult (and error-prone, I think) in order to simplify automated > installs. Now you've acknowledged that your script can do what > you want it to, and in fact already does. Why is it good to make my > life more difficult to make a script's easier? Cycles are cheap. > I like to think that my time is worth something. The script's been out there for awhile. It does some things well, and some things not so well. The config files are still coresident with the database, and backup is more difficult than it can be. Meeting all these needs (with configure switches, configuration file directives, etc) would be a good thing. And that's what I'm after; maximum usability for the maximum audience. I believe pretty strongly that the usage to which you or I would put PostgreSQL is probably quite different from the average user's way of using PostgreSQL. Most probably the typical user has a single installation with multiple databases with little need to run isolated postmasters. > Nor will I buy an argument that only a few developers have need for test > installations. Ordinary users will want to do that anytime they are > doing preliminary tests on a new PG version before migrating their > production database to it. To the extent that you make manual selection > of a nonstandard datadir location more difficult and error-prone, you > are hurting them too. While I'm not going to speak for all users, I know that I don't put a development database system on my production servers. The production machine only runs production servers, period. Hardware is cheap. I have development machines for development databases. One also has the error-prone PATH issues with multiple versions, which, if you are running a typical RPM installation becomes quite difficult to manage, since the RPM version's executables are in /usr/bin. This could be changed; I've thought about changing it. But I'm not sure of the best way to make multiple versions peacefully and seamlessly coexist -- particularly when older versions may not even build on the newer OS: but we've been over that discussion. Care for a poll? -- Lamar Owen WGCR Internet Radio 1 Peter 4:11
Tom Lane wrote: > Currently, I don't need to do anything more than set PGDATA or say -D > to initdb in order to set up the data directory wherever I like. I also > don't need to worry about whether I'm selecting the wrong config file. So in your case, what's the advantage of having initdb write anything to a config file, when you're probably also relying on PGDATA or -D to start the database (if you're not, then fair enough. But see below)? I'd expect initdb to initialize a database. If I were running initdb without a lot of foreknowledge of its side effects, I think I'd probably be a bit surprised to find that it had touched my config file. Initdb doesn't have prior knowledge of how you intend to start the database so that it refers to the database initdb just created, so it can't really know whether it's desirable to touch the config file. If it's desirable for initdb to be able to write to the config file, wouldn't it be more appropriate for that to an option that has to be explicitly enabled on initdb's command line? I don't know how often you'd want it to write into the config file for your purposes, but having it do so automatically seems to violate the principle of least surprise. -- Kevin Brown kevin@sysexperts.com
Tom Lane writes: > AFAICS, you can either set -C to /etc if you want your PG config files > loose in /etc, or you can set it to /etc/postgresql/ if you want them > in a privately-owned directory. Which other arrangements are needed? People might want to share them between servers, or allow a user to select from a few pre-configured ones that which reside in the same directory. -- Peter Eisentraut peter_e@gmx.net
Peter Eisentraut <peter_e@gmx.net> writes:
> Tom Lane writes:
>> AFAICS, you can either set -C to /etc if you want your PG config files
>> loose in /etc, or you can set it to /etc/postgresql/ if you want them
>> in a privately-owned directory. Which other arrangements are needed?
> People might want to share them between servers, or allow a user to select
> from a few pre-configured ones that which reside in the same directory.
You can accomplish that without the need to customize the .conf file
names; you just make, eg,
/etc/postgres/myconfig/postgresql.conf/etc/postgres/yourconfig/postgresql.conf/etc/postgres/herconfig/postgresql.conf
(plus additional config files as needed in each of these directories)
and then the postmaster start command is
postmaster -C /etc/postgres/myconfig
I see no real gain in flexibility in allowing people to choose random
names for the individual config files. Also, it'd defeat the
ultimate-fallback approach of doing "find / -name postgresql.conf"
to figure out where the config files are hiding in an unfamiliar
installation.
regards, tom lane
Lamar Owen <lamar.owen@wgcr.org> writes:
> The script's been out there for awhile. It does some things well, and some
> things not so well. The config files are still coresident with the database,
> and backup is more difficult than it can be. Meeting all these needs (with
> configure switches, configuration file directives, etc) would be a good
> thing.
Sure. I'm happy to change the software in a way that *allows* moving the
config files elsewhere. But it's not apparent to me why you insist on
forcing people who are perfectly happy with their existing configuration
arrangements to change them. I have not seen any reason in this
discussion why we can't support both a separate-config-location approach
and the traditional single-location one.
Please remember that the existing approach has been evolved over quite
a few releases. It may not satisfy the dictates of the FHS religion,
but it does meet some people's needs perfectly well. Let's look for a
solution that permits coexistence, rather than one that forces change
on people who don't need or want change.
regards, tom lane
Kevin Brown <kevin@sysexperts.com> writes:
> So in your case, what's the advantage of having initdb write anything
> to a config file, when you're probably also relying on PGDATA or -D to
> start the database (if you're not, then fair enough. But see below)?
Keep in mind that initdb doesn't currently *need* to put the datadir
location into the config file. It *will* need to do so if we separate
config and data dirs. Or at least, *somebody* will need to do so.
It's not apparent to me how it simplifies life not to have initdb do it.
Especially when there are other configuration items that initdb should
or already does record: locale settings, database encoding. And we
have already been talking about improving PG's self-tuning capability.
initdb would be the natural place to look around for information like
available RAM and adjust the config-file settings like sort_mem
accordingly.
Basically, the notion that initdb shouldn't write a config file seems
like a complete dead end to me. It cannot possibly be more convenient
than the alternatives. We'd be giving up a lot of current and future
functionality --- and for what?
> I'd expect initdb to initialize a database. If I were running initdb
> without a lot of foreknowledge of its side effects, I think I'd
> probably be a bit surprised to find that it had touched my config
> file.
If we do it the way I suggested (dump into the datadir, which is
initially empty, same as always) then it cannot overwrite your existing
config files. Think of it as providing a suggested config file to
compare against what you have.
regards, tom lane
On Sunday 16 February 2003 13:15, Tom Lane wrote: > Sure. I'm happy to change the software in a way that *allows* moving the > config files elsewhere. So we agree. Perfect. > But it's not apparent to me why you insist on > forcing people who are perfectly happy with their existing configuration > arrangements to change them. Me? Trying to force things to change? You misunderstand me. No, I'm trying to understand the rationale for a (relative to the way other designed-multiple daemons do things) different, non-standard configuration process. I understand better now; the exercise was a success. Many thanks. -- Lamar Owen WGCR Internet Radio 1 Peter 4:11
Tom Lane wrote: >Peter Eisentraut <peter_e@gmx.net> writes: > > >>Tom Lane writes: >> >> >>>I would favor a setup that allows a -C *directory* (not file) to be >>>specified as a postmaster parameter separately from the -D directory; >>> >>> > > > >>A directory is not going to satisfy people. >> >> > >Why not? Who won't it satisfy, and what's their objection? > >AFAICS, you can either set -C to /etc if you want your PG config files >loose in /etc, or you can set it to /etc/postgresql/ if you want them >in a privately-owned directory. Which other arrangements are needed? > > > The idea of using a "directory" puts us back to using symlinks to share files. While I know the core development teams thinks that symlinks are a viable configuration option, most admins, myself included, do not like to use symlinks because they do not have the ability to carry documentation, i.e. comments in a configuration file, and are DANGEROUS in a production environment. Any configuration strategy that depends on symlinks is inadequate and poorly designed. > >
mlw <pgsql@mohawksoft.com> writes:
> The idea of using a "directory" puts us back to using symlinks to share
> files.
So? If you want to share files, you're probably sharing all three
config files and don't need a separate directory at all. This is
not a sufficient argument to make me buy into the mess of letting
people choose nonstandard configuration file names --- especially
when most of the opposite camp seems to be more interested in choosing
*standard* names for things. Why does that policy stop short at the
directory name?
regards, tom lane
Tom Lane wrote: > Keep in mind that initdb doesn't currently *need* to put the datadir > location into the config file. It *will* need to do so if we separate > config and data dirs. Or at least, *somebody* will need to do so. > It's not apparent to me how it simplifies life not to have initdb do it. > Especially when there are other configuration items that initdb should > or already does record: locale settings, database encoding. Is it possible for the database engine to properly deal with a database when it is told to use a different database encoding than the one the database was initdb'd with? If it's not, then that suggests to me that the database encoding is something that doesn't belong in the configuration file but rather in some other place that is intimately tied with the database itself and which is difficult/impossible to change, like perhaps a read-only system table that gets created at initdb time. > And we have already been talking about improving PG's self-tuning > capability. initdb would be the natural place to look around for > information like available RAM and adjust the config-file settings > like sort_mem accordingly. I agree here, and since you're thinking of just putting the resulting config file in the database data directory, then as a DBA I wouldn't be terribly surprised by it ... especially if it came back with a message that told me what it had done. > If we do it the way I suggested (dump into the datadir, which is > initially empty, same as always) then it cannot overwrite your existing > config files. Think of it as providing a suggested config file to > compare against what you have. There is one minor complication: what if there's an existing config file in the target directory? One use for initdb would be as a quick way to completely wipe the database and start over (e.g., if the encoding were found to be incorrect), but the config file that's already there could easily contain a lot of customization that the administrator would want to retain. Which suggests that we should consider writing to a file using a slightly different name (e.g., postgresql.conf.initdb), at least in the event that a config file already exists in the target directory. Not sure what the overall right thing to do here is... -- Kevin Brown kevin@sysexperts.com
Tom Lane wrote: >mlw <pgsql@mohawksoft.com> writes: > > >>The idea of using a "directory" puts us back to using symlinks to share >>files. >> >> > >So? If you want to share files, you're probably sharing all three >config files and don't need a separate directory at all. This is >not a sufficient argument to make me buy into the mess of letting >people choose nonstandard configuration file names --- especially >when most of the opposite camp seems to be more interested in choosing >*standard* names for things. Why does that policy stop short at the >directory name? > > symlinks suck. Sorry Tom, but they are *BAD* in a production server. You can not add comments to symlinks. Most of the admins I know, myself included, HATE symlinks and use them as a last resort. Requiring symlinks is just pointless, we are talking about a few lines of code hat has nothing to do with performance. The patch that I submitted allows PostgreSQL to work as it always has, but adds the ability for a configuration file to do what is normally done with fixed names in $PGDATA. I have said before, I do not like policy, I like flexibility, forcing a directory is similarly restricting as requiring the files in $PGDATA. Why is this such a problem? MANY people want to configure PostgreSQL this way, but the patch I submitted allows it, but does not force anything. Any configuration solution that requires symlinks is flawed. > >
mlw wrote: > symlinks suck. Sorry Tom, but they are *BAD* in a production > server. Well, at least they're better than hard links. ;-) -- Kevin Brown kevin@sysexperts.com
Kevin Brown <kevin@sysexperts.com> writes:
> Is it possible for the database engine to properly deal with a
> database when it is told to use a different database encoding than the
> one the database was initdb'd with?
It can't be "told to use a different database encoding". However, the
default *client* encoding matches the database encoding, and that is
something that can be set in the config file.
>> If we do it the way I suggested (dump into the datadir, which is
>> initially empty, same as always) then it cannot overwrite your existing
>> config files. Think of it as providing a suggested config file to
>> compare against what you have.
> There is one minor complication: what if there's an existing config
> file in the target directory?
If there's anything at all in the target directory, initdb refuses to
run.
regards, tom lane
On Fri, Feb 14, 2003 at 10:35:41AM -0500, Tom Lane wrote: > FHS or no FHS, I would think that the preferred arrangement would be to > keep Postgres' config files in a postgres-owned subdirectory, not > directly in /etc. That way you need not be root to edit them. (My idea Besides, what are you going to do for people installing on a box where they don't have root? Are they going to need a whole mess of extra directories in their private copy? > of an editor, Emacs, always wants to write a backup file, so I dislike > having to edit files that live in directories I can't write.) > > Here's a pretty topic for a flamewar: should it be /etc/postgres/ or > /etc/postgresql/ ? Wow, two flamewar topics in one mail. I'm impressed. Andrew "ed is the one true editor" Sullivan -- ---- Andrew Sullivan 204-4141 Yonge Street Liberty RMS Toronto, Ontario Canada <andrew@libertyrms.info> M2P 2A8 +1 416 646 3304 x110
On Sun, Feb 16, 2003 at 12:16:44AM -0500, Tom Lane wrote: > Nor will I buy an argument that only a few developers have need for test > installations. Ordinary users will want to do that anytime they are > doing preliminary tests on a new PG version before migrating their > production database to it. To the extent that you make manual selection > of a nonstandard datadir location more difficult and error-prone, you > are hurting them too. Not only that. For safety's sake, you may need to run multiple postmasters on one machine (so that database user X can't DoS database user Y, for instance). And making that sort of production-grade work more difficult and error-prone would also be bad. A -- ---- Andrew Sullivan 204-4141 Yonge Street Liberty RMS Toronto, Ontario Canada <andrew@libertyrms.info> M2P 2A8 +1 416 646 3304 x110
On Sat, 15 Feb 2003, Curt Sampson wrote: > On Fri, 14 Feb 2003, scott.marlowe wrote: > > > Asking for everything in a directory with the name local in it to be > > shared is kind of counter intuitive to me. > > Not really. If you install a particular program that doesn't come with > the OS on one machine on your site, why would you not want to install it > separately on all of the others? > > Typically, I want my favourite non-OS utilities on all machines, not > just one. (Even if I don't use them on all machines.) Thus /usr/local is > for site-local stuff. Good point. Of course, in apache, it's quite easy to use the -f switch to pick the file you're running on. so, with a httpd -f /usr/local/apache/conf/`uname -a|cut -d " " -f 2`.conf I can pick and choose the file to run. So, yes, I would gladly use it in a cluster, and all the files would be in one place, easy to backup.
I have a new idea. You know how we have search_path where you can specify multiple schema names. What if we allow the config_dirs/-C to specify multiple directories to search for config files. That way, we can use only one variable, and we can allow people to place different config files in different directories. --------------------------------------------------------------------------- Andrew Sullivan wrote: > On Sun, Feb 16, 2003 at 12:16:44AM -0500, Tom Lane wrote: > > Nor will I buy an argument that only a few developers have need for test > > installations. Ordinary users will want to do that anytime they are > > doing preliminary tests on a new PG version before migrating their > > production database to it. To the extent that you make manual selection > > of a nonstandard data_dir location more difficult and error-prone, you > > are hurting them too. > > Not only that. For safety's sake, you may need to run multiple > postmasters on one machine (so that database user X can't DoS > database user Y, for instance). And making that sort of > production-grade work more difficult and error-prone would also be > bad. > > A > > -- > ---- > Andrew Sullivan 204-4141 Yonge Street > Liberty RMS Toronto, Ontario Canada > <andrew@libertyrms.info> M2P 2A8 > +1 416 646 3304 x110 > > > ---------------------------(end of broadcast)--------------------------- > TIP 2: you can get off all lists at once with the unregister command > (send "unregister YourEmailAddressHere" to majordomo@postgresql.org) > -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001+ If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania19073
Bruce Momjian wrote: > > I have a new idea. You know how we have search_path where you can > specify multiple schema names. What if we allow the config_dirs/-C to > specify multiple directories to search for config files. That way, we > can use only one variable, and we can allow people to place different > config files in different directories. That's an interesting idea. Were you thinking, perhaps, that you could put, say, a postgresql.conf file in multiple directories and have the settings in the latest one override the settings in earlier ones? That would mean you could set up a single postgresql.conf that has settings common to all of your instances (settings related to the system such as shared buffers, and default settings that would apply to any instance if not overridden), and a postgresql.conf file for each instance that defines the instance-specific configuration information. I'm not sure that's quite what you had in mind, though. :-) -- Kevin Brown kevin@sysexperts.com
Bruce Momjian <pgman@candle.pha.pa.us> writes:
> I have a new idea. You know how we have search_path where you can
> specify multiple schema names. What if we allow the config_dirs/-C to
> specify multiple directories to search for config files. That way, we
> can use only one variable, and we can allow people to place different
> config files in different directories.
Hm, a search path for config files? I could support that if it
satisfies the folks who object to specifying config directories
rather than file names.
One thing that would have to be thought about is whether to re-search
the path on each config file reload --- if you first find pg_hba.conf
in, say, the third directory on the path, should you pay attention if
one materializes in the second directory later? Or do you keep going
back to the same well? I can see arguments either way.
regards, tom lane
The problem I have with Bruce's scheme is that you could put your config file where you want it and someone else puts one somewhere higher in the search path and you have no idea what went wrong. It sounds to me like a recipe for an SA's nightmare. Other people have claimed to speak from the SA perspective - do they see this too? andrew ----- Original Message ----- From: "Tom Lane" <tgl@sss.pgh.pa.us> > Bruce Momjian <pgman@candle.pha.pa.us> writes: > > I have a new idea. You know how we have search_path where you can > > specify multiple schema names. What if we allow the config_dirs/-C to > > specify multiple directories to search for config files. That way, we > > can use only one variable, and we can allow people to place different > > config files in different directories. > > Hm, a search path for config files? I could support that if it > satisfies the folks who object to specifying config directories > rather than file names. > > One thing that would have to be thought about is whether to re-search > the path on each config file reload --- if you first find pg_hba.conf > in, say, the third directory on the path, should you pay attention if > one materializes in the second directory later? Or do you keep going > back to the same well? I can see arguments either way.
Tom Lane wrote: > Bruce Momjian <pgman@candle.pha.pa.us> writes: > > I have a new idea. You know how we have search_path where you can > > specify multiple schema names. What if we allow the config_dirs/-C to > > specify multiple directories to search for config files. That way, we > > can use only one variable, and we can allow people to place different > > config files in different directories. > > Hm, a search path for config files? I could support that if it > satisfies the folks who object to specifying config directories > rather than file names. > > One thing that would have to be thought about is whether to re-search > the path on each config file reload --- if you first find pg_hba.conf > in, say, the third directory on the path, should you pay attention if > one materializes in the second directory later? Or do you keep going > back to the same well? I can see arguments either way. Oh, I hadn't thought of that. I would vote for researching the path, but I am not sure why. The bigger question is whether you can modify config_dirs while the postmaster is running. I would think not. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001+ If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania19073
Bruce Momjian <pgman@candle.pha.pa.us> writes:
> The bigger question is whether you can modify config_dirs while the
> postmaster is running. I would think not.
There would be no way to do that, because the only way to set it would
be from -C on the command line or a PGCONFIG environment variable.
But I can't see a real good reason why you'd need to.
regards, tom lane
"Andrew Dunstan" <andrew@dunslane.net> writes:
> The problem I have with Bruce's scheme is that you could put your config
> file where you want it and someone else puts one somewhere higher in the
> search path and you have no idea what went wrong. It sounds to me like a
> recipe for an SA's nightmare. Other people have claimed to speak from the SA
> perspective - do they see this too?
If you have "your" file you'd put it in the directory at the front of
the search path. End of problem. Any additional directories would be
for config files that you *want* to share.
Offhand I find it hard to visualize needing more than two directories in
this path (private and shared), unless we grow to having many more
config files than we do now. But we may as well build the feature with
no artificial restriction about path length.
Search path management seems well understood for $PATH --- people do
get burnt by having the wrong $PATH, but it doesn't qualify as a
nightmare...
regards, tom lane