Thread: Server tries to read a different config file than it is supposed to
I thought I understood how specifying a config file path for the server works, but that's apparently not the case.
The cluster data is at /storage/postgresql/9.4/data.
The config files are at /etc/postgresql/9.4/main (this is the default location on Ubuntu).
This is how the beginning of /etc/postgresql/9.4/main/postgresql.conf looks like: data_directory = '/storage/postgresql/9.4/data'
hba_file = '/etc/postgresql/9.4/main/pg_hba.conf'
ident_file = '/etc/postgresql/9.4/main/pg_ident.conf'
So I wrote a few scripts to make my life easier, e.g. pg94start.sh:su postgres -c "/usr/lib/postgresql/9.4/bin/pg_ctl -D /storage/postgresql/9.4/data -o '-c config_file=/etc/postgresql/9.4/main/postgresql.conf'"
But running this script did not work, the server would not start. So I checked the log file and there was:FATAL: could not open file "/storage/postgresql/9.4/data/postgresql.conf": Permission denied
After fixing the ownership of this file, it worked.
What's the reason the server was trying to access that file? Why does not the override given by the config_file parameter work? Thank you.
View this message in context: Server tries to read a different config file than it is supposed to
Sent from the PostgreSQL - general mailing list archive at Nabble.com.
On Sat, 2015-05-23 at 04:23 -0700, twoflower wrote: > I thought I understood how specifying a config file path for the > server works, but that's apparently not the case. > > The cluster data is at /storage/postgresql/9.4/data. > > The config files are at /etc/postgresql/9.4/main (this is the default > location on Ubuntu). > > This is how the beginning of /etc/postgresql/9.4/main/postgresql.conf > looks like: > > data_directory = '/storage/postgresql/9.4/data' > hba_file = '/etc/postgresql/9.4/main/pg_hba.conf' > ident_file = '/etc/postgresql/9.4/main/pg_ident.conf' > > So I wrote a few scripts to make my life easier, e.g. pg94start.sh: > > su postgres -c "/usr/lib/postgresql/9.4/bin/pg_ctl > -D /storage/postgresql/9.4/data -o '-c > config_file=/etc/postgresql/9.4/main/postgresql.conf'" > > But running this script did not work, the server would not start. So I > checked the log file and there was: > > FATAL: could not open file > "/storage/postgresql/9.4/data/postgresql.conf": Permission denied > > After fixing the ownership of this file, it worked. > > What's the reason the server was trying to access that file? Why does > not the override given by the config_file parameter work? > > Thank you. > > ______________________________________________________________________ > View this message in context: Server tries to read a different config > file than it is supposed to > Sent from the PostgreSQL - general mailing list archive at Nabble.com. Similar hassles with Debian. I use scripts like this to start the server. #! /bin/sh PGDATA=/home/postgres/data94;export PGDATA PATH=$PATH:/usr/lib/postgresql/9.4/bin;export PATH PGBINARIES=/usr/lib/postgresql/9.4/bin;export PGBINARIES cd /usr/lib/postgresql/9.4/bin ./pg_ctl start exit 0 I have different scripts depending on the version of Postgres that I need. I know that I could have a generic script and just pass in the variables. I run them as the postgres user. It reads the conf file from the PGDATA path. HTH. Robert
On 05/23/2015 04:23 AM, twoflower wrote: > I thought I understood how specifying a config file path for the server > works, but that's apparently not the case. > > The cluster data is at */storage/postgresql/9.4/data*. > > The config files are at */etc/postgresql/9.4/main* (this is the default > location on Ubuntu). > > This is how the beginning of */etc/postgresql/9.4/main/postgresql.conf* > looks like: > > |data_directory = '/storage/postgresql/9.4/data' > hba_file = '/etc/postgresql/9.4/main/pg_hba.conf' > ident_file = '/etc/postgresql/9.4/main/pg_ident.conf' | > > So I wrote a few scripts to make my life easier, e.g. *pg94start.sh*: > > |su postgres -c "/usr/lib/postgresql/9.4/bin/pg_ctl -D > /storage/postgresql/9.4/data -o '-c > config_file=/etc/postgresql/9.4/main/postgresql.conf'"| > > But running this script did not work, the server would not start. So I > checked the log file and there was: > > *FATAL: could not open file > "/storage/postgresql/9.4/data/postgresql.conf": Permission denied* > > After fixing the ownership of this file, it worked. > > What's the reason the server was trying to access that file? Why does > not the override given by the *|config_file|* parameter work? I think you are seeing the effects of the below: http://www.postgresql.org/docs/9.4/static/runtime-config-file-locations.html "If you wish to keep the configuration files elsewhere than the data directory, the postgres -D command-line option or PGDATA environment variable must point to the directory containing the configuration files, and the data_directory parameter must be set in postgresql.conf (or on the command line) to show where the data directory is actually located. Notice that data_directory overrides -D and PGDATA for the location of the data directory, but not for the location of the configuration files." By specifying -D you pointing at /storage/postgresql/9.4/data as the source for the conf files, then changing that with the -o switch. Try your command without the -D switch and let the postgresql.conf file in /etc/postgresql/9.4/main/ provide the data_directory. > > Thank you. > ------------------------------------------------------------------------ > View this message in context: Server tries to read a different config > file than it is supposed to > <http://postgresql.nabble.com/Server-tries-to-read-a-different-config-file-than-it-is-supposed-to-tp5850752.html> > Sent from the PostgreSQL - general mailing list archive > <http://postgresql.nabble.com/PostgreSQL-general-f1843780.html> at > Nabble.com. -- Adrian Klaver adrian.klaver@aklaver.com
On 05/23/2015 07:01 AM, rob stone wrote: > > > > On Sat, 2015-05-23 at 04:23 -0700, twoflower wrote: >> I thought I understood how specifying a config file path for the >> server works, but that's apparently not the case. >> >> The cluster data is at /storage/postgresql/9.4/data. >> >> The config files are at /etc/postgresql/9.4/main (this is the default >> location on Ubuntu). >> >> This is how the beginning of /etc/postgresql/9.4/main/postgresql.conf >> looks like: >> >> data_directory = '/storage/postgresql/9.4/data' >> hba_file = '/etc/postgresql/9.4/main/pg_hba.conf' >> ident_file = '/etc/postgresql/9.4/main/pg_ident.conf' >> >> So I wrote a few scripts to make my life easier, e.g. pg94start.sh: >> >> su postgres -c "/usr/lib/postgresql/9.4/bin/pg_ctl >> -D /storage/postgresql/9.4/data -o '-c >> config_file=/etc/postgresql/9.4/main/postgresql.conf'" >> >> But running this script did not work, the server would not start. So I >> checked the log file and there was: >> >> FATAL: could not open file >> "/storage/postgresql/9.4/data/postgresql.conf": Permission denied >> >> After fixing the ownership of this file, it worked. >> >> What's the reason the server was trying to access that file? Why does >> not the override given by the config_file parameter work? >> >> Thank you. >> >> ______________________________________________________________________ >> View this message in context: Server tries to read a different config >> file than it is supposed to >> Sent from the PostgreSQL - general mailing list archive at Nabble.com. > > > Similar hassles with Debian. I use scripts like this to start the > server. > > #! /bin/sh > PGDATA=/home/postgres/data94;export PGDATA > PATH=$PATH:/usr/lib/postgresql/9.4/bin;export PATH > PGBINARIES=/usr/lib/postgresql/9.4/bin;export PGBINARIES > cd /usr/lib/postgresql/9.4/bin > ./pg_ctl start > exit 0 > > I have different scripts depending on the version of Postgres that I > need. I know that I could have a generic script and just pass in the > variables. I run them as the postgres user. > It reads the conf file from the PGDATA path. Any particular reason you do not use pg_ctlcluster?: https://wiki.debian.org/PostgreSql#pg_ctl_replacement > > HTH. > > Robert > > > -- Adrian Klaver adrian.klaver@aklaver.com
twoflower <standa.kurik@gmail.com> writes:
> So I wrote a few scripts to make my life easier, e.g. *pg94start.sh*:
> su postgres -c "/usr/lib/postgresql/9.4/bin/pg_ctl -D
> /storage/postgresql/9.4/data -o '-c
> config_file=/etc/postgresql/9.4/main/postgresql.conf'"
> But running this script did not work, the server would not start.
Testing this, the problem appears to be that you forgot the keyword
"start", so pg_ctl didn't really do anything. It's always a good
idea to redirect pg_ctl's stdout/stderr somewhere, so that you can
look at it in event of problems. (It will *not* magically go to the
server log file.)
> So I
> checked the log file and there was:
> *FATAL: could not open file "/storage/postgresql/9.4/data/postgresql.conf":
> Permission denied*
I suspect this was left over from some previous attempt.
> After fixing the ownership of this file, it worked.
I can't explain that claim, but for me, -c config_file=something
seems to work as you'd expect, and a look at the server source
code confirms that it should honor that (cf SelectConfigFiles()).
I think the documentation Adrian pointed to is a bit out of date,
or at least oversimplified.
One possible theory is that you had an "include" directive in
the config file in /etc, causing it to try to read the other one?
regards, tom lane
I am sorry, that was just a mistake on my part here, it is in the script.Testing this, the problem appears to be that you forgot the keyword "start", so pg_ctl didn't really do anything.
It doesn't look like it. I tried several times, always looked in the log file after that. The timestamp also suggests it was a result of my attempts.I suspect this was left over from some previous attempt.
I checked now, no "include" in there.One possible theory is that you had an "include" directive in the config file in /etc, causing it to try to read the other one?
View this message in context: Re: Server tries to read a different config file than it is supposed to
Sent from the PostgreSQL - general mailing list archive at Nabble.com.
On 05/23/2015 08:06 AM, Tom Lane wrote: > twoflower <standa.kurik@gmail.com> writes: >> So I wrote a few scripts to make my life easier, e.g. *pg94start.sh*: >> su postgres -c "/usr/lib/postgresql/9.4/bin/pg_ctl -D >> /storage/postgresql/9.4/data -o '-c >> config_file=/etc/postgresql/9.4/main/postgresql.conf'" >> But running this script did not work, the server would not start. > > Testing this, the problem appears to be that you forgot the keyword > "start", so pg_ctl didn't really do anything. It's always a good > idea to redirect pg_ctl's stdout/stderr somewhere, so that you can > look at it in event of problems. (It will *not* magically go to the > server log file.) > >> So I >> checked the log file and there was: >> *FATAL: could not open file "/storage/postgresql/9.4/data/postgresql.conf": >> Permission denied* > > I suspect this was left over from some previous attempt. > >> After fixing the ownership of this file, it worked. > > I can't explain that claim, but for me, -c config_file=something > seems to work as you'd expect, and a look at the server source > code confirms that it should honor that (cf SelectConfigFiles()). > I think the documentation Adrian pointed to is a bit out of date, > or at least oversimplified. So order on the the command line is not important, the start up code sets its own precedence? > > One possible theory is that you had an "include" directive in > the config file in /etc, causing it to try to read the other one? > > regards, tom lane > > -- Adrian Klaver adrian.klaver@aklaver.com
Adrian Klaver <adrian.klaver@aklaver.com> writes:
> On 05/23/2015 08:06 AM, Tom Lane wrote:
>> I can't explain that claim, but for me, -c config_file=something
>> seems to work as you'd expect, and a look at the server source
>> code confirms that it should honor that (cf SelectConfigFiles()).
>> I think the documentation Adrian pointed to is a bit out of date,
>> or at least oversimplified.
> So order on the the command line is not important, the start up code
> sets its own precedence?
Order on the command line would only matter if you wrote -c config_file
twice. The point here is that an explicit setting for config_file on the
command line overrides the default based on -D. Or at least it's supposed
to. I'm not sure now what's going wrong for the OP.
regards, tom lane
Re: Re: Server tries to read a different config file than it is supposed to
From
Adrian Klaver
Date:
On 05/23/2015 08:23 AM, twoflower wrote: > Testing this, the problem appears to be that you forgot the keyword > "start", so pg_ctl didn't really do anything. > > I am sorry, that was just a mistake on my part here, it is in the script. > > I suspect this was left over from some previous attempt. > > It doesn't look like it. I tried several times, always looked in the log > file after that. The timestamp also suggests it was a result of my > attempts. So on my Ubuntu installs it does not set up the postgres user to allow login, so how are you getting to: su postgres -c ... Have you overridden the Ubuntu settings or created your own postgres user? The reason I ask is that I can the below to work on Ubuntu 14.04: aklaver@arkansas:~/postgres_test$ sudo su postgres -c "/usr/lib/postgresql/9.4/bin/pg_ctl -D /home/aklaver/postgres_test/9.4/main/-o '-c config_file=/etc/postgresql/9.4/main/postgresql.conf' start" server starting aklaver@arkansas:~/postgres_test$ 2015-05-23 14:02:15 PDT [20033-1] LOG: database system was shut down at 2015-05-23 13:49:04PDT 2015-05-23 14:02:15 PDT [20037-1] LOG: autovacuum launcher started 2015-05-23 14:02:15 PDT [20032-1] LOG: database system is ready to accept connections > > One possible theory is that you had an "include" directive in the > config file in /etc, causing it to try to read the other one? > > I checked now, no "include" in there. > ------------------------------------------------------------------------ > View this message in context: Re: Server tries to read a different > config file than it is supposed to > <http://postgresql.nabble.com/Server-tries-to-read-a-different-config-file-than-it-is-supposed-to-tp5850752p5850763.html> > Sent from the PostgreSQL - general mailing list archive > <http://postgresql.nabble.com/PostgreSQL-general-f1843780.html> at > Nabble.com. -- Adrian Klaver adrian.klaver@aklaver.com
Adrian Klaver <adrian.klaver@aklaver.com> writes: > So on my Ubuntu installs it does not set up the postgres user to allow login, so how are you getting to: > su postgres -c ... From root, presumably ... I thought of a different theory: maybe the server's complaint is not due to trying to read that file as a config file, but it's just because there is an unreadable/unwritable file in the data directory. See Christoph Berg's complaint at http://www.postgresql.org/message-id/20150523172627.GA24277@msg.df7cb.de This would only apply if the OP was trying to use this week's releases though. Also, I thought the fsync-everything code would only run if the server had been shut down uncleanly. Which maybe it was, but that bit of info wasn't provided either. regards, tom lane
Re: Re: Server tries to read a different config file than it is supposed to
From
Adrian Klaver
Date:
On 05/23/2015 02:42 PM, Tom Lane wrote: > Adrian Klaver <adrian.klaver@aklaver.com> writes: >> So on my Ubuntu installs it does not set up the postgres user to allow login, so how are you getting to: >> su postgres -c ... > >From root, presumably ... Aargh, so used to the little voice telling me not to do things as root I forgot that possibility. > > I thought of a different theory: maybe the server's complaint is not due > to trying to read that file as a config file, but it's just because there > is an unreadable/unwritable file in the data directory. See Christoph > Berg's complaint at > http://www.postgresql.org/message-id/20150523172627.GA24277@msg.df7cb.de > > This would only apply if the OP was trying to use this week's releases > though. Also, I thought the fsync-everything code would only run if > the server had been shut down uncleanly. Which maybe it was, but that > bit of info wasn't provided either. > > regards, tom lane > > -- Adrian Klaver adrian.klaver@aklaver.com
YesFrom root, presumably ...
I thought of a different theory: maybe the server's complaint is not due to trying to read that file as a config file, but it's just because there is an unreadable/unwritable file in the data directory. See Christoph Berg's complaint at http://www.postgresql.org/message-id/20150523172627.GA24277@... This would only apply if the OP was trying to use this week's releases though. Also, I thought the fsync-everything code would only run if the server had been shut down uncleanly. Which maybe it was, but that bit of info wasn't provided either.
I was doing this after I upgraded to 9.4.2, yes. As for the shut down: I suspect the server was rebooted without explicitly stopping Postgres. Not sure how this plays out in terms of cleanliness. This is everything relevant in the log file after I ran the start script: 2015-05-23 10:36:39.999 GMT [2102][0]: [1] LOG: database system was interrupted; last known up at 2015-05-23 08:59:41 GMT
2015-05-23 10:36:40.053 GMT [2102][0]: [2] FATAL: could not open file "/storage/postgresql/9.4/data/postgresql.conf": Permission denied
2015-05-23 10:36:40.054 GMT [2100][0]: [3] LOG: startup process (PID 2102) exited with exit code 1
2015-05-23 10:36:40.054 GMT [2100][0]: [4] LOG: aborting startup due to startup process failure
I also tried the same situation on two other Ubuntu servers with the same version of Postgres (also upgraded to 9.4.2) and the same directory layout - made postgresql.conf in the data directory unaccessible, even renamed it, and everything worked fine. The only difference is that these are streaming-replicated standby servers. They also had been restarted without explicitly terminating Postgres.
View this message in context: Re: Server tries to read a different config file than it is supposed to
Sent from the PostgreSQL - general mailing list archive at Nabble.com.
twoflower <standa.kurik@gmail.com> writes:
> I was doing this after I upgraded to 9.4.2, yes. As for the shut down: I
> suspect the server was rebooted without explicitly stopping Postgres. Not
> sure how this plays out in terms of cleanliness. This is everything relevant
> in the log file after I ran the start script:
> 2015-05-23 10:36:39.999 GMT [2102][0]: [1] LOG: database system was
> interrupted; last known up at 2015-05-23 08:59:41 GMT
> 2015-05-23 10:36:40.053 GMT [2102][0]: [2] FATAL: could not open file
> "/storage/postgresql/9.4/data/postgresql.conf": Permission denied
> 2015-05-23 10:36:40.054 GMT [2100][0]: [3] LOG: startup process (PID 2102)
> exited with exit code 1
> 2015-05-23 10:36:40.054 GMT [2100][0]: [4] LOG: aborting startup due to
> startup process failure
Yeah, so this is long after the real config file has been read.
I think that that unwritable postgresql.conf file had probably been
hanging around in your data directory for some time. It was not causing
any particular problem until we decided we ought to fsync everything in
the data directory after a crash. So this is indeed the same case
Christoph was complaining about. But really you should remove that file
not just change its permissions; as is it's just causing confusion.
> I also tried the same situation on two other Ubuntu servers with the same
> version of Postgres (also upgraded to 9.4.2) and the same directory layout -
> made *postgresql.conf* in the data directory unaccessible, even renamed it,
> and everything worked fine. The only difference is that these are
> streaming-replicated standby servers. They also had been restarted without
> explicitly terminating Postgres.
Hm. I wonder why we aren't fsync'ing on crash restart on standby servers
as well.
regards, tom lane
Thank you very much Tom for explaining the cause, now I will make sure I am not leaving any unused files in the data directory anymore.I think that that unwritable postgresql.conf file had probably been hanging around in your data directory for some time. It was not causing any particular problem until we decided we ought to fsync everything in the data directory after a crash. So this is indeed the same case Christoph was complaining about. But really you should remove that file not just change its permissions; as is it's just causing confusion.
View this message in context: Re: Server tries to read a different config file than it is supposed to
Sent from the PostgreSQL - general mailing list archive at Nabble.com.