Thread: Postgresql fails to start?

Postgresql fails to start?

From
Dianne Yumul
Date:
Hello list,

We have Fedora Core 4 Test 3 installed on a development box and just
recently installed Postgresql 8.0.3. Sometimes, when starting
postgresql with the /etc/rc.d/init.d/postgresql script, it gives me the
following:

/usr/bin/chcon: can't apply partial context to unlabeled file
/home/postgres/pgstartup.log

Sometimes it is followed by the OK message (postgresql started OK), and
sometimes by the FAILED message. But either way, when you do a ps aux |
grep postmaster it shows postmaster is running.

I googled and found that it is related to SELinux, but I already have
it disabled and even passed selinux=0 to the kernel via grub.conf. Am I
missing something? Sorry, but I'm very new to this SELinux stuff and
was hoping will get help here from you gurus before asking the SELinux
gurus.

Thank you so much.

Dianne


Re: Postgresql fails to start?

From
Tom Lane
Date:
Dianne Yumul <dianne@wellsgaming.com> writes:
> We have Fedora Core 4 Test 3 installed on a development box and just
> recently installed Postgresql 8.0.3. Sometimes, when starting
> postgresql with the /etc/rc.d/init.d/postgresql script, it gives me the
> following:

> /usr/bin/chcon: can't apply partial context to unlabeled file
> /home/postgres/pgstartup.log

Um, exactly which Postgres RPMs do you have installed?  I recall that
there was a build that had this problem (because I hadn't read the chcon
man page ;-)) but I did not think it had escaped into the wild.

            regards, tom lane

Re: Postgresql fails to start?

From
Dianne Yumul
Date:
> Um, exactly which Postgres RPMs do you have installed?

Thanks for the response. I have the following installed:

postgresql-server-8.0.3-1
postgresql-8.0.3-1
postgresql-jdbc-8.0.3-1
postgresql-libs-8.0.3-1
postgresql-devel-8.0.3-1

I downloaded these from the fedora development repository. I used to
have 8.0.2 installed but I upgraded to 8.0.3, hoping that would fix it
but it didn't.

Thank you.

Dianne

On Jun 6, 2005, at 8:00 PM, Tom Lane wrote:

> Dianne Yumul <dianne@wellsgaming.com> writes:
>> We have Fedora Core 4 Test 3 installed on a development box and just
>> recently installed Postgresql 8.0.3. Sometimes, when starting
>> postgresql with the /etc/rc.d/init.d/postgresql script, it gives me
>> the
>> following:
>
>> /usr/bin/chcon: can't apply partial context to unlabeled file
>> /home/postgres/pgstartup.log
>
> Um, exactly which Postgres RPMs do you have installed?  I recall that
> there was a build that had this problem (because I hadn't read the
> chcon
> man page ;-)) but I did not think it had escaped into the wild.
>
>             regards, tom lane
>
> ---------------------------(end of
> broadcast)---------------------------
> TIP 9: the planner will ignore your desire to choose an index scan if
> your
>       joining column's datatypes do not match
>


Re: Postgresql fails to start?

From
Dianne Yumul
Date:
> Can you say anything about the difference between the times where it
> gives the message and the times where it doesn't?

I get the message when I remove pgstartup.log before starting
postgresql. But if I leave pgstartup.log, it just says it failed to
start. Please see output below:

# /etc/rc.d/init.d/postgresql stop
Stopping postgresql service:                               [  OK  ]
# ps aux | grep postmaster
root     10427  0.0  0.0   3752   664 pts/0    R+   09:24   0:00 grep
postmaster
# rm -f pgstartup.log
# /etc/rc.d/init.d/postgresql start
/usr/bin/chcon: can't apply partial context to unlabeled file
/home/postgres/pgstartup.log
Starting postgresql service:                               [FAILED]
# ps aux | grep postmaster
postgres 10470 34.1  0.6 112404  5248 pts/0    S    09:24   0:02
/usr/bin/postmaster -p 5432 -D /home/postgres/data
root     10480  0.0  0.0   3756   668 pts/0    R+   09:24   0:00 grep
postmaster

The pgstartup.log is empty. Also, data/pg_log/postgresql-Tue.log does
not contain any errors just stuff about database shutting down and
starting up. Now, trying without removing pgstartup.log:

# /etc/rc.d/init.d/postgresql stop
Stopping postgresql service:                               [  OK  ]
# ps aux | grep postmaster
root     10244  0.0  0.0   3756   700 pts/0    R+   09:20   0:00 grep
postmaster
# /etc/rc.d/init.d/postgresql start
Starting postgresql service:                               [FAILED]
# ps aux | grep postmaster
postgres 10566 20.6  0.6 112408  5312 pts/0    S    09:26   0:02
/usr/bin/postmaster -p 5432 -D /home/postgres/data
root     10577  0.0  0.0   3752   668 pts/0    R+   09:26   0:00 grep
postmaster

I remember getting the message that postgresql started OK once or twice
yesterday, but could not get it to do now : (

>     ls -Z /home/postgres/pgstartup.log

Gives:

Sorry, this option can only be used on a SELinux kernel.

But after setting selinux to permissive mode, ls -Z
/home/postgres/pgstartup.log gives:

-rw-------  postgres postgres system_u:object_r:postgresql_log_t
/home/postgres/pgstartup.log

And I also tried starting postgresql with selinux in permissive mode,
but no dice.

> Oh, and is the message really complaining
> about /home/postgres/pgstartup.log?  AFAICS the initscript should be
> messing with /var/lib/pgsql/pgstartup.log.

Sorry, I forgot to mention that I changed some of the defaults via
/etc/sysconfig/pgsql/postgresql file, which looks like this:

PGDATA=/home/postgres/data
PGLOG=/home/postgres/pgstartup.log

But even leaving the defaults alone gives me the same results:

# /etc/rc.d/init.d/postgresql start
/usr/bin/chcon: can't apply partial context to unlabeled file
/var/lib/pgsql/pgstartup.log
Initializing database:                                     [  OK  ]
Starting postgresql service:                               [FAILED]
# ps aux | grep postmaster
postgres 11003  9.4  0.3  19464  2988 pts/0    S    09:36   0:02
/usr/bin/postmaster -p 5432 -D /var/lib/pgsql/data

Many thanks for the help.

Dianne

On Jun 7, 2005, at 9:02 AM, Tom Lane wrote:

> Dianne Yumul <dianne@wellsgaming.com> writes:
>>> Um, exactly which Postgres RPMs do you have installed?
>> Thanks for the response. I have the following installed:
>
>> postgresql-server-8.0.3-1
>> postgresql-8.0.3-1
>> postgresql-jdbc-8.0.3-1
>> postgresql-libs-8.0.3-1
>> postgresql-devel-8.0.3-1
>
> Hm, those are the most current versions.
>
> Can you say anything about the difference between the times where it
> gives the message and the times where it doesn't?  Does
>     ls -Z /home/postgres/pgstartup.log
> show anything different?  Oh, and is the message really complaining
> about /home/postgres/pgstartup.log?  AFAICS the initscript should be
> messing with /var/lib/pgsql/pgstartup.log.
>
>             regards, tom lane
>


Re: Postgresql fails to start?

From
Tom Lane
Date:
Dianne Yumul <dianne@wellsgaming.com> writes:
> # rm -f pgstartup.log
> # /etc/rc.d/init.d/postgresql start
> /usr/bin/chcon: can't apply partial context to unlabeled file
> /home/postgres/pgstartup.log
> Starting postgresql service:                               [FAILED]
> # ps aux | grep postmaster
> postgres 10470 34.1  0.6 112404  5248 pts/0    S    09:24   0:02
> /usr/bin/postmaster -p 5432 -D /home/postgres/data
> root     10480  0.0  0.0   3756   668 pts/0    R+   09:24   0:00 grep
> postmaster

Odd.  I don't see any reason that you'd not get the OK message, given
that the postmaster started.  I've seen reports of that before, eg,
https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=152931
but it's far from clear why some people see this and some don't.
In any case I suspect that the FAILED message is unrelated to the chcon
problem.

> ls -Z /home/postgres/pgstartup.log
> Gives:
> Sorry, this option can only be used on a SELinux kernel.

> But after setting selinux to permissive mode, ls -Z
> /home/postgres/pgstartup.log gives:

> -rw-------  postgres postgres system_u:object_r:postgresql_log_t
> /home/postgres/pgstartup.log

There is something very flaky about either your kernel or your selinux
utilities.  Are you up-to-date on kernel version?  (Let's see ... my
FC3 machine is at 2.6.11-1.14_FC3, and I think I may be behind ...)

I tried to reproduce the problem by doing what the initscript does,
without any success:

[tgl@rh1 ~]$ sudo mkdir /home/postgres
[tgl@rh1 ~]$ sudo chown postgres:postgres /home/postgres
[tgl@rh1 ~]$ PGLOG=/home/postgres/pgstartup.log
[tgl@rh1 ~]$ sudo touch "$PGLOG"
[tgl@rh1 ~]$ sudo ls -Z $PGLOG
-rw-r--r--  root     root     root:object_r:user_home_t        /home/postgres/pgstartup.log
[tgl@rh1 ~]$ sudo chown postgres:postgres "$PGLOG"
[tgl@rh1 ~]$ sudo chmod go-rwx "$PGLOG"
[tgl@rh1 ~]$ sudo /usr/bin/chcon -u system_u -r object_r -t postgresql_log_t "$PGLOG"
[tgl@rh1 ~]$ sudo ls -Z $PGLOG
-rw-------  postgres postgres system_u:object_r:postgresql_log_t /home/postgres/pgstartup.log
[tgl@rh1 ~]$ uname -r
2.6.11-1.14_FC3
[tgl@rh1 ~]$ rpm -qf /usr/bin/chcon
coreutils-5.2.1-31

so I suspect that what you are running into is either a kernel bug or a
chcon bug; it'd be worth making sure both kernel and coreutils are up to
date.  If you try the sequence by hand, what do you get?

>> Oh, and is the message really complaining
>> about /home/postgres/pgstartup.log?  AFAICS the initscript should be
>> messing with /var/lib/pgsql/pgstartup.log.

> Sorry, I forgot to mention that I changed some of the defaults via
> /etc/sysconfig/pgsql/postgresql file, which looks like this:
> PGDATA=/home/postgres/data
> PGLOG=/home/postgres/pgstartup.log

That is going to give you some problems if you have SELinux in
enforcement mode, because the standard policy will disallow the postgres
daemon from writing anywhere under /home.  You can fix this by fooling
around with the policy and/or manually relabeling the database directory
and files, but the path of least resistance is probably to leave the
database directory in the normal place.

            regards, tom lane

Re: Postgresql fails to start?

From
Dianne Yumul
Date:
> I've seen reports of that before, eg,
> https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=152931
> but it's far from clear why some people see this and some don't.

I feel such a goof ball. I searched bugzilla for an hour yesterday but
for some reason I did not find that bug. But that solved my problem. I
just set sleep 3 and now it works.

I still get the /usr/bin/chcon message but only when I run the command
by hand, even after upgrading coreutils:

# PGLOG=/home/postgres/pgstartup.log
# touch "$PGLOG"
# ls -Z $PGLOG
-rw-r--r--  root     root
/home/postgres/pgstartup.log
# chown postgres:postgres "$PGLOG"
# chmod go-rwx "$PGLOG"
# /usr/bin/chcon -u system_u -r object_r -t postgresql_log_t "$PGLOG"
/usr/bin/chcon: can't apply partial context to unlabeled file
/home/postgres/pgstartup.log
# ls -Z $PGLOG
-rw-------  postgres postgres
/home/postgres/pgstartup.log
# uname -r
2.6.11-1.1286_FC4
# rpm -qf /usr/bin/chcon
coreutils-5.2.1-48

I will try upgrading the kernel to see if that will resolve that issue.

> That is going to give you some problems if you have SELinux in
> enforcement mode, because the standard policy will disallow the
> postgres
> daemon from writing anywhere under /home.

I really should take the time and learn SELinux but I'm crunched right
now and will disable it for now.

Thank you soooooo much for the all the help.

Dianne

On Jun 7, 2005, at 10:58 AM, Tom Lane wrote:

> Dianne Yumul <dianne@wellsgaming.com> writes:
>> # rm -f pgstartup.log
>> # /etc/rc.d/init.d/postgresql start
>> /usr/bin/chcon: can't apply partial context to unlabeled file
>> /home/postgres/pgstartup.log
>> Starting postgresql service:                               [FAILED]
>> # ps aux | grep postmaster
>> postgres 10470 34.1  0.6 112404  5248 pts/0    S    09:24   0:02
>> /usr/bin/postmaster -p 5432 -D /home/postgres/data
>> root     10480  0.0  0.0   3756   668 pts/0    R+   09:24   0:00 grep
>> postmaster
>
> Odd.  I don't see any reason that you'd not get the OK message, given
> that the postmaster started.  I've seen reports of that before, eg,
> https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=152931
> but it's far from clear why some people see this and some don't.
> In any case I suspect that the FAILED message is unrelated to the chcon
> problem.
>
>> ls -Z /home/postgres/pgstartup.log
>> Gives:
>> Sorry, this option can only be used on a SELinux kernel.
>
>> But after setting selinux to permissive mode, ls -Z
>> /home/postgres/pgstartup.log gives:
>
>> -rw-------  postgres postgres system_u:object_r:postgresql_log_t
>> /home/postgres/pgstartup.log
>
> There is something very flaky about either your kernel or your selinux
> utilities.  Are you up-to-date on kernel version?  (Let's see ... my
> FC3 machine is at 2.6.11-1.14_FC3, and I think I may be behind ...)
>
> I tried to reproduce the problem by doing what the initscript does,
> without any success:
>
> [tgl@rh1 ~]$ sudo mkdir /home/postgres
> [tgl@rh1 ~]$ sudo chown postgres:postgres /home/postgres
> [tgl@rh1 ~]$ PGLOG=/home/postgres/pgstartup.log
> [tgl@rh1 ~]$ sudo touch "$PGLOG"
> [tgl@rh1 ~]$ sudo ls -Z $PGLOG
> -rw-r--r--  root     root     root:object_r:user_home_t
> /home/postgres/pgstartup.log
> [tgl@rh1 ~]$ sudo chown postgres:postgres "$PGLOG"
> [tgl@rh1 ~]$ sudo chmod go-rwx "$PGLOG"
> [tgl@rh1 ~]$ sudo /usr/bin/chcon -u system_u -r object_r -t
> postgresql_log_t "$PGLOG"
> [tgl@rh1 ~]$ sudo ls -Z $PGLOG
> -rw-------  postgres postgres system_u:object_r:postgresql_log_t
> /home/postgres/pgstartup.log
> [tgl@rh1 ~]$ uname -r
> 2.6.11-1.14_FC3
> [tgl@rh1 ~]$ rpm -qf /usr/bin/chcon
> coreutils-5.2.1-31
>
> so I suspect that what you are running into is either a kernel bug or a
> chcon bug; it'd be worth making sure both kernel and coreutils are up
> to
> date.  If you try the sequence by hand, what do you get?
>
>>> Oh, and is the message really complaining
>>> about /home/postgres/pgstartup.log?  AFAICS the initscript should be
>>> messing with /var/lib/pgsql/pgstartup.log.
>
>> Sorry, I forgot to mention that I changed some of the defaults via
>> /etc/sysconfig/pgsql/postgresql file, which looks like this:
>> PGDATA=/home/postgres/data
>> PGLOG=/home/postgres/pgstartup.log
>
> That is going to give you some problems if you have SELinux in
> enforcement mode, because the standard policy will disallow the
> postgres
> daemon from writing anywhere under /home.  You can fix this by fooling
> around with the policy and/or manually relabeling the database
> directory
> and files, but the path of least resistance is probably to leave the
> database directory in the normal place.
>
>             regards, tom lane
>
> ---------------------------(end of
> broadcast)---------------------------
> TIP 4: Don't 'kill -9' the postmaster
>


Re: Postgresql fails to start?

From
Tom Lane
Date:
Dianne Yumul <dianne@wellsgaming.com> writes:
> I still get the /usr/bin/chcon message but only when I run the command
> by hand, even after upgrading coreutils:

> # /usr/bin/chcon -u system_u -r object_r -t postgresql_log_t "$PGLOG"
> /usr/bin/chcon: can't apply partial context to unlabeled file

Weird.

> # uname -r
> 2.6.11-1.1286_FC4

It just sunk in that you are running FC4 and not FC3.  That's pretty
bleeding edge stuff ;-) ... maybe what you need is a downgrade not an
upgrade.  But if you still see the problem once you're on the latest
kernel, please do file a bugzilla entry about it.  It seems possible
that the FC4 version of chcon wants another argument beyond the ones
that FC3 knows about.  Either that or it's just a plain bug in chcon.

            regards, tom lane