Thread: PG84 and SELinux
OS: CentOS-5.5
Earlier today I attempted to upgrade a production server from 8.1 to
8.4 using the pgdg-84-centos.repo. I say attempted because I could
never get it to support ssl connections and as that is a requirement
I had to roll back to 8.1.
Whatever was the cause of the ssl problem I also encountered a
surprising number of SELinux violations. The following details the
SELinux settings that I ultimately had to apply as a local module.
This took a considerable period of time as each had to be triggered
in turn in order that the error be identified.
#============= postgresql_t ==============
allow postgresql_t var_lib_t:dir rmdir;
allow postgresql_t var_lib_t:file { write getattr link read unlink
append };
Is this to be expected?
--
*** E-Mail is NOT a SECURE channel ***
James B. Byrne mailto:ByrneJB@Harte-Lyne.ca
Harte & Lyne Limited http://www.harte-lyne.ca
9 Brockley Drive vox: +1 905 561 1241
Hamilton, Ontario fax: +1 905 561 0757
Canada L8E 3C3
"James B. Byrne" <byrnejb@harte-lyne.ca> writes:
> Earlier today I attempted to upgrade a production server from 8.1 to
> 8.4 using the pgdg-84-centos.repo. I say attempted because I could
> never get it to support ssl connections and as that is a requirement
> I had to roll back to 8.1.
Can't comment on that without a lot more detail.
> Whatever was the cause of the ssl problem I also encountered a
> surprising number of SELinux violations. The following details the
> SELinux settings that I ultimately had to apply as a local module.
> This took a considerable period of time as each had to be triggered
> in turn in order that the error be identified.
> #============= postgresql_t ==============
> allow postgresql_t var_lib_t:dir rmdir;
> allow postgresql_t var_lib_t:file { write getattr link read unlink
> append };
> Is this to be expected?
AFAIK, the Red Hat RPMs work out-of-the-box with SELinux; I'm a bit
surprised to hear that the PGDG ones don't, because last I heard
they use the same file layout. What the above sounds like to me is that
the data directory tree wasn't correctly labeled as postgresql_db_t.
Maybe a restorecon would have helped?
regards, tom lane
On Wed, December 1, 2010 16:54, Tom Lane wrote:
>> Whatever was the cause of the ssl problem I also encountered a
>> surprising number of SELinux violations. The following details
>> the
>> SELinux settings that I ultimately had to apply as a local module.
>> This took a considerable period of time as each had to be
>> triggered
>> in turn in order that the error be identified.
>
>> #============= postgresql_t ==============
>> allow postgresql_t var_lib_t:dir rmdir;
>> allow postgresql_t var_lib_t:file { write getattr link read unlink
>> append };
>
>> Is this to be expected?
>
> AFAIK, the Red Hat RPMs work out-of-the-box with SELinux; I'm a bit
> surprised to hear that the PGDG ones don't, because last I heard
> they use the same file layout. What the above sounds like to me is
> that
> the data directory tree wasn't correctly labeled as postgresql_db_t.
> Maybe a restorecon would have helped?
>
> regards, tom lane
>
I tried a restorecon as suggested by sealert at the first error. It
had no effect insofar as I could determine.
--
*** E-Mail is NOT a SECURE channel ***
James B. Byrne mailto:ByrneJB@Harte-Lyne.ca
Harte & Lyne Limited http://www.harte-lyne.ca
9 Brockley Drive vox: +1 905 561 1241
Hamilton, Ontario fax: +1 905 561 0757
Canada L8E 3C3
On Wed, December 1, 2010 16:54, Tom Lane wrote: > "James B. Byrne" <byrnejb@harte-lyne.ca> writes: >> Earlier today I attempted to upgrade a production server from 8.1 >> to >> 8.4 using the pgdg-84-centos.repo. I say attempted because I >> could >> never get it to support ssl connections and as that is a >> requirement >> I had to roll back to 8.1. > > Can't comment on that without a lot more detail. Well, the only thing that I had to do was uncomment #ssl = off and set it to ssl = on. Then on service restart I obtained an error that looked similar to this: . . . configuration file routines:STR_COPY:variable has no value:conf_def.c:629:line 207 I cannot remember the exact error unfortunately and as I rolled back the update I have no way to reproduce it at will. -- *** E-Mail is NOT a SECURE channel *** James B. Byrne mailto:ByrneJB@Harte-Lyne.ca Harte & Lyne Limited http://www.harte-lyne.ca 9 Brockley Drive vox: +1 905 561 1241 Hamilton, Ontario fax: +1 905 561 0757 Canada L8E 3C3
On Wed, 2010-12-01 at 16:54 -0500, Tom Lane wrote: > AFAIK, the Red Hat RPMs work out-of-the-box with SELinux; I'm a bit > surprised to hear that the PGDG ones don't, because last I heard > they use the same file layout. They should -- we are using the same routines for initdb'ing. -- Devrim GÜNDÜZ PostgreSQL Danışmanı/Consultant, Red Hat Certified Engineer PostgreSQL RPM Repository: http://yum.pgrpms.org Community: devrim~PostgreSQL.org, devrim.gunduz~linux.org.tr http://www.gunduz.org Twitter: http://twitter.com/devrimgunduz
Attachment
On Thu, December 2, 2010 15:23, Devrim GÜNDÜZ wrote: > On Wed, 2010-12-01 at 16:54 -0500, Tom Lane wrote: >> AFAIK, the Red Hat RPMs work out-of-the-box with SELinux; > > They should -- we are using the same routines for initdb'ing. > I will do a touch /.autorelabel and restart the server before I try again. If there was just something odd about the SELinux contexts on that particular host then that should clear it up. I will report whichever way it goes thereafter. -- *** E-Mail is NOT a SECURE channel *** James B. Byrne mailto:ByrneJB@Harte-Lyne.ca Harte & Lyne Limited http://www.harte-lyne.ca 9 Brockley Drive vox: +1 905 561 1241 Hamilton, Ontario fax: +1 905 561 0757 Canada L8E 3C3
On Thu, December 2, 2010 15:32, James B. Byrne wrote: > > On Thu, December 2, 2010 15:23, Devrim GÜNDÜZ wrote: >> On Wed, 2010-12-01 at 16:54 -0500, Tom Lane wrote: >>> AFAIK, the Red Hat RPMs work out-of-the-box with SELinux; >> >> They should -- we are using the same routines for initdb'ing. >> > > I will do a touch /.autorelabel and restart the server before I try > again. If there was just something odd about the SELinux contexts > on that particular host then that should clear it up. I will report > whichever way it goes thereafter. I restarted the server this morning, waited for the relabel to finish ( a very long time ), and then upgraded to pg-8.4 without any problems. SSL works fine as well. I can only infer that something went seriously wrong with the SELinux context labels on that host. Thank you for the help. -- *** E-Mail is NOT a SECURE channel *** James B. Byrne mailto:ByrneJB@Harte-Lyne.ca Harte & Lyne Limited http://www.harte-lyne.ca 9 Brockley Drive vox: +1 905 561 1241 Hamilton, Ontario fax: +1 905 561 0757 Canada L8E 3C3
On Wed, December 1, 2010 16:54, Tom Lane wrote: > "James B. Byrne" <byrnejb@harte-lyne.ca> writes: >> Earlier today I attempted to upgrade a production server >> from 8.1 to 8.4 using the pgdg-84-centos.repo. I say >> attempted because I could never get it to support ssl >> connections and as that is a requirement I had to roll >> back to 8.1. > > Can't comment on that without a lot more detail. > On Fri, December 3, 2010 07:40, James B. Byrne wrote: > > I restarted the server this morning, waited for the relabel to > finish ( a very long time ), and then upgraded to pg-8.4 without any > problems. SSL works fine as well. . . I wrote too soon. What I did was uncomment the ssl option. I neglected to change the setting from off to on. When I try to start the server with ssl=on it fails with this error: Auto configuration failed 29006:error:0E065068:configuration file routines:STR_COPY:variable has no value:conf_def.c:629:line 207 I have checked gpsql/data and the server.key and server.crt files are both present: -rw------- 1 postgres postgres 5213 Dec 12 2007 server.crt -rw------- 1 postgres postgres 1675 Dec 12 2007 server.key The only change made in the postgresql.conf file that triggered this was changing 'off' to 'on' for ssl. Changing it back to 'off' makes the problem disappear. -- *** E-Mail is NOT a SECURE channel *** James B. Byrne mailto:ByrneJB@Harte-Lyne.ca Harte & Lyne Limited http://www.harte-lyne.ca 9 Brockley Drive vox: +1 905 561 1241 Hamilton, Ontario fax: +1 905 561 0757 Canada L8E 3C3
"James B. Byrne" <byrnejb@harte-lyne.ca> writes:
> I wrote too soon. What I did was uncomment the ssl option. I
> neglected to change the setting from off to on.
> When I try to start the server with ssl=on it fails with this error:
> Auto configuration failed
> 29006:error:0E065068:configuration file routines:STR_COPY:variable
> has no value:conf_def.c:629:line 207
AFAIK there is no place in the standard Postgres sources that could emit
an error message even vaguely like that. I'm guessing that you are
using some add-on code that tries to parse postgresql.conf, but I don't
know what that would be. Whose init script are you using?
regards, tom lane
On Sat, December 4, 2010 01:11, Tom Lane wrote: > "James B. Byrne" <byrnejb@harte-lyne.ca> writes: >> I wrote too soon. What I did was uncomment the ssl option. I >> neglected to change the setting from off to on. > >> When I try to start the server with ssl=on it fails with this >> error: > >> Auto configuration failed >> 29006:error:0E065068:configuration file routines:STR_COPY:variable >> has no value:conf_def.c:629:line 207 > > AFAIK there is no place in the standard Postgres sources that could > emit an error message even vaguely like that. I'm guessing that > you are using some add-on code that tries to parse postgresql.conf, > but I don't know what that would be. Whose init script are you > using? > > regards, tom lane > I see some familiar names. . . # cat postgresql #!/bin/sh # postgresql This is the init script for starting up the PostgreSQL # server # # chkconfig: - 64 36 # description: Starts and stops the PostgreSQL backend daemon that handles \ # all database requests. # processname: postmaster # pidfile: /var/run/postmaster.pid # Version 6.5.3-2 Lamar Owen # Added code to determine if PGDATA exists, whether it is current version # or not, and initdb if no PGDATA (initdb will not overwrite a database). # Version 7.0 Lamar Owen # Added logging code # Changed PGDATA. # Version 7.0.2 Trond Eivind Glomsrd <teg@redhat.com> # use functions, add conditional restart # Version 7.0.3 Lamar Owen <lamar@postgresql.org> # Check for the existence of functions before blindly using them # in particular -- check for success () and failure () before using. # More Cross-distribution support -- PGVERSION variable, and docdir checks. # Version 7.1 Release Candidate Lamar Owen <lamar@postgresql.org> # initdb parameters have changed. # Version 7.1.2 Trond Eivind Glomsrd <teg@redhat.com> # Specify shell for su # Handle stop better - kill unwanted output, make it wait until the database is ready # Handle locales slightly differently - always using "C" isn't a valid option # Kill output from database initialization # Mark messages for translation # Version 7.1.2-2.PGDG Lamar Owen <lamar.owen@wgcr.org> # sync up. # Karl's fixes for some quoting issues. # Version 7.2b2 Lamar Owen <lamar.owen@wgcr.org> # version change. # Version 7.2 final. Lamar Owen <lamar.owen@wgcr.org> # reload from Peter E. # Eliminate the pidof postmaster test in stop -- we're using pg_ctl so we don't need pidof. # Tested the $? return for the stop script -- it does in fact propagate. # TODO: multiple postmasters. # Version 7.3 Lamar Owen <lamar.owen@ramifordistat.net> # Multiple postmasters, courtesy Karl DeBisschop # Version 7.4 Lamar Owen. # Version 7.4.3 Tom Lane <tgl@sss.pgh.pa.us> # Support condstop for uninstall # Minor other changes suggested by Fernando Nasser. # Version 7.4.5 Tom Lane <tgl@sss.pgh.pa.us> # Rewrite to start postmaster directly, rather than via pg_ctl; this avoids # fooling the postmaster's stale-lockfile check by having too many # postgres-owned processes laying about. # Version 8.1 Devrim Gunduz <devrim@PostgreSQL.org> # Increased sleep time from 1 sec to 2 sec. # Version 8.2 Devrim Gunduz <devrim@CommandPrompt.com> # Set initdb as a seperate option. # Version 8.3 Devrim Gunduz <devrim@CommandPrompt.com> # Version 8.4 Devrim Gunduz <devrim@CommandPrompt.com> # Remove "sameuser" from initdb, to match the new hba conf file. # Get rid of duplicate PGDATA assignment. # Ensure pgstartup.log gets the right ownership/permissions during initdb # PGVERSION is the full package version, e.g., 8.4.0 # Note: the specfile ordinarily updates this during install PGVERSION=8.4.4 # PGMAJORVERSION is major version, e.g., 8.4 (this should match PG_VERSION) PGMAJORVERSION=`echo "$PGVERSION" | sed 's/^\([0-9]*\.[0-9]*\).*$/\1/'` -- *** E-Mail is NOT a SECURE channel *** James B. Byrne mailto:ByrneJB@Harte-Lyne.ca Harte & Lyne Limited http://www.harte-lyne.ca 9 Brockley Drive vox: +1 905 561 1241 Hamilton, Ontario fax: +1 905 561 0757 Canada L8E 3C3
Tom Lane wrote:
That looks to be the str_copy routine from conf_def.c in the OpenSSL code, i.e. line 624 of the version at:
http://code.google.com/p/commitmonitor/source/browse/trunk/common/openssl/crypto/conf/conf_def.c
So guessing something in the SSL autonegotiation is failing here in a really unexpected way.
"James B. Byrne" <byrnejb@harte-lyne.ca> writes:I wrote too soon. What I did was uncomment the ssl option. I neglected to change the setting from off to on.When I try to start the server with ssl=on it fails with this error:Auto configuration failed 29006:error:0E065068:configuration file routines:STR_COPY:variable has no value:conf_def.c:629:line 207AFAIK there is no place in the standard Postgres sources that could emit an error message even vaguely like that.
That looks to be the str_copy routine from conf_def.c in the OpenSSL code, i.e. line 624 of the version at:
http://code.google.com/p/commitmonitor/source/browse/trunk/common/openssl/crypto/conf/conf_def.c
So guessing something in the SSL autonegotiation is failing here in a really unexpected way.
-- Greg Smith 2ndQuadrant US greg@2ndQuadrant.com Baltimore, MD PostgreSQL Training, Services and Support www.2ndQuadrant.us
On Mon, December 6, 2010 00:47, Greg Smith wrote: > > That looks to be the str_copy routine from conf_def.c in the OpenSSL > code, i.e. line 624 of the version at: > > http://code.google.com/p/commitmonitor/source/browse/trunk/common/openssl/crypto/conf/conf_def.c > > So guessing something in the SSL autonegotiation is failing here in > a really unexpected way. > The problem was an expired pki certificate. When we first used ssl for pg we did not have our private CA set up. So we generated a self-signed certificate. That certificate expired this past July and I infer that while 8.1 did not care 8.4 evidently does. In any case, we generated a new key and had a certificate signing request signed by our CA. We installed both as server.key and server.crt in the pgsql/data directory with chmod 600 and chown postgres:postgres. Setting the postgresql.conf ssl option to on and restarting the server no longer causes any error. Than you all for the help. -- *** E-Mail is NOT a SECURE channel *** James B. Byrne mailto:ByrneJB@Harte-Lyne.ca Harte & Lyne Limited http://www.harte-lyne.ca 9 Brockley Drive vox: +1 905 561 1241 Hamilton, Ontario fax: +1 905 561 0757 Canada L8E 3C3
On Mon, December 6, 2010 13:29, James B. Byrne wrote: >> > > The problem was an expired pki certificate. When we first used ssl > for pg we did not have our private CA set up. So we generated a > self-signed certificate. That certificate expired this past July > and I infer that while 8.1 did not care 8.4 evidently does. Wrong again. I misinterpreted what I was seeing. Even with the new cert and key I get the same error. Auto configuration failed 16276:error:0E065068:configuration file routines:STR_COPY:variable has no value:conf_def.c:629:line 207 I will try to get help on the CentOS list. -- *** E-Mail is NOT a SECURE channel *** James B. Byrne mailto:ByrneJB@Harte-Lyne.ca Harte & Lyne Limited http://www.harte-lyne.ca 9 Brockley Drive vox: +1 905 561 1241 Hamilton, Ontario fax: +1 905 561 0757 Canada L8E 3C3
I received absolutely no reply to my question on the CentOS mailing list so I have to turn to this venue again for help. I note the following things: postgresql-server.i386 8.4.4-2PGDG.el5 installed openssl.i686 0.9.8e-12.el5_4.6 installed Might there be a problem between the server being compiled for i386 and openssl for i686? I cannot for the life of me determine what configuration problem causes this error. On Fri, December 3, 2010 16:04, James B. Byrne wrote: > When I try to start the server with ssl=on it fails with this error: > > Auto configuration failed > 29006:error:0E065068:configuration file routines:STR_COPY:variable > has no value:conf_def.c:629:line 207 > -- *** E-Mail is NOT a SECURE channel *** James B. Byrne mailto:ByrneJB@Harte-Lyne.ca Harte & Lyne Limited http://www.harte-lyne.ca 9 Brockley Drive vox: +1 905 561 1241 Hamilton, Ontario fax: +1 905 561 0757 Canada L8E 3C3
On Tue, 2010-12-07 at 16:54 -0500, James B. Byrne wrote: > I received absolutely no reply to my question on the CentOS mailing > list so I have to turn to this venue again for help. > > I note the following things: > > postgresql-server.i386 8.4.4-2PGDG.el5 > installed > > openssl.i686 0.9.8e-12.el5_4.6 > installed > > > Might there be a problem between the server being compiled for i386 > and openssl for i686? I cannot for the life of me determine what > configuration problem causes this error. No those lib differences are both still 32bit. You would have a problem if one was 64bit. So you should be fine there. Joshua D. Drake -- PostgreSQL.org Major Contributor Command Prompt, Inc: http://www.commandprompt.com/ - 509.416.6579 Consulting, Training, Support, Custom Development, Engineering http://twitter.com/cmdpromptinc | http://identi.ca/commandprompt
On Tue, December 7, 2010 16:56, Joshua D. Drake wrote: > > No those lib differences are both still 32bit. You would have a > problem if one was 64bit. So you should be fine there. > > Joshua D. Drake > Ok. How do I get postgresql to cough up more processing detail on startup? The message that I presently get makes no sense at all to me. -- *** E-Mail is NOT a SECURE channel *** James B. Byrne mailto:ByrneJB@Harte-Lyne.ca Harte & Lyne Limited http://www.harte-lyne.ca 9 Brockley Drive vox: +1 905 561 1241 Hamilton, Ontario fax: +1 905 561 0757 Canada L8E 3C3
"James B. Byrne" <byrnejb@harte-lyne.ca> writes:
> Ok. How do I get postgresql to cough up more processing detail on
> startup? The message that I presently get makes no sense at all to
> me.
The message isn't coming from postgres --- it's openssl that you're
wishing would be more verbose.
What I'd try next is strace'ing the postmaster so you can see what
happened right before the error report. With luck that will point you
at a specific configuration file that's (presumably) messed up.
regards, tom lane
On Tue, Dec 07, 2010 at 05:15:45PM -0500, James B. Byrne wrote: > > On Tue, December 7, 2010 16:56, Joshua D. Drake wrote: > > > > > No those lib differences are both still 32bit. You would have a > > problem if one was 64bit. So you should be fine there. > > > > Joshua D. Drake > > > > Ok. How do I get postgresql to cough up more processing detail on > startup? The message that I presently get makes no sense at all to > me. do we know that pg was compiled with ssl? maybe a bonehead question, but low hanging fruit is my specialty.
I have now tracked down and resolved the problem. There were clues to the solution in the error message but I lacked sufficient experience with ssl to realize it. The error was an uncommented line in /etc/pki/tls/openssl.cnf that depended upon an environment variable (ALTNAME) being set (subjectAltName=$ENV::ALTNAME). This was line 270 in that file. Note the error message: > Auto configuration failed > 29006:error:0E065068:configuration file routines:STR_COPY:variable > has no value:conf_def.c:629:line 207 Given what I know now I infer that conf_def is the variable that holds the actual file name of whatever configuration file is passed to openssl. The error message would have been far more informative had it provided the variable value rather than the variable name. And, I have no idea why PG84 choked on this and PG81 did not. Anyway, our upgraded PG84 service is now running with ssl enabled. Many thanks for the hints and suggestions. They did in fact eventually point me in the right direction. -- *** E-Mail is NOT a SECURE channel *** James B. Byrne mailto:ByrneJB@Harte-Lyne.ca Harte & Lyne Limited http://www.harte-lyne.ca 9 Brockley Drive vox: +1 905 561 1241 Hamilton, Ontario fax: +1 905 561 0757 Canada L8E 3C3