Thread: Making PostgreSQL 7.4 (CVS) work properly on OS X 10.3 (7B85)
Here's my notes on how to build PostgreSQL 7.4 (CVS) on OS X 10.3 (7B85) with (seems to be working, but I haven't really ran any tests) python, tcl, perl, readline. I noticed today that you guys are just about to release, so I figure I should bring this up right now. Please feel free to email me if you have any questions/comments, copy me off-list if possible because I probably won't stay subscribed for very long. Notes: ---------- * CVS co pgsql at about Nov 7 2003 5pm EST * These instructions PROBABLY DO NOT WORK ON 10.2! If these changes to into a configure script, make sure to trap them against darwin >= 7.0.0 * downloaded and built static readline * downloaded and built latest bison 1.875 * /usr/local/bin comes before /usr/bin in my path! (this matters for bison, probably) * Probably a good idea to include kerberos support as well, since OS X Server 10.3 uses it a lot, but I don't know enough about it to try.. * I have set env MAC_OS_X_DEPLOYMENT_TARGET 10.3 * I ditched the system.c hack, assuming Apple has fixed them by 10.3 -- because it breaks tcl and python if you do.. I did: #include <AccessibilityMacros.h> #ifdef MAC_OS_X_10_3 ... original code #endif This can be done nicer, of course. It seems that you can't sysctl some things after they've been set once.. so I had to sudo vi /etc/rc file, changed the following two parameters sysctl -w kern.sysv.shmmax=167772160 sysctl -w kern.sysv.shmall=65536 Then rebooted. Shell commands: ------------------------ cd pgsql mkdir build_osx cd build_osx ../configure --with-tcl --without-tk --with-tclconfig=/Library/Frameworks/Tcl.framework/Versions/8.4 --with-perl --with-python --without-java --with-rendezvous --with-openssl make Problems: -------------- *** Cannot build PL/Python because libpython is not a shared library. *** You might have to rebuild your Python installation. Refer to *** the documentation for details. Ok, that's not quite true, so: I edited src/Makefile.global to have python_libspec = -framework Python You could probably just grep the LINKFORSHARED line out of /System/Library/Frameworks/Python.framework/Versions/2.3/lib/python2.3/ config/Makefile, but it links to things that aren't necessary for the plugin bundle. Then I edited src/pl/plpython/Makefile and changed the "ifneq" that makes it not build to "ifneq (justbuildit, ....." so it would try and build it anyways. This is a stupid hack. These things should be fixed in the configure process, but I'm too lazy to hack that at the moment. Shell commands: ------------------------ make sudo make install Shell commands: ------------------------ sudo mkdir /usr/local/pgsql/data sudo chown mysql /usr/local/pgsql/data sudo -u mysql /usr/local/pgsql/bin/initdb -D /usr/local/pgsql/data sudo -u mysql /usr/local/pgsql/bin/pg_ctl -D /usr/local/pgsql/data -l /usr/local/pgsql/data/postgres.log start sudo -u mysql /usr/local/pgsql/bin/createdb test sudo -u mysql /usr/local/pgsql/bin/createlang pltcl test sudo -u mysql /usr/local/pgsql/bin/createlang plpythonu test sudo -u mysql /usr/local/pgsql/bin/createlang plperl test Notes: --------- * Yes I know it's blasphemous to use mysql as the username, but OS X comes with a mysql account, just think of it as an "easy to use uid 74". Yes, I'm lazy. * Yes, my only test is createlang.. but that crashes for tcl and python w/o the system fix.
Bob Ippolito <bob@redivi.com> writes:
> Here's my notes on how to build PostgreSQL 7.4 (CVS) on OS X 10.3
> (7B85) with (seems to be working, but I haven't really ran any tests)
> python, tcl, perl, readline.
I have just in the past couple hours realized that ps_status.c is
seriously broken on OS X 10.3. It appears that Apple has randomly
decided to start #define'ing "BSD", which they did not do in any
prior OS X release, and this confuses ps_status.c into choosing the
wrong method of updating the ps-visible command line. Aside from
causing ps to not show any useful info about postmaster child processes,
this seems to cause dynamic loading of libraries to fail in some
cases :-( ... still trying to understand what's happening there ...
> I noticed today that you guys are just
> about to release, so I figure I should bring this up right now.
Indeed. We need to fix this yesterday.
> * I ditched the system.c hack, assuming Apple has fixed them by 10.3 --
> because it breaks tcl and python if you do.
I don't see why system.c would affect the problem I'm seeing --- does
this really fix pltcl for you?
regards, tom lane
On Nov 8, 2003, at 12:31 AM, Tom Lane wrote: > Bob Ippolito <bob@redivi.com> writes: >> Here's my notes on how to build PostgreSQL 7.4 (CVS) on OS X 10.3 >> (7B85) with (seems to be working, but I haven't really ran any tests) >> python, tcl, perl, readline. > > I have just in the past couple hours realized that ps_status.c is > seriously broken on OS X 10.3. It appears that Apple has randomly > decided to start #define'ing "BSD", which they did not do in any > prior OS X release, and this confuses ps_status.c into choosing the > wrong method of updating the ps-visible command line. Aside from > causing ps to not show any useful info about postmaster child > processes, > this seems to cause dynamic loading of libraries to fail in some > cases :-( ... still trying to understand what's happening there ... __APPLE__ is usually the only define you should really be depending on (other than things you pass in yourself via configure). >> I noticed today that you guys are just >> about to release, so I figure I should bring this up right now. > > Indeed. We need to fix this yesterday. > >> * I ditched the system.c hack, assuming Apple has fixed them by 10.3 >> -- >> because it breaks tcl and python if you do. > > I don't see why system.c would affect the problem I'm seeing --- does > this really fix pltcl for you? If you're getting the NSLinkModule -> mmap crash, then yes. It's the multiply defined symbol that kills it (system in postmaster, system in libSystem). The other crashes I saw were related to the shared mem limit that you can only set *once*.. so you have to sudo vi /etc/rc instead of /etc/sysctl.conf :( -bob
Bob Ippolito <bob@redivi.com> writes:
> On Nov 8, 2003, at 12:31 AM, Tom Lane wrote:
>> I have just in the past couple hours realized that ps_status.c is
>> seriously broken on OS X 10.3. It appears that Apple has randomly
>> decided to start #define'ing "BSD",
> __APPLE__ is usually the only define you should really be depending on
It is not that we are failing to recognize Darwin, it is that Darwin is
falsely claiming to be something else.
>> I don't see why system.c would affect the problem I'm seeing --- does
>> this really fix pltcl for you?
> If you're getting the NSLinkModule -> mmap crash, then yes.
No, I'm seeing an NSLinkModule -> memcmp crash. memcmp() is crashing
because it is being passed a null pointer, and the pointer in question
is null because ps_status.c zeroed it via "argv[1] = NULL".
I modified ps_status to use the correct status update method
(PS_CLOBBER_ARGV) but it still failed :-(. It sorta looks like
something in the dynamic loader may be relying on a stale copy of
the "environ" pointer ... does that ring a bell at all?
regards, tom lane
On Nov 8, 2003, at 1:00 AM, Tom Lane wrote: > Bob Ippolito <bob@redivi.com> writes: >> On Nov 8, 2003, at 12:31 AM, Tom Lane wrote: >>> I have just in the past couple hours realized that ps_status.c is >>> seriously broken on OS X 10.3. It appears that Apple has randomly >>> decided to start #define'ing "BSD", > >> __APPLE__ is usually the only define you should really be depending on > > It is not that we are failing to recognize Darwin, it is that Darwin is > falsely claiming to be something else. > >>> I don't see why system.c would affect the problem I'm seeing --- does >>> this really fix pltcl for you? > >> If you're getting the NSLinkModule -> mmap crash, then yes. > > No, I'm seeing an NSLinkModule -> memcmp crash. memcmp() is crashing > because it is being passed a null pointer, and the pointer in question > is null because ps_status.c zeroed it via "argv[1] = NULL". > > I modified ps_status to use the correct status update method > (PS_CLOBBER_ARGV) but it still failed :-(. It sorta looks like > something in the dynamic loader may be relying on a stale copy of > the "environ" pointer ... does that ring a bell at all? Er... I meant memcmp.. Have you tried removing the system.c hack? That's what fixed it for me. I'm pretty sure that the multiply defined _system is messing with things. Is it possible to use execve to rewrite argv[0]? I've had better luck with that. -bob
Bob Ippolito <bob@redivi.com> writes:
> On Nov 8, 2003, at 12:31 AM, Tom Lane wrote:
>> I have just in the past couple hours realized that ps_status.c is
>> seriously broken on OS X 10.3.
> Er... I meant memcmp.. Have you tried removing the system.c hack?
> That's what fixed it for me.
AFAICT system.c hasn't got anything to do with the problem that I'm
seeing; it's purely a matter of ps_status.c clobbering argv[] contents
that the dynamic loader depends on for some weird reason. It looks
like Apple's implementation stores a copy of the original argc count,
and there is a bit of code in the loader that for some reason is
examining each argv string from 0..original_argc-1. Who knows why :-(
... but where we set save_argv[1] to NULL, we create a null pointer
crash in the loader. Take that out, no crash. You would not see this
crash if you started the postmaster with no command-line arguments, btw.
I'm planning to change ps_status so that instead of zeroing
save_argv[1], it makes all the original argv strings be pointers to ""
except for argv[0].
As for getting rid of system.c, I am not eager to do that since it would
certainly break compatibility with OS X 10.1. We could conditionally
compile it out perhaps. Do you know what #define symbol we could test
for to determine which OS X version we are on?
regards, tom lane
On Nov 8, 2003, at 1:13 PM, Tom Lane wrote: > Bob Ippolito <bob@redivi.com> writes: >> On Nov 8, 2003, at 12:31 AM, Tom Lane wrote: >>> I have just in the past couple hours realized that ps_status.c is >>> seriously broken on OS X 10.3. > >> Er... I meant memcmp.. Have you tried removing the system.c hack? >> That's what fixed it for me. > > AFAICT system.c hasn't got anything to do with the problem that I'm > seeing; it's purely a matter of ps_status.c clobbering argv[] contents > that the dynamic loader depends on for some weird reason. It looks > like Apple's implementation stores a copy of the original argc count, > and there is a bit of code in the loader that for some reason is > examining each argv string from 0..original_argc-1. Who knows why :-( > ... but where we set save_argv[1] to NULL, we create a null pointer > crash in the loader. Take that out, no crash. You would not see this > crash if you started the postmaster with no command-line arguments, > btw. > > I'm planning to change ps_status so that instead of zeroing > save_argv[1], it makes all the original argv strings be pointers to "" > except for argv[0]. It may be causing problems because dyld does this thing called @executable_path substitution so it can find dylibs relative to the executable. Also, the WindowServer and several other things (CoreFoundation, Foundation) use argv[0] to determine whether the executable is inside a bundle or not. I don't think OS X people would make a big fuss about argv[0] not being as useful as it is on other platforms, personally I'd just take it out if it's not working. > As for getting rid of system.c, I am not eager to do that since it > would > certainly break compatibility with OS X 10.1. We could conditionally > compile it out perhaps. Do you know what #define symbol we could test > for to determine which OS X version we are on? See /usr/include/AvailabilityMacros.h -bob
Bob Ippolito <bob@redivi.com> writes:
> On Nov 8, 2003, at 1:13 PM, Tom Lane wrote:
>> As for getting rid of system.c, I am not eager to do that since it
>> would
>> certainly break compatibility with OS X 10.1. We could conditionally
>> compile it out perhaps. Do you know what #define symbol we could test
>> for to determine which OS X version we are on?
> See /usr/include/AvailabilityMacros.h
I don't see anything there that we can use in the form
#ifdef OSX_VERSION_10_2
or
#if OSX_VERSION >= something
My 10.2.6 copy already has MAC_OS_X_VERSION_10_3 in it, so they are
obviously not intending that the highest defined symbol of that
series is the OS version.
regards, tom lane
On Nov 8, 2003, at 1:44 PM, Tom Lane wrote: > Bob Ippolito <bob@redivi.com> writes: >> On Nov 8, 2003, at 1:13 PM, Tom Lane wrote: >>> As for getting rid of system.c, I am not eager to do that since it >>> would >>> certainly break compatibility with OS X 10.1. We could conditionally >>> compile it out perhaps. Do you know what #define symbol we could >>> test >>> for to determine which OS X version we are on? > >> See /usr/include/AvailabilityMacros.h > > I don't see anything there that we can use in the form > > #ifdef OSX_VERSION_10_2 > or > #if OSX_VERSION >= something > > My 10.2.6 copy already has MAC_OS_X_VERSION_10_3 in it, so they are > obviously not intending that the highest defined symbol of that > series is the OS version. MAC_OS_X_VERSION_MIN_REQUIRED * If min OS not specified, assume 10.0 * Note: gcc driver may set MAC_OS_X_VERSION_MIN_REQUIRED based on MACOSX_DEPLOYMENT_TARGET environment variable Compiling software for 10.3 should setenv MACOSX_DEPLOYMENT_TARGET 10.3 -bob
Bob Ippolito <bob@redivi.com> writes:
> * I ditched the system.c hack, assuming Apple has fixed them by 10.3 --
> because it breaks tcl and python if you do.. I did:
After fixing the ps_status problems, I cannot observe any problem, with
or without system.c. However, I agree that it's a bad idea to propagate
that hack forward when it's no longer needed. I've #ifdef'd out
system.c for OS X 10.3 and later. (I was tempted to disable it on
10.2.* as well, but I can only prove system() works in 10.2.6, not
having older OS X handy to test.)
> Ok, that's not quite true, so:
> I edited src/Makefile.global to have
> python_libspec = -framework Python
I'm not going to touch that at this point in the release cycle. It'll
just have to wait for a future release.
regards, tom lane
On Nov 8, 2003, at 3:19 PM, Tom Lane wrote: > Bob Ippolito <bob@redivi.com> writes: >> * I ditched the system.c hack, assuming Apple has fixed them by 10.3 >> -- >> because it breaks tcl and python if you do.. I did: > > After fixing the ps_status problems, I cannot observe any problem, with > or without system.c. However, I agree that it's a bad idea to > propagate > that hack forward when it's no longer needed. I've #ifdef'd out > system.c for OS X 10.3 and later. (I was tempted to disable it on > 10.2.* as well, but I can only prove system() works in 10.2.6, not > having older OS X handy to test.) Sounds great >> Ok, that's not quite true, so: >> I edited src/Makefile.global to have >> python_libspec = -framework Python > > I'm not going to touch that at this point in the release cycle. It'll > just have to wait for a future release. That's fine, does anyone even use plpython? I only compiled it because I knew how to fix it.. -bob
On 8 Nov 2003, at 22:19, Tom Lane wrote: > After fixing the ps_status problems, I cannot observe any problem, with > or without system.c. However, I agree that it's a bad idea to > propagate > that hack forward when it's no longer needed. I've #ifdef'd out > system.c for OS X 10.3 and later. (I was tempted to disable it on > 10.2.* as well, but I can only prove system() works in 10.2.6, not > having older OS X handy to test.) As a general rule of thumb, I don't think it's necessary to support anything other than the latest revision in 10.2.x and 10.3.x at any time. The revisions are free, they mainly fix bugs, they don't affect hardware requirements and other software vendors see requiring them as a non-issue. This, at the very least, applies to major pgsql versions such as 7.4 and 7.5 -- you might disagree about a OS version requirement changing in a point release. But in my opinion, if system() is fixed in 10.2.6, you can require Jaguar users to upgrade to that version for 7.4. mk
Marko Karppinen <marko@karppinen.fi> writes:
> But in my opinion, if system() is fixed in 10.2.6, you can require
> Jaguar users to upgrade to that version for 7.4.
[shrug] If there were any need to, I'd do so. But we have no evidence
of any trouble caused by the system() hack in 10.2, so I think the
most prudent course is just to leave it alone.
regards, tom lane