Thread: Threaded Python vs. PostGreSQL plpython
Ok, I was a good boy and tried -interfaces first. No answer. I'm trying to get a functioning version of plpython on FreeBSD, while using python from the ports system. The problem is that the ports system build python with thread support. postmaster doesn't have thread support, so when the libpython2.2.so is dynamically loaded, it fails to find the thread functions, and the load fails. The first workaround I tried was to build a custom version of the python library that doesn't have thread support. Given that plpython won't let me import the thread modules, this isn't a problem. However, it does mean I have a copy of libpython2.2.so where they dynamic loader can find it, meaning the linker will find it, meaning that future builds of other embedded software - like apache's mod_python - will wind up with the non-threaded library. This is a bad thing, and I'd like to avoid it. I tried building linking plpython.so against the static library instead of the dynamica library, but that doesn't work properly when loaded. I'm not sure what the problem is. The ideal solution would be to build PostGreSQL with thread support. I'd rather not find out the hard way that this doesn't work. Does anyone know whether or not I can do that without mangling PostGreSQL? Alternatively, getting a statically linked version of plpython built would mean I could delete the shared library. Anyone have any clues on how to go about getting plpython built with a statically linked libpython? If there's another approach that might work, I'd be interested in hearing about that as well. Thanks,<mike -- Mike Meyer <mwm@mired.org> http://www.mired.org/consulting.html Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.
Mike Meyer writes: > The problem is that the ports system build python with thread > support. postmaster doesn't have thread support, so when the > libpython2.2.so is dynamically loaded, it fails to find the thread > functions, and the load fails. What is "thread support", what are "the thread functions", why does it try to find them in the postmaster, and what are the exact details anyway? > The first workaround I tried was to build a custom version of the > python library that doesn't have thread support. Given that plpython > won't let me import the thread modules, this isn't a problem. However, > it does mean I have a copy of libpython2.2.so where they dynamic > loader can find it, meaning the linker will find it, meaning that > future builds of other embedded software - like apache's mod_python - > will wind up with the non-threaded library. This is a bad thing, and > I'd like to avoid it. With judicious use of rpath you can keep several versions of a shared libpython around without the dynamic loader getting confused. > I tried building linking plpython.so against the static library > instead of the dynamica library, but that doesn't work properly when > loaded. I'm not sure what the problem is. Possibly, the dynamic library would in turn automatically load some other dynamic libraries, either because of implicit linkage or through the run-time dynamic loading mechanism. If you have a static library, then you miss all those things. Hard to tell without getting exact details, though. I know on some platforms it has been shown to be possible to use a static libpython for plpython. -- Peter Eisentraut peter_e@gmx.net
On 24 Mar 2003 at 11:45, Mike Meyer wrote: > In <200303241039.04880.shridhar_daithankar@persistent.co.in>, "Shridhar Daithankar<shridhar_daithankar@persistent.co.in>" <shridhar_daithankar@persistent.co.in> typed: > > Well, I believe, if you link a library against libc_r explicitly, then even if > > it is loaded by a program which is not linked against it would do. > > > > One problem on freeBSD is that the thread library is C library itself. So if > > postgresql is linked against libc.so and libpython is linked against > > libc_r.so, then you *might* get symbol collision. > > > > I suggest you try following. > > It's possible you mean something other than what I think you mean by > "link against", so my replies might not make sense. In which case, > please be more explicit. I mean nothing more than adding -lc_r flag in link time option. That is link against. Hope this is clear. > > 1) Link libpython.so against linuxthreads. > > Ugh. I'd really rather not have a my python depending on > linuxthreads. That's marginally better than having no threads at all, > but that's about it. It's not clear it's worth installing the > linuxthreads port for. OK let me explain. > > 2) Link libpython.so explicitly against libc_r.so, in case it depends upon > > thread fucntions to be provided by loader program.. Check ldd libpython.so > > I know it depends on thread functions provided by the loader > program. It lists one of those as missing when I try and do a > createlang on the shared library built from the installed python. > > Building a shared library from the threaded version and explicitly > linking in libc_r.so - so that ldd shows libc_r as a requirement - > causes the server to terminate on a signal 11 when I try and do a > createlang on plpython. OK. Let's try to explain what is happening here. You have linked libpython.so against libc_r.so. Postgresql is linked against libc.so. These two libraries have large number of symbols common between them. It is quite possible that the SIGSEGV you are getting is due to some symbol collision. Now if you link libpython.so against linuxthreads, it *should* work perfectly with postgresql because linuxthreads library provides only thread functions. libc.so has none of them. So there is no chance of collision and it *should* work. I hope you find this logical. That is the reason I abandoned linuxthreads in one of my projects because I could not stably link linuxthreads and libc_r.so in same program. Anyway this is just to get things linked against and running. You can not run threaded python apps with such a library because libc against which it is linked is not threaded or thread safe. I consider this as a bit of fault with freeBSD libc. Although I can live with it. OTOH, you can link postgresql and libpython.so with libc_r. That should work as well and you can use threads in python too. I would recommend this approach. HTH ByeShridhar -- Bubble Memory, n.: A derogatory term, usually referring to a person's intelligence. See also "vacuum tube".
In <3E803616.19976.2E2BDDB@localhost>, Shridhar Daithankar <shridhar_daithankar@persistent.co.in> typed: > On 24 Mar 2003 at 11:45, Mike Meyer wrote: Just to clarify, the current situation is that I have things working, but I also have a libpython.so that isn't threaded. I'm not happy about the latter, and worry about rebuilding an embedded applications and suddenly having my threaded scripots quit working because it picked up the wrong library. I also have problems in that some modules I should be able to import with plpython don't, because they are linked against the wrong libc. > > In <200303241039.04880.shridhar_daithankar@persistent.co.in>, "Shridhar Daithankar<shridhar_daithankar@persistent.co.in>" <shridhar_daithankar@persistent.co.in> typed: > Now if you link libpython.so against linuxthreads, it *should* work perfectly > with postgresql because linuxthreads library provides only thread functions. > libc.so has none of them. So there is no chance of collision and it *should* > work. > Anyway this is just to get things linked against and running. You can not run > threaded python apps with such a library because libc against which it is > linked is not threaded or thread safe. So this would be similar to my current solution - my real python would be threaded and linked against libc_r, and I'd have a "special" libpython.so that looked like it could be threaded, but in reality couldn't because it uses linuxthreads and a not-thread-safe libc. This is not an improvement - in fact, I'd say it's worse. > OTOH, you can link postgresql and libpython.so with libc_r. That should work as > well and you can use threads in python too. I would recommend this approach. This was my second choice. I was hoping that someone would tell me this was safe before I went to the trouble of trying it. Well, I was hoping someone had a better solution than that, but if this works, I'll do it this way. Thank you,<mike -- Mike Meyer <mwm@mired.org> http://www.mired.org/consulting.html Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.