Thread: LD_LIBRARY_PATH versus rpath
Over at http://archives.postgresql.org/pgsql-general/2010-05/msg00091.php we have a complaint about "make check" failing when the install is intended to overwrite existing libraries (in particular, replacing 8.4 with 9.0 libpq). I've done some off-list investigation and found that this appears to be a generic issue on Linux. pg_regress invokes psql, which depends on libpq.so, and if psql fails due to picking up the wrong libpq.so then you get behavior as described. Now, pg_regress tries to ensure that the temporary installation will work as desired by setting LD_LIBRARY_PATH to point at the temp installation's lib/ directory. However, the psql executable will by default get built with a DT_RPATH entry pointing at the intended final installation lib/. And DT_RPATH overrides LD_LIBRARY_PATH, in the Linux dynamic loader. man ld.so says: The shared libraries needed by the program are searched for in the fol- lowing order: o (ELF only) Using the directories specified in the DT_RPATH dynamic section attribute of the binary ifpresent and DT_RUNPATH attribute does not exist. Use of DT_RPATH is deprecated. o Using the environment variable LD_LIBRARY_PATH. Except if the exe- cutable is a set-user-ID/set-group-IDbinary, in which case it is ignored. o (ELF only) Using the directories specified in the DT_RUNPATH dynamic section attribute of the binary ifpresent. o (etc etc) Given that deprecation note, and the fact that what we're doing entirely fails to work as desired, it seems like what we need to do is set DT_RUNPATH instead of DT_RPATH. Further reading discloses that the way to do that is to add "--enable-new-dtags" to the linker switches. So the question is, should we modify Makefile.linux along the lines of -rpath = -Wl,-rpath,'$(rpathdir)' +rpath = -Wl,-rpath,'$(rpathdir)',--enable-new-dtags I asked around at Red Hat and was told that this would be unlikely to have any negative side-effects, but I'm not sure how thoroughly those guys thought about the consequences for non-mainstream Linux machines. (In particular, I'm worried about really old distros possibly not having this switch.) My inclination is to try this in HEAD only and see if any problems emerge during the beta cycle. Comments? regards, tom lane
On Thu, May 6, 2010 at 12:20 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote: > Now, pg_regress tries to ensure that the temporary installation > will work as desired by setting LD_LIBRARY_PATH to point at the > temp installation's lib/ directory. However, the psql executable > will by default get built with a DT_RPATH entry pointing at the > intended final installation lib/. And DT_RPATH overrides > LD_LIBRARY_PATH, in the Linux dynamic loader. man ld.so says: > We only set RPATH if the install location isn't part of the default ld library path specified by /etc/ld.so.conf right? Setting it if it is in the default path would be antisocial. -- greg
On ons, 2010-05-05 at 19:20 -0400, Tom Lane wrote: > Over at > http://archives.postgresql.org/pgsql-general/2010-05/msg00091.php > we have a complaint about "make check" failing when the install is > intended to overwrite existing libraries (in particular, replacing > 8.4 with 9.0 libpq). I've done some off-list investigation and > found that this appears to be a generic issue on Linux. pg_regress > invokes psql, which depends on libpq.so, and if psql fails due to > picking up the wrong libpq.so then you get behavior as described. Yeah, that's been broken since forever. > The shared libraries needed by the program are searched for in the fol- > lowing order: > > o (ELF only) Using the directories specified in the DT_RPATH dynamic > section attribute of the binary if present and DT_RUNPATH attribute > does not exist. Use of DT_RPATH is deprecated. > > o Using the environment variable LD_LIBRARY_PATH. Except if the exe- > cutable is a set-user-ID/set-group-ID binary, in which case it is > ignored. > > o (ELF only) Using the directories specified in the DT_RUNPATH dynamic > section attribute of the binary if present. Ah, that sounds good. > So the question is, should we modify Makefile.linux along the lines of > > -rpath = -Wl,-rpath,'$(rpathdir)' > +rpath = -Wl,-rpath,'$(rpathdir)',--enable-new-dtags I see this feature was added in 2001, so it should be OK to use. > My inclination is to try this in HEAD only and see if any problems > emerge during the beta cycle. I wouldn't consider backpatching it at all.
On tor, 2010-05-06 at 11:20 +0100, Greg Stark wrote: > We only set RPATH if the install location isn't part of the default ld > library path specified by /etc/ld.so.conf right? No. How would you determine that? > Setting it if it is > in the default path would be antisocial. That's why there is --disable-rpath.
Greg Stark wrote > > We only set RPATH if the install location isn't part of the default ld > library path specified by /etc/ld.so.conf right? Setting it if it is > in the default path would be antisocial. > > How are we going to know at build time what is in the ld.soconf of the installation machine? cheers andrew
Andrew Dunstan <andrew@dunslane.net> writes: > Greg Stark wrote >> We only set RPATH if the install location isn't part of the default ld >> library path specified by /etc/ld.so.conf right? Setting it if it is >> in the default path would be antisocial. > How are we going to know at build time what is in the ld.soconf of the > installation machine? Exactly. In practice, it's on the packager's head to specify --disable-rpath if he intends to install into the platform's regular library search path. Funny point here: in the Fedora/RHEL RPMs, I use --disable-rpath because "don't use RPATH" is part of the standard packaging guidelines for that platform. However, pl/perl has to double back and use rpath anyway because libperl.so doesn't exist in the ldconfig path; it's in some version-numbered directory and they don't provide any link or ldconfig entry so you could find it otherwise. Annoying as heck. I've always wondered how many other packagers have to carry patches similar to http://cvs.fedoraproject.org/viewvc/rpms/postgresql/devel/postgresql-perl-rpath.patch regards, tom lane
On tor, 2010-05-06 at 09:38 -0400, Tom Lane wrote: > Funny point here: in the Fedora/RHEL RPMs, I use --disable-rpath > because "don't use RPATH" is part of the standard packaging guidelines > for that platform. However, pl/perl has to double back and use rpath > anyway because libperl.so doesn't exist in the ldconfig path; it's in > some version-numbered directory and they don't provide any link or > ldconfig entry so you could find it otherwise. Annoying as heck. > I've always wondered how many other packagers have to carry patches > similar to > http://cvs.fedoraproject.org/viewvc/rpms/postgresql/devel/postgresql-perl-rpath.patch Debian has libperl in /usr/lib, so there is no issue. But if there were, there is a relatively new policy that you can should use rpath if you need a library that is installed in a nonstandard path. (Should actually use this new runpath thing, perhaps.) The same new policy prohibits packages from modifying /etc/ld.so.conf.