Thread: O_DIRECT and performance
Well, O_DIRECT has finally made it into the Linux kernel. It lets you open a file in such a way that reads and writes don't go to the buffer cache but straight to the disk. Accesses must be aligned on filesystem block boundaries. Is there any case where PG would benefit from this? I can see it reducing memory pressure and duplication of data between PG shared buffers and the block buffer cache. OTOH, it does require that writes be batched up for decent performance, since each write has an implicit fsync() involved (just as with O_SYNC). Anyone played with this on systems that already support it (Solaris?) -Doug -- In a world of steel-eyed death, and men who are fighting to be warm, Come in, she said, I'll give you shelter from the storm. -Dylan
> Well, O_DIRECT has finally made it into the Linux kernel. It lets you > open a file in such a way that reads and writes don't go to the buffer > cache but straight to the disk. Accesses must be aligned on > filesystem block boundaries. > > Is there any case where PG would benefit from this? I can see it > reducing memory pressure and duplication of data between PG shared > buffers and the block buffer cache. OTOH, it does require that writes > be batched up for decent performance, since each write has an implicit > fsync() involved (just as with O_SYNC). > > Anyone played with this on systems that already support it (Solaris?) I have heard there are many cases there O_DIRECT on Solaris is slower for databases than normal I/O. I think bulk copy was faster but not normal operation. Probably not something we are going to get into soon. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 853-3000+ If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup. | Drexel Hill, Pennsylvania19026
Red Hat Linux 7.2, postgresql 7.2b2 configured like this: ./configure --enable-syslog --prefix=/usr --sysconfdir=/etc --with-pam --with-openssl --with-unixodbc --with-tcl --with-perl --with-python --enable-locale --enable-recode --enable-multibyte --enable-nls "make check" fails on initdb: ============== creating temporary installation ============== ============== initializing database system ============== pg_regress: initdb failed Examine ./log/initdb.log for the reason. teg@halden postgresql-7.2b2]$ cat src/test/regress/log/initdb.log Running with noclean mode on. Mistakes will not be cleaned up. /home/devel/teg/postgresql-7.2b2/src/test/regress/./tmp_check/install//usr/bin/pg_encoding: relocation error: /home/devel/teg/postgresql-7.2b2/src/test/regress/./tmp_check/install//usr/bin/pg_encoding:undefined symbol: pg_valid_server_encoding initdb: pg_encoding failed Perhaps you did not configure PostgreSQL for multibyte support or the program was not successfully installed. [teg@halden postgresql-7.2b2]$ -- Trond Eivind Glomsrød Red Hat, Inc.
Re: 7.2b2 "make check" failure on Red Hat Linux 7.2
From
teg@redhat.com (Trond Eivind Glomsrød)
Date:
teg@redhat.com (Trond Eivind Glomsrød) writes: > Red Hat Linux 7.2, postgresql 7.2b2 configured like this: > > ./configure --enable-syslog --prefix=/usr --sysconfdir=/etc --with-pam > --with-openssl --with-unixodbc --with-tcl --with-perl --with-python > --enable-locale --enable-recode --enable-multibyte --enable-nls When just using "configure" it works - apart from a false failure in the regression tests (geometry), it seems to work (a couple cases of "-0.121320343559642" vs. "-0.121320343559643" - a better way than raw diff for comparing would be useful). -- Trond Eivind Glomsrød Red Hat, Inc.
teg@redhat.com (Trond Eivind Glomsrød) writes:
> /home/devel/teg/postgresql-7.2b2/src/test/regress/./tmp_check/install//usr/bin/pg_encoding: relocation error:
/home/devel/teg/postgresql-7.2b2/src/test/regress/./tmp_check/install//usr/bin/pg_encoding:undefined symbol:
pg_valid_server_encoding
> initdb: pg_encoding failed
pg_encoding relies on libpq to supply the pg_valid_server_encoding()
subroutine, but that subroutine is only compiled into libpq in a
MULTIBYTE build. I speculate that your executable was picking up
a non-MULTIBYTE libpq shared library from someplace. Check ldconfig
and all that stuff...
FWIW, both multibyte and non-multibyte builds are working OK for me
in current sources.
regards, tom lane
Re: 7.2b2 "make check" failure on Red Hat Linux 7.2
From
teg@redhat.com (Trond Eivind Glomsrød)
Date:
Tom Lane <tgl@sss.pgh.pa.us> writes: > teg@redhat.com (Trond Eivind Glomsrød) writes: > > /home/devel/teg/postgresql-7.2b2/src/test/regress/./tmp_check/install//usr/bin/pg_encoding: relocation error: /home/devel/teg/postgresql-7.2b2/src/test/regress/./tmp_check/install//usr/bin/pg_encoding:undefined symbol: pg_valid_server_encoding > > initdb: pg_encoding failed > > pg_encoding relies on libpq to supply the pg_valid_server_encoding() > subroutine, but that subroutine is only compiled into libpq in a > MULTIBYTE build. I speculate that your executable was picking up > a non-MULTIBYTE libpq shared library from someplace. Check ldconfig > and all that stuff... I have an existing installation of 7.1 on the system, that's why I did "make check" in the build directory. "--prefix=/usr" seems to be the "culprit" - without it, it regression tests run just fine. -- Trond Eivind Glomsrød Red Hat, Inc.
Re: 7.2b2 "make check" failure on Red Hat Linux 7.2
From
teg@redhat.com (Trond Eivind Glomsrød)
Date:
Tom Lane <tgl@sss.pgh.pa.us> writes: > teg@redhat.com (Trond Eivind Glomsrød) writes: > > Tom Lane <tgl@sss.pgh.pa.us> writes: > >> I speculate that your executable was picking up > >> a non-MULTIBYTE libpq shared library from someplace. Check ldconfig > >> and all that stuff... > > > I have an existing installation of 7.1 on the system, that's why I did > > "make check" in the build directory. > > > "--prefix=/usr" seems to be the "culprit" - without it, it regression > > tests run just fine. > > The pg_regress script sets LD_LIBRARY_PATH to try to cause libpq and > the other shlibs to be picked up from the temp installation tree. > Perhaps this is wrong or insufficient on your platform. It certainly > sounds like the dynamic linker is choosing the installed libpq over > the one that we want it to use. Any thoughts on fixing that? Since it works when prefix isn't /usr, I'd guess that the build process sets rpath. This takes precedence over LD_LIBRARY_PATH. Fix? Don't use rpath - it's evil, and should be avoided anyway. ld(1) contains some info on how libraries are resolved. -- Trond Eivind Glomsrød Red Hat, Inc.
teg@redhat.com (Trond Eivind Glomsrød) writes:
> Tom Lane <tgl@sss.pgh.pa.us> writes:
>> I speculate that your executable was picking up
>> a non-MULTIBYTE libpq shared library from someplace. Check ldconfig
>> and all that stuff...
> I have an existing installation of 7.1 on the system, that's why I did
> "make check" in the build directory.
> "--prefix=/usr" seems to be the "culprit" - without it, it regression
> tests run just fine.
The pg_regress script sets LD_LIBRARY_PATH to try to cause libpq and
the other shlibs to be picked up from the temp installation tree.
Perhaps this is wrong or insufficient on your platform. It certainly
sounds like the dynamic linker is choosing the installed libpq over
the one that we want it to use. Any thoughts on fixing that?
regards, tom lane
teg@redhat.com (Trond Eivind Glomsrød) writes:
> Since it works when prefix isn't /usr, I'd guess that the build
> process sets rpath. This takes precedence over LD_LIBRARY_PATH.
> Fix? Don't use rpath - it's evil, and should be avoided anyway.
Peter, any thoughts on that? It's not clear to me that removing
rpath settings would be an improvement overall, even if it makes
"make check" more reliable.
regards, tom lane
* Tom Lane <tgl@sss.pgh.pa.us> [011116 16:56]: > teg@redhat.com (Trond Eivind Glomsrød) writes: > > Since it works when prefix isn't /usr, I'd guess that the build > > process sets rpath. This takes precedence over LD_LIBRARY_PATH. > > > Fix? Don't use rpath - it's evil, and should be avoided anyway. > > Peter, any thoughts on that? It's not clear to me that removing > rpath settings would be an improvement overall, even if it makes > "make check" more reliable. My general take is rpath is a *GOOD* thing, as it prevents USERS from not being able to find REQUIRED shared libs. I vote for it staying. LER > > regards, tom lane > > ---------------------------(end of broadcast)--------------------------- > TIP 6: Have you searched our list archives? > > http://archives.postgresql.org -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 972-414-9812 E-Mail: ler@lerctr.org US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749
Trond Eivind Glomsrød writes: > Fix? Don't use rpath - it's evil, and should be avoided anyway. configure --disable-rpath is your friend. Overall, rpath gives us much more benefit than the occasional trouble it creates. -- Peter Eisentraut peter_e@gmx.net
On Sun, 18 Nov 2001, Peter Eisentraut wrote: > Trond Eivind Glomsrød writes: > > > Fix? Don't use rpath - it's evil, and should be avoided anyway. > > configure --disable-rpath is your friend. Didn't know about that one - it will certainly be put to use :) > Overall, rpath gives us much more benefit than the occasional trouble it > creates. When you install (not configure for, just install) into a separate tree (for easier packaging), it's a hole which can be exploited - some packages will rpath into /var/tmp/<foo>, for instance. Hackers can then put their own library there. One big offender here is perl's automatic module creation script which will change the rpaths from what you told it when you built it to what you do when you install it. -- Trond Eivind Glomsrød Red Hat, Inc.
Trond Eivind Glomsrød writes: > When you install (not configure for, just install) into a separate tree > (for easier packaging), it's a hole which can be exploited - some packages > will rpath into /var/tmp/<foo>, for instance. Hackers can then put their > own library there. "Some packages"... ;-) > One big offender here is perl's automatic module > creation script which will change the rpaths from what you told it when > you built it to what you do when you install it. This should be fixed now, although the perl module will actually not obey the --disable-rpath switch. Can't have everything, I guess... -- Peter Eisentraut peter_e@gmx.net
Re: 7.2b2 "make check" failure on Red Hat Linux 7.2
From
teg@redhat.com (Trond Eivind Glomsrød)
Date:
Larry Rosenman <ler@lerctr.org> writes: > * Tom Lane <tgl@sss.pgh.pa.us> [011116 16:56]: > > teg@redhat.com (Trond Eivind Glomsrød) writes: > > > Since it works when prefix isn't /usr, I'd guess that the build > > > process sets rpath. This takes precedence over LD_LIBRARY_PATH. > > > > > Fix? Don't use rpath - it's evil, and should be avoided anyway. > > > > Peter, any thoughts on that? It's not clear to me that removing > > rpath settings would be an improvement overall, even if it makes > > "make check" more reliable. > My general take is rpath is a *GOOD* thing, as it prevents USERS from > not being able to find REQUIRED shared libs. Lib-directories should be listed in /etc/ld.so.conf, and no rpath used - at least on linux. -- Trond Eivind Glomsrød Red Hat, Inc.
* Trond Eivind Glomsr?d <teg@redhat.com> [011119 10:47]: > Larry Rosenman <ler@lerctr.org> writes: > > > * Tom Lane <tgl@sss.pgh.pa.us> [011116 16:56]: > > > teg@redhat.com (Trond Eivind Glomsrød) writes: > > > > Since it works when prefix isn't /usr, I'd guess that the build > > > > process sets rpath. This takes precedence over LD_LIBRARY_PATH. > > > > > > > Fix? Don't use rpath - it's evil, and should be avoided anyway. > > > > > > Peter, any thoughts on that? It's not clear to me that removing > > > rpath settings would be an improvement overall, even if it makes > > > "make check" more reliable. > > My general take is rpath is a *GOOD* thing, as it prevents USERS from > > not being able to find REQUIRED shared libs. > > Lib-directories should be listed in /etc/ld.so.conf, and no rpath used > - at least on linux. no such animal on OpenUNIX 8 (In native "unix" mode). > > -- > Trond Eivind Glomsrød > Red Hat, Inc. -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 972-414-9812 E-Mail: ler@lerctr.org US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749