Postgres Pro
Downloads
Home   >   mailing lists

Thread: pg_test_fsync problem

pg_test_fsync problem

From
Vladimir Kokovic
Date:
Hi,

Something is wrong with contrib/pg_test_fsync on Ubuntu 10.10 maverick.

I've changed method:
static void
die(char *str)
{
fprintf(stderr, "%s errno=%d(%s)\n", str, errno, strerror(errno));
exit(1);
}

===
root@vlD-kuci:/media/sda5/postgresql-9.1devel# ./build.sh
set -e
cd /media/sda5/postgresql-9.1devel/build
export CFLAGS="-g3 -gdwarf-2"
../../tmp/postgresql-git/postgresql/configure '--srcdir=../../tmp/postgresql-git/postgresql' '--enable-cassert' '--enable-nls' '--enable-integer-datetimes' '--with-perl' '--with-python' '--with-tcl' '--with-krb5' '--with-openssl' '--enable-thread-safety' '--with-ldap' '--prefix=/media/sda5/postgresql-9.1devel/20110122' > configure-out1.log 2>&1
make world > make-out1.log 2>&1
make install-world > make-install-out1.log 2>&1
exit 0

===
root@vlD-kuci:/media/sda5/postgresql-9.1devel# ldd -d -r /media/sda5/postgresql-9.1devel/20110122/bin/pg_test_fsync
linux-gate.so.1 => (0x0023d000)
libc.so.6 => /lib/libc.so.6 (0x00d75000)
/lib/ld-linux.so.2 (0x00603000)

===
root@vlD-kuci:/media/sda5/postgresql-9.1devel# cd /tmp

===
root@vlD-kuci:/tmp# /media/sda5/postgresql-9.1devel/20110122/bin/pg_test_fsync
2000 operations per test

Compare file sync methods using one 8k write:
(in wal_sync_method preference order, except fdatasync
is Linux's default)
open_datasync (non-direct I/O)* 1882.670 ops/sec
open_datasync (direct I/O) 2020.857 ops/sec
fdatasync 1920.381 ops/sec
fsync 1877.340 ops/sec
fsync_writethrough n/a
open_sync (non-direct I/O)* 1835.053 ops/sec
open_sync (direct I/O) 1995.341 ops/sec
* This non-direct I/O mode is not used by Postgres.

Compare file sync methods using two 8k writes:
(in wal_sync_method preference order, except fdatasync
is Linux's default)
open_datasync (non-direct I/O)* 1056.251 ops/sec
open_datasync (direct I/O) 1058.650 ops/sec
fdatasync 1220.119 ops/sec
fsync 1214.043 ops/sec
fsync_writethrough n/a
open_sync (non-direct I/O)* 1035.295 ops/sec
open_sync (direct I/O) 1034.949 ops/sec
* This non-direct I/O mode is not used by Postgres.

Compare open_sync with different write sizes:
(This is designed to compare the cost of writing 16k
in different write open_sync sizes.)
1 16k open_sync write write failed errno=22(Invalid argument)

===
root@vlD-kuci:/tmp# ls -la pg*
-rw------- 1 root root 16777216 2011-01-22 08:20 pg_test_fsync.out

===
root@vlD-kuci:/tmp# uname -a
Linux vlD-kuci 2.6.35-25-generic #43-Ubuntu SMP Thu Jan 6 22:25:16 UTC 2011 i686 GNU/Linux
root@vlD-kuci:/tmp#


Best regards,
Vladimir Kokovic, DP senior, Belgrade, Serbia



Re: pg_test_fsync problem

From
Robert Haas
Date:
On Sat, Jan 22, 2011 at 2:46 AM, Vladimir Kokovic
<vladimir.kokovic@gmail.com> wrote:
> Something is wrong with contrib/pg_test_fsync on Ubuntu 10.10 maverick.

What, specifically, is wrong?

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Re: pg_test_fsync problem

From
Tom Lane
Date:
Robert Haas <robertmhaas@gmail.com> writes:
> On Sat, Jan 22, 2011 at 2:46 AM, Vladimir Kokovic
> <vladimir.kokovic@gmail.com> wrote:
>> Something is wrong with contrib/pg_test_fsync on Ubuntu 10.10 maverick.

> What, specifically, is wrong?

He's complaining that it dies with EINVAL.

I notice that (1) it's using O_DIRECT even though the printout claims
otherwise, and (2) it's writing from a buffer that has no better than
char alignment, which is certainly not OK for O_DIRECT.  Either one
of those could plausibly result in EINVAL ...
        regards, tom lane

Re: pg_test_fsync problem

From
Tom Lane
Date:
I wrote:
> He's complaining that it dies with EINVAL.

> I notice that (1) it's using O_DIRECT even though the printout claims
> otherwise, and (2) it's writing from a buffer that has no better than
> char alignment, which is certainly not OK for O_DIRECT.  Either one
> of those could plausibly result in EINVAL ...

Oh, scratch that: the buffer is properly aligned, it's the length that's
bogus for O_DIRECT.  I rather imagine that test_open_sync is meant to be
writing so many kilobytes, not so many bytes.
        regards, tom lane

Re: pg_test_fsync problem

From
Bruce Momjian
Date:
Tom Lane wrote:
> I wrote:
> > He's complaining that it dies with EINVAL.
>
> > I notice that (1) it's using O_DIRECT even though the printout claims
> > otherwise, and (2) it's writing from a buffer that has no better than
> > char alignment, which is certainly not OK for O_DIRECT.  Either one
> > of those could plausibly result in EINVAL ...
>
> Oh, scratch that: the buffer is properly aligned, it's the length that's
> bogus for O_DIRECT.  I rather imagine that test_open_sync is meant to be
> writing so many kilobytes, not so many bytes.

Yes, that was a bug in my code that I have fixed with the attached,
applied patch.  Thanks for the report.

--
  Bruce Momjian  <bruce@momjian.us>        http://momjian.us
  EnterpriseDB                             http://enterprisedb.com

  + It's impossible for everything to be true. +
diff --git a/contrib/pg_test_fsync/pg_test_fsync.c b/contrib/pg_test_fsync/pg_test_fsync.c
index 7ece9b9..d8099a5 100644
*** /tmp/pgdiff.10514/8NsZJd_pg_test_fsync.c    Mon Jan 24 19:41:20 2011
--- contrib/pg_test_fsync/pg_test_fsync.c    Mon Jan 24 19:39:48 2011
*************** test_open_sync(const char *msg, int writ
*** 421,427 ****
          for (ops = 0; ops < ops_per_test; ops++)
          {
              for (writes = 0; writes < 16 / writes_size; writes++)
!                 if (write(tmpfile, buf, writes_size) != writes_size)
                      die("write failed");
              if (lseek(tmpfile, 0, SEEK_SET) == -1)
                  die("seek failed");
--- 421,428 ----
          for (ops = 0; ops < ops_per_test; ops++)
          {
              for (writes = 0; writes < 16 / writes_size; writes++)
!                 if (write(tmpfile, buf, writes_size * 1024) !=
!                     writes_size * 1024)
                      die("write failed");
              if (lseek(tmpfile, 0, SEEK_SET) == -1)
                  die("seek failed");