Thread: Sun performance - Major discovery!

Sun performance - Major discovery!

From
Jeff
Date:
Well, as you guys know I've been tinkering with sun-vs-linux postgres for
a while trying to come up with reasons for the HUGE performance
differences. We've all had our anecdotal thoughts (fork sucks, ipc sucks,
ufs sucks, etc) and I've had a breakthrough.

Knowing that GCC only produces good code on x86 (and powerpc with apple's
mods, but it is doubtful that is as good as ibm's power compiler) I
decided to try out Sunsoft CC.  I'd heard from more than one person/place
that gcc makes abysmal sparc code.  Given that the performance profiles
for both the linux and sun boxes showed the same functions taking up most
of the time I thought I'd see what a difference sunsoft could give me.

So - hardware -
Sun E450 4x400mhz ultrasparc IIi, 4GB ram, scsi soemthing disk. (not
raid) solaris 2.6

Linux - 2xP3 500mhz, 2GB, scsi disk of some flavor (not raid) linux 2.2.17
(old I know!)

So here's the results using my load tester (single connection per beater,
repeats the query 1000 times with different input each time (we'll get
~20k rows back), the query is a common query around here.

I discounted the first run of the test as caches populated.

Linux - 1x - 35 seconds, 20x - 180 seconds

Sun - gcc - 1x 60 seconds  20x 245 seconds
Sun - sunsoft defaults - 1x 52 seonds 20x [similar to gcc most likely]
Sun - sunsoft -fast  - 1x 28 seconds  20x 164 seconds

As you math guru's can probably deduce - that is a rather large
improvement.  And by rather large I mean hugely significant.  With results
like this, I think it warrants mentioning in the FAQ_Solaris, and probably
the performance guide.

Connecting will always be a bit slower. But I think most people realize
that connecting to a db is not cheap.

I think update/etc will cause more locking, but I think IO will become the
bottle neck much sooner than lock/unlock will. (This is mostly anecdotal
given how fast solaris can lock/unlock a semaphore and how much IO I know
I have)

Oh yes, with was with 7.3.4 and sunsoft cc Sun WorkShop 6 update 1 C
5.2 2000/09/11 (which is old, perhaps newer ones make even better code?)

I'm not sure of PG's policy of non-gcc things in configure, but perhaps if
we detect sunsoft we toss in the -fast flag and maybe make it the
preferred one on sun? [btw, it compiled with no changes but it did spew
out tons of warnings]

comments?

--
Jeff Trout <jeff@jefftrout.com>
http://www.jefftrout.com/
http://www.stuarthamm.net/



Re: Sun performance - Major discovery!

From
Andrew Sullivan
Date:
On Wed, Oct 08, 2003 at 08:36:56AM -0400, Jeff wrote:
>
> So here's the results using my load tester (single connection per beater,
> repeats the query 1000 times with different input each time (we'll get
> ~20k rows back), the query is a common query around here.

My worry about this test is that it gives us precious little
knowledge about concurrent connection slowness, which is where I find
the most significant problems.  When we tried a Sunsoft cc vs gcc 2.95
on Sol 7 about 1 1/2 years ago, we found more or less no difference
once we added more than 5 connections (and we always have more than 5
connections).  It might be worth trying again, though, since we moved
to Sol 8.

Thanks for the result.

--
----
Andrew Sullivan                         204-4141 Yonge Street
Afilias Canada                        Toronto, Ontario Canada
<andrew@libertyrms.info>                              M2P 2A8
                                         +1 416 646 3304 x110


Re: Sun performance - Major discovery!

From
Neil Conway
Date:
On Wed, 2003-10-08 at 08:36, Jeff wrote:
> So here's the results using my load tester (single connection per beater,
> repeats the query 1000 times with different input each time (we'll get
> ~20k rows back), the query is a common query around here.

What is the query?

> Linux - 1x - 35 seconds, 20x - 180 seconds

"20x" means 20 concurrent testing processes, right?

> Sun - gcc - 1x 60 seconds  20x 245 seconds
> Sun - sunsoft defaults - 1x 52 seonds 20x [similar to gcc most likely]
> Sun - sunsoft -fast  - 1x 28 seconds  20x 164 seconds

Interesting (and surprising that the performance differential is that
large, to me at least). Can you tell if the performance gain comes from
an improvement in a particular subsystem? (i.e. could you get a profile
of Sun/gcc and compare it with Sun/sunsoft).

-Neil



Re: Sun performance - Major discovery!

From
Jeff
Date:
On Wed, 8 Oct 2003, Andrew Sullivan wrote:

> My worry about this test is that it gives us precious little
> knowledge about concurrent connection slowness, which is where I find
> the most significant problems.  When we tried a Sunsoft cc vs gcc 2.95
> on Sol 7 about 1 1/2 years ago, we found more or less no difference
> once we added more than 5 connections (and we always have more than 5
> connections).  It might be worth trying again, though, since we moved
> to Sol 8.
>

The 20x column are the results when I fired up 20 beater concurrently.


--
Jeff Trout <jeff@jefftrout.com>
http://www.jefftrout.com/
http://www.stuarthamm.net/



Re: Sun performance - Major discovery!

From
Jeff
Date:
On Wed, 8 Oct 2003, Neil Conway wrote:

> What is the query?
>

It retrieves an index listing for our boards. The boards are flat (not
threaded) and messages are numbered starting at 1 for each board.

If you pass in 0 for the start_from it assumes the latest 60.

And it should be noted - in some cases some boards have nearly 2M posts.
Index on board_name, number.

I cannot give out too too much stuff ;)

create or replace function get_index2(integer, varchar, varchar)
    returns setof snippet
    as '
DECLARE
    p_start alias for $1;
    p_board alias for $2;
    v_start integer;
    v_num integer;
    v_body text;
    v_sender varchar(35);
    v_time timestamptz;
    v_finish integer;
    v_row record;
    v_ret snippet;
BEGIN

    v_start := p_start;

    if v_start = 0 then
        select * into v_start from get_high_msg(p_board);
        v_start := v_start - 59;
    end if;

    v_finish := v_start + 60;

    for v_row in
        select number, substr(body, 0, 50) as snip, member_handle,
timestamp
            from posts
            where board_name = p_board and
            number >= v_start and
            number < v_finish
            order by number desc
    LOOP
        return next v_row;
    END LOOP;

    return;
END;
' language 'plpgsql';


> Interesting (and surprising that the performance differential is that
> large, to me at least). Can you tell if the performance gain comes from
> an improvement in a particular subsystem? (i.e. could you get a profile
> of Sun/gcc and compare it with Sun/sunsoft).
>

I'll get these later today.

--
Jeff Trout <jeff@jefftrout.com>
http://www.jefftrout.com/
http://www.stuarthamm.net/



Re: Sun performance - Major discovery!

From
Jeff
Date:
On Wed, 8 Oct 2003, Neil Conway wrote:

> Interesting (and surprising that the performance differential is that
> large, to me at least). Can you tell if the performance gain comes from
> an improvement in a particular subsystem? (i.e. could you get a profile
> of Sun/gcc and compare it with Sun/sunsoft).
>

Yeah - like I expected it was able to generate much better code for
_bt_checkkeys which was the #1 function in gcc on both sun & linux.

and as you can see, suncc was just able to generate much nicer code. I'd
look at the assembler output but that won't be useful since I am very
unfamiliar with the [ultra]sparc instruction set..


Here's the prof and gprof output for the latest run:
GCC:
  %   cumulative   self              self     total
 time   seconds   seconds    calls  ms/call  ms/call  name
 31.52     19.44    19.44                             internal_mcount
 20.28     31.95    12.51  8199466     0.00     0.00  _bt_checkkeys
  5.61     35.41     3.46  8197422     0.00     0.00  _bt_step
  5.01     38.50     3.09 24738620     0.00     0.00  FunctionCall2
  3.00     40.35     1.85  8194186     0.00     0.00  varchareq
  2.61     41.96     1.61    24309     0.07     0.28  _bt_next
  2.42     43.45     1.49     1003     1.49     1.51  AtEOXact_Buffers
  2.37     44.91     1.46    12642     0.12     0.12  _read
  2.33     46.35     1.44 16517771     0.00     0.00  pg_detoast_datum
  2.08     47.63     1.28  8193186     0.00     0.00  int4lt
  1.35     48.46     0.83  8237204     0.00     0.00  BufferGetBlockNumber
  1.35     49.29     0.83  8193888     0.00     0.00  int4ge
  1.35     50.12     0.83                             _mcount


SunCC -pg -fast.
 %Time Seconds Cumsecs  #Calls   msec/call  Name

  23.2    4.27    4.27108922056      0.0000  _mcount
  20.7    3.82    8.09 8304052      0.0005  _bt_checkkeys
  13.7    2.53   10.6225054788      0.0001  FunctionCall2
   5.1    0.94   11.56   24002      0.0392  _bt_next
   4.4    0.81   12.37 8301867      0.0001  _bt_step
   3.4    0.63   13.00 8298219      0.0001  varchareq
   2.7    0.50   13.5016726855      0.0000  pg_detoast_datum
   2.4    0.45   13.95 8342464      0.0001  BufferGetBlockNumber
   2.4    0.44   14.39 8297941      0.0001  int4ge
   2.2    0.41   14.80    1003      0.409   AtEOXact_Buffers
   2.0    0.37   15.17 4220349      0.0001  lc_collate_is_c
   2.0    0.37   15.54 8297219      0.0000  int4lt
   1.6    0.29   15.83   26537      0.0109  AllocSetContextCreate
   0.9    0.16   15.99    1887      0.085   pglz_decompress
   0.7    0.13   16.12  159966      0.0008  nocachegetattr
   0.7    0.13   16.25 4220349      0.0000  varstr_cmp
   0.6    0.11   16.36  937576      0.0001  MemoryContextAlloc
   0.5    0.09   16.45  150453      0.0006  hash_search





> -Neil
>
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org
>
>

--
Jeff Trout <jeff@jefftrout.com>
http://www.jefftrout.com/
http://www.stuarthamm.net/



Re: Sun performance - Major discovery!

From
Neil Conway
Date:
On Wed, 2003-10-08 at 10:48, Andrew Sullivan wrote:
> My worry about this test is that it gives us precious little
> knowledge about concurrent connection slowness, which is where I find
> the most significant problems.

As Jeff points out, the second set of results is for 20 concurrent
connections. Note that the advantage sunsoft cc has over gcc decreases
as the number of connections increases (which makes sense, as the 20x
workload is likely to be more I/O bound).

-Neil



Re: Sun performance - Major discovery!

From
Neil Conway
Date:
On Wed, 2003-10-08 at 11:46, Jeff wrote:
> Yeah - like I expected it was able to generate much better code for
> _bt_checkkeys which was the #1 function in gcc on both sun & linux.
>
> and as you can see, suncc was just able to generate much nicer code.

What CFLAGS does configure pick for gcc? From
src/backend/template/solaris, I'd guess it's not enabling any
optimization. Is that the case? If so, some gcc numbers with -O and -O2
would be useful.

-Neil



Re: Sun performance - Major discovery!

From
Jeff
Date:
On Wed, 8 Oct 2003, Neil Conway wrote:

>
> What CFLAGS does configure pick for gcc? From
> src/backend/template/solaris, I'd guess it's not enabling any
> optimization. Is that the case? If so, some gcc numbers with -O and -O2
> would be useful.
>

I can't believe I didn't think of this before! heh.
Turns out gcc was getting nothing for flags.

I added -O2 to CFLAGS and my 60 seconds went down to 21.  A rather mild
improvment huh?

I did a few more tests and suncc still beats it out - but not by too much
now (Not enought to justify buying a license just for compiling pg)

I'll go run the regression test suite with my gcc -O2 pg and the suncc pg.
See if they pass the test.

If they do we should consider adding -O2 and -fast to the CFLAGS.

--
Jeff Trout <jeff@jefftrout.com>
http://www.jefftrout.com/
http://www.stuarthamm.net/



Re: Sun performance - Major discovery!

From
Bruce Momjian
Date:
Jeff wrote:
> On Wed, 8 Oct 2003, Neil Conway wrote:
>
> >
> > What CFLAGS does configure pick for gcc? From
> > src/backend/template/solaris, I'd guess it's not enabling any
> > optimization. Is that the case? If so, some gcc numbers with -O and -O2
> > would be useful.
> >
>
> I can't believe I didn't think of this before! heh.
> Turns out gcc was getting nothing for flags.
>
> I added -O2 to CFLAGS and my 60 seconds went down to 21.  A rather mild
> improvment huh?
>
> I did a few more tests and suncc still beats it out - but not by too much
> now (Not enought to justify buying a license just for compiling pg)
>
> I'll go run the regression test suite with my gcc -O2 pg and the suncc pg.
> See if they pass the test.
>
> If they do we should consider adding -O2 and -fast to the CFLAGS.

[ CC added for hackers.]

Well, this is really embarassing.  I can't imagine why we would not set
at least -O on all platforms.  Looking at the template files, I see
these have no optimization set:

    darwin
    dgux
    freebsd (non-alpha)
    irix5
    nextstep
    osf (gcc)
    qnx4
    solaris
    sunos4
    svr4
    ultrix4

I thought we used to have code that did -O for any platforms that set no
cflags, but I don't see that around anywhere.  I recommend adding -O2,
or at leaset -O to all these platforms --- we can then use platform
testing to make sure they are working.

--
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Re: Sun performance - Major discovery!

From
Neil Conway
Date:
On Wed, 2003-10-08 at 14:31, Bruce Momjian wrote:
> Well, this is really embarassing.  I can't imagine why we would not set
> at least -O on all platforms.

ISTM the most legitimate reason for not enabling compilater
optimizations on a given compiler/OS/architecture combination is might
cause compiler errors / bad code generation.

Can we get these optimizations enabled in time for the next 7.4 beta? It
might also be good to add an item in the release notes about it.

-Neil



Re: [HACKERS] Sun performance - Major discovery!

From
Jeff
Date:
On Wed, 8 Oct 2003, Neil Conway wrote:

> ISTM the most legitimate reason for not enabling compilater
> optimizations on a given compiler/OS/architecture combination is might
> cause compiler errors / bad code generation.
>
> Can we get these optimizations enabled in time for the next 7.4 beta? It
> might also be good to add an item in the release notes about it.
>
> -Neil
>

I just ran make check for sun with gcc -O2 and suncc -fast and both
passed.

We'll need other arguments to suncc to supress some warnings, etc. (-fast
generates a warning for every file compiled telling you it will only
run on ultrasparc machines)


--
Jeff Trout <jeff@jefftrout.com>
http://www.jefftrout.com/
http://www.stuarthamm.net/



Re: Sun performance - Major discovery!

From
pgsql-performance@sysd.com
Date:
In message <Pine.BSF.4.44.0310081408370.64781-100000@torgo.978.org>, Jeff writes:

    I'll go run the regression test suite with my gcc -O2 pg and the suncc pg.
    See if they pass the test.

My default set of gcc optimization flags is:

-O3 -funroll-loops -frerun-cse-after-loop -frerun-loop-opt -falign-functions -mcpu=i686 -march=i686

Obviously the last two flags product CPU specific code, so would have
to differ...autoconf is always possible, but so is just lopping them off.

I have found these flags to produce faster code that a simple -O2, but
I understand the exact combination which is best for you is
code-dependent.  Of course, if you are getting really excited, you can
use -fbranch-probabilities, but as you will see if you investigate
that requires some profiling information, so is not very easy to
actually practically use.

                                        -Seth Robertson

Re: Sun performance - Major discovery!

From
Tom Lane
Date:
Neil Conway <neilc@samurai.com> writes:
> On Wed, 2003-10-08 at 14:31, Bruce Momjian wrote:
>> Well, this is really embarassing.  I can't imagine why we would not set
>> at least -O on all platforms.

I believe that autoconf will automatically select -O2 (when CFLAGS isn't
already set) *if* it's chosen gcc.  It won't select anything for vendor
ccs.

> Can we get these optimizations enabled in time for the next 7.4 beta?

I think it's too late in the beta cycle to add optimization flags except
for platforms we can get specific success results for.  (Solaris is
probably okay for instance.)  The risk of breaking things seems too
high.

            regards, tom lane

Re: Sun performance - Major discovery!

From
Bruce Momjian
Date:
Tom Lane wrote:
> Neil Conway <neilc@samurai.com> writes:
> > On Wed, 2003-10-08 at 14:31, Bruce Momjian wrote:
> >> Well, this is really embarassing.  I can't imagine why we would not set
> >> at least -O on all platforms.
>
> I believe that autoconf will automatically select -O2 (when CFLAGS isn't
> already set) *if* it's chosen gcc.  It won't select anything for vendor
> ccs.

I think the problem is that template/solaris overrides that with:

  CFLAGS=

> > Can we get these optimizations enabled in time for the next 7.4 beta?
>
> I think it's too late in the beta cycle to add optimization flags except
> for platforms we can get specific success results for.  (Solaris is
> probably okay for instance.)  The risk of breaking things seems too
> high.

Agreed.  Do we set them all to -O2, then remove it from the ones we
don't get successful reports on?

--
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Re: [HACKERS] Sun performance - Major discovery!

From
Bruce Momjian
Date:
Tom Lane wrote:
> Neil Conway <neilc@samurai.com> writes:
> > On Wed, 2003-10-08 at 14:31, Bruce Momjian wrote:
> >> Well, this is really embarassing.  I can't imagine why we would not set
> >> at least -O on all platforms.
>
> I believe that autoconf will automatically select -O2 (when CFLAGS isn't
> already set) *if* it's chosen gcc.  It won't select anything for vendor
> ccs.
>
> > Can we get these optimizations enabled in time for the next 7.4 beta?
>
> I think it's too late in the beta cycle to add optimization flags except
> for platforms we can get specific success results for.  (Solaris is
> probably okay for instance.)  The risk of breaking things seems too
> high.

OK, patch attached and applied.  It centralizes the optimization
defaults into configure.in, rather than having CFLAGS= in the template
files.

It used -O2 for gcc (generated automatically by autoconf), and -O for
non-gcc, unless the template overrides it.

--
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073
Index: configure
===================================================================
RCS file: /cvsroot/pgsql-server/configure,v
retrieving revision 1.302
diff -c -c -r1.302 configure
*** configure    3 Oct 2003 03:08:14 -0000    1.302
--- configure    9 Oct 2003 03:16:44 -0000
***************
*** 2393,2398 ****
--- 2393,2402 ----
  if test "$ac_env_CFLAGS_set" = set; then
    CFLAGS=$ac_env_CFLAGS_value
  fi
+ # configure sets CFLAGS to -O2 for gcc, so this is only for non-gcc
+ if test x"$CFLAGS" = x""; then
+     CFLAGS="-O"
+ fi
  if test "$enable_debug" = yes && test "$ac_cv_prog_cc_g" = yes; then
    CFLAGS="$CFLAGS -g"
  fi
Index: configure.in
===================================================================
RCS file: /cvsroot/pgsql-server/configure.in,v
retrieving revision 1.293
diff -c -c -r1.293 configure.in
*** configure.in    3 Oct 2003 03:08:14 -0000    1.293
--- configure.in    9 Oct 2003 03:16:46 -0000
***************
*** 238,243 ****
--- 238,247 ----
  if test "$ac_env_CFLAGS_set" = set; then
    CFLAGS=$ac_env_CFLAGS_value
  fi
+ # configure sets CFLAGS to -O2 for gcc, so this is only for non-gcc
+ if test x"$CFLAGS" = x""; then
+     CFLAGS="-O"
+ fi
  if test "$enable_debug" = yes && test "$ac_cv_prog_cc_g" = yes; then
    CFLAGS="$CFLAGS -g"
  fi
Index: src/template/beos
===================================================================
RCS file: /cvsroot/pgsql-server/src/template/beos,v
retrieving revision 1.6
diff -c -c -r1.6 beos
*** src/template/beos    21 Oct 2000 22:36:13 -0000    1.6
--- src/template/beos    9 Oct 2003 03:16:51 -0000
***************
*** 1 ****
- CFLAGS='-O2'
--- 0 ----
Index: src/template/bsdi
===================================================================
RCS file: /cvsroot/pgsql-server/src/template/bsdi,v
retrieving revision 1.16
diff -c -c -r1.16 bsdi
*** src/template/bsdi    27 Sep 2003 16:24:44 -0000    1.16
--- src/template/bsdi    9 Oct 2003 03:16:51 -0000
***************
*** 5,13 ****
  esac

  case $host_os in
!   bsdi2.0 | bsdi2.1 | bsdi3*)
!     CC=gcc2
!     ;;
  esac

  THREAD_SUPPORT=yes
--- 5,11 ----
  esac

  case $host_os in
!   bsdi2.0 | bsdi2.1 | bsdi3*) CC=gcc2;;
  esac

  THREAD_SUPPORT=yes
Index: src/template/cygwin
===================================================================
RCS file: /cvsroot/pgsql-server/src/template/cygwin,v
retrieving revision 1.2
diff -c -c -r1.2 cygwin
*** src/template/cygwin    9 Oct 2003 02:37:09 -0000    1.2
--- src/template/cygwin    9 Oct 2003 03:16:51 -0000
***************
*** 1,2 ****
- CFLAGS='-O2'
  SRCH_LIB='/usr/local/lib'
--- 1 ----
Index: src/template/dgux
===================================================================
RCS file: /cvsroot/pgsql-server/src/template/dgux,v
retrieving revision 1.10
diff -c -c -r1.10 dgux
*** src/template/dgux    21 Oct 2000 22:36:13 -0000    1.10
--- src/template/dgux    9 Oct 2003 03:16:51 -0000
***************
*** 1 ****
- CFLAGS=
--- 0 ----
Index: src/template/freebsd
===================================================================
RCS file: /cvsroot/pgsql-server/src/template/freebsd,v
retrieving revision 1.23
diff -c -c -r1.23 freebsd
*** src/template/freebsd    27 Sep 2003 16:24:44 -0000    1.23
--- src/template/freebsd    9 Oct 2003 03:16:51 -0000
***************
*** 1,17 ****
- CFLAGS='-pipe'
-
  case $host_cpu in
!   alpha*)   CFLAGS="$CFLAGS -O" ;;
  esac

  THREAD_SUPPORT=yes
  NEED_REENTRANT_FUNCS=yes
  THREAD_CPPFLAGS="-D_THREAD_SAFE"
  case $host_os in
!         freebsd2*|freebsd3*|freebsd4*)
!             THREAD_LIBS="-pthread"
!             ;;
!         *)
!             THREAD_LIBS="-lc_r"
!             ;;
  esac
--- 1,11 ----
  case $host_cpu in
!   alpha*)   CFLAGS="-O";;
  esac

  THREAD_SUPPORT=yes
  NEED_REENTRANT_FUNCS=yes
  THREAD_CPPFLAGS="-D_THREAD_SAFE"
  case $host_os in
!     freebsd2*|freebsd3*|freebsd4*) THREAD_LIBS="-pthread";;
!     *) THREAD_LIBS="-lc_r";;
  esac
Index: src/template/hpux
===================================================================
RCS file: /cvsroot/pgsql-server/src/template/hpux,v
retrieving revision 1.7
diff -c -c -r1.7 hpux
*** src/template/hpux    2 Apr 2003 00:49:28 -0000    1.7
--- src/template/hpux    9 Oct 2003 03:16:51 -0000
***************
*** 1,8 ****
! if test "$GCC" = yes ; then
!   CPPFLAGS="-D_XOPEN_SOURCE_EXTENDED"
!   CFLAGS="-O2"
! else
    CC="$CC -Ae"
-   CPPFLAGS="-D_XOPEN_SOURCE_EXTENDED"
    CFLAGS="+O2"
  fi
--- 1,6 ----
! CPPFLAGS="-D_XOPEN_SOURCE_EXTENDED"
!
! if test "$GCC" != yes ; then
    CC="$CC -Ae"
    CFLAGS="+O2"
  fi
Index: src/template/irix5
===================================================================
RCS file: /cvsroot/pgsql-server/src/template/irix5,v
retrieving revision 1.9
diff -c -c -r1.9 irix5
*** src/template/irix5    21 Oct 2000 22:36:13 -0000    1.9
--- src/template/irix5    9 Oct 2003 03:16:51 -0000
***************
*** 1 ****
- CFLAGS=
--- 0 ----
Index: src/template/linux
===================================================================
RCS file: /cvsroot/pgsql-server/src/template/linux,v
retrieving revision 1.18
diff -c -c -r1.18 linux
*** src/template/linux    27 Sep 2003 22:23:35 -0000    1.18
--- src/template/linux    9 Oct 2003 03:16:51 -0000
***************
*** 1,4 ****
- CFLAGS=-O2
  # Force _GNU_SOURCE on; plperl is broken with Perl 5.8.0 otherwise
  CPPFLAGS="-D_GNU_SOURCE"

--- 1,3 ----
***************
*** 6,9 ****
  NEED_REENTRANT_FUNCS=yes    # Debian kernel 2.2 2003-09-27
  THREAD_CPPFLAGS="-D_REENTRANT -D_THREAD_SAFE -D_POSIX_PTHREAD_SEMANTICS"
  THREAD_LIBS="-lpthread"
-
--- 5,7 ----
Index: src/template/netbsd
===================================================================
RCS file: /cvsroot/pgsql-server/src/template/netbsd,v
retrieving revision 1.13
diff -c -c -r1.13 netbsd
*** src/template/netbsd    27 Sep 2003 16:24:44 -0000    1.13
--- src/template/netbsd    9 Oct 2003 03:16:51 -0000
***************
*** 1,4 ****
- CFLAGS='-O2 -pipe'
-
  THREAD_SUPPORT=yes
  NEED_REENTRANT_FUNCS=yes    # 1.6 2003-09-14
--- 1,2 ----
Index: src/template/nextstep
===================================================================
RCS file: /cvsroot/pgsql-server/src/template/nextstep,v
retrieving revision 1.7
diff -c -c -r1.7 nextstep
*** src/template/nextstep    15 Jul 2000 15:54:52 -0000    1.7
--- src/template/nextstep    9 Oct 2003 03:16:51 -0000
***************
*** 1,4 ****
  AROPT=rc
- CFLAGS=
  SHARED_LIB=
  DLSUFFIX=.o
--- 1,3 ----
Index: src/template/openbsd
===================================================================
RCS file: /cvsroot/pgsql-server/src/template/openbsd,v
retrieving revision 1.8
diff -c -c -r1.8 openbsd
*** src/template/openbsd    21 Oct 2000 22:36:14 -0000    1.8
--- src/template/openbsd    9 Oct 2003 03:16:51 -0000
***************
*** 1 ****
- CFLAGS='-O2 -pipe'
--- 0 ----
Index: src/template/osf
===================================================================
RCS file: /cvsroot/pgsql-server/src/template/osf,v
retrieving revision 1.10
diff -c -c -r1.10 osf
*** src/template/osf    27 Sep 2003 16:24:45 -0000    1.10
--- src/template/osf    9 Oct 2003 03:16:51 -0000
***************
*** 1,6 ****
! if test "$GCC" = yes ; then
!   CFLAGS=
! else
    CC="$CC -std"
    CFLAGS='-O4 -Olimit 2000'
  fi
--- 1,4 ----
! if test "$GCC" != yes ; then
    CC="$CC -std"
    CFLAGS='-O4 -Olimit 2000'
  fi
Index: src/template/qnx4
===================================================================
RCS file: /cvsroot/pgsql-server/src/template/qnx4,v
retrieving revision 1.4
diff -c -c -r1.4 qnx4
*** src/template/qnx4    24 May 2001 22:33:18 -0000    1.4
--- src/template/qnx4    9 Oct 2003 03:16:51 -0000
***************
*** 1,2 ****
! CFLAGS=-I/usr/local/include
! LIBS=-lunix
--- 1,2 ----
! CFLAGS="-O2 -I/usr/local/include"
! LIBS="-lunix"
Index: src/template/sco
===================================================================
RCS file: /cvsroot/pgsql-server/src/template/sco,v
retrieving revision 1.10
diff -c -c -r1.10 sco
*** src/template/sco    11 Dec 2002 22:27:26 -0000    1.10
--- src/template/sco    9 Oct 2003 03:16:51 -0000
***************
*** 1,7 ****
- if test "$GCC" = yes; then
-   CFLAGS=-O2
- else
-   CFLAGS=-O
- fi
  CC="$CC -b elf"

--- 1,2 ----
Index: src/template/solaris
===================================================================
RCS file: /cvsroot/pgsql-server/src/template/solaris,v
retrieving revision 1.5
diff -c -c -r1.5 solaris
*** src/template/solaris    27 Sep 2003 16:24:45 -0000    1.5
--- src/template/solaris    9 Oct 2003 03:16:51 -0000
***************
*** 1,8 ****
! if test "$GCC" = yes ; then
!   CFLAGS=
! else
    CC="$CC -Xa"            # relaxed ISO C mode
!   CFLAGS=-v            # -v is like gcc -Wall
  fi

  THREAD_SUPPORT=yes
--- 1,6 ----
! if test "$GCC" != yes ; then
    CC="$CC -Xa"            # relaxed ISO C mode
!   CFLAGS="-O -v"        # -v is like gcc -Wall
  fi

  THREAD_SUPPORT=yes
Index: src/template/sunos4
===================================================================
RCS file: /cvsroot/pgsql-server/src/template/sunos4,v
retrieving revision 1.2
diff -c -c -r1.2 sunos4
*** src/template/sunos4    21 Oct 2000 22:36:14 -0000    1.2
--- src/template/sunos4    9 Oct 2003 03:16:51 -0000
***************
*** 1 ****
- CFLAGS=
--- 0 ----
Index: src/template/svr4
===================================================================
RCS file: /cvsroot/pgsql-server/src/template/svr4,v
retrieving revision 1.10
diff -c -c -r1.10 svr4
*** src/template/svr4    21 Oct 2000 22:36:14 -0000    1.10
--- src/template/svr4    9 Oct 2003 03:16:51 -0000
***************
*** 1 ****
- CFLAGS=
--- 0 ----
Index: src/template/ultrix4
===================================================================
RCS file: /cvsroot/pgsql-server/src/template/ultrix4,v
retrieving revision 1.10
diff -c -c -r1.10 ultrix4
*** src/template/ultrix4    21 Oct 2000 22:36:14 -0000    1.10
--- src/template/ultrix4    9 Oct 2003 03:16:51 -0000
***************
*** 1 ****
- CFLAGS=
--- 0 ----
Index: src/template/univel
===================================================================
RCS file: /cvsroot/pgsql-server/src/template/univel,v
retrieving revision 1.13
diff -c -c -r1.13 univel
*** src/template/univel    21 Oct 2000 22:36:14 -0000    1.13
--- src/template/univel    9 Oct 2003 03:16:51 -0000
***************
*** 1,2 ****
  CFLAGS='-v -O -K i486,host,inline,loop_unroll -Dsvr4'
! LIBS=-lc89
--- 1,2 ----
  CFLAGS='-v -O -K i486,host,inline,loop_unroll -Dsvr4'
! LIBS="-lc89"
Index: src/template/unixware
===================================================================
RCS file: /cvsroot/pgsql-server/src/template/unixware,v
retrieving revision 1.24
diff -c -c -r1.24 unixware
*** src/template/unixware    27 Sep 2003 16:24:45 -0000    1.24
--- src/template/unixware    9 Oct 2003 03:16:51 -0000
***************
*** 1,5 ****
  if test "$GCC" = yes; then
-   CFLAGS=-O2
    THREAD_CPPFLAGS="-pthread"
  else
  # the -Kno_host is temporary for a bug in the compiler.  See -hackers
--- 1,4 ----
Index: src/template/win
===================================================================
RCS file: /cvsroot/pgsql-server/src/template/win,v
retrieving revision 1.5
diff -c -c -r1.5 win
*** src/template/win    8 Oct 2003 18:23:08 -0000    1.5
--- src/template/win    9 Oct 2003 03:16:51 -0000
***************
*** 1,3 ****
- if test "$GCC" = yes; then
-   CFLAGS="-O2"
- fi
--- 0 ----
Index: src/template/win32
===================================================================
RCS file: /cvsroot/pgsql-server/src/template/win32,v
retrieving revision 1.1
diff -c -c -r1.1 win32
*** src/template/win32    15 May 2003 16:35:30 -0000    1.1
--- src/template/win32    9 Oct 2003 03:16:51 -0000
***************
*** 1,3 ****
- if test "$GCC" = yes; then
-   CFLAGS="-O2"
- fi
--- 0 ----

Re: [HACKERS] Sun performance - Major discovery!

From
Tom Lane
Date:

Re: Sun performance - Major discovery!

From
Jeff
Date:
On Wed, 8 Oct 2003, Neil Conway wrote:

> Hey Jeff,
>
> On Wed, 2003-10-08 at 11:46, Jeff wrote:
> > Yeah - like I expected it was able to generate much better code for
> > _bt_checkkeys which was the #1 function in gcc on both sun & linux.
>
> If you get a minute, would it be possible to compare the performance of
> your benchmark under linux/gcc and solaris/gcc when PostgreSQL is
> compiled with "-O3"?
>
Sun:
gcc:
none: 60 seconds
-O: 21 seconds
-O2: 20 seconds
-O3: 19 seconds

suncc:
none: 52 seconds
-fast: 20 secondsish.

-fast is actually a macro that expands to the "best settings" for the
platform that is doing the compilation.


Linux:
-O2: 35
-O3: 40
Odd.. I wonder why it took longer. Perhaps gcc built some bad code?
I thought the results were odd there so I ran the test many times.. same
results! Swapped the binaries back (so -O2 was running) and boom. back to
35.

Sun gcc -O2 and suncc -fast both pass make check.


--
Jeff Trout <jeff@jefftrout.com>
http://www.jefftrout.com/
http://www.stuarthamm.net/



Re: Sun performance - Major discovery!

From
Bruce Momjian
Date:
So you want -fast added as default for non-gcc Solaris?  You mentioned
there is a warning generated that we have to deal with?

---------------------------------------------------------------------------

Jeff wrote:
> On Wed, 8 Oct 2003, Neil Conway wrote:
>
> > Hey Jeff,
> >
> > On Wed, 2003-10-08 at 11:46, Jeff wrote:
> > > Yeah - like I expected it was able to generate much better code for
> > > _bt_checkkeys which was the #1 function in gcc on both sun & linux.
> >
> > If you get a minute, would it be possible to compare the performance of
> > your benchmark under linux/gcc and solaris/gcc when PostgreSQL is
> > compiled with "-O3"?
> >
> Sun:
> gcc:
> none: 60 seconds
> -O: 21 seconds
> -O2: 20 seconds
> -O3: 19 seconds
>
> suncc:
> none: 52 seconds
> -fast: 20 secondsish.
>
> -fast is actually a macro that expands to the "best settings" for the
> platform that is doing the compilation.
>
>
> Linux:
> -O2: 35
> -O3: 40
> Odd.. I wonder why it took longer. Perhaps gcc built some bad code?
> I thought the results were odd there so I ran the test many times.. same
> results! Swapped the binaries back (so -O2 was running) and boom. back to
> 35.
>
> Sun gcc -O2 and suncc -fast both pass make check.
>
>
> --
> Jeff Trout <jeff@jefftrout.com>
> http://www.jefftrout.com/
> http://www.stuarthamm.net/
>
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 3: if posting/reading through Usenet, please send an appropriate
>       subscribe-nomail command to majordomo@postgresql.org so that your
>       message can get through to the mailing list cleanly
>

--
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Re: [HACKERS] Sun performance - Major discovery!

From
Andrew Sullivan
Date:
On Wed, Oct 08, 2003 at 02:31:29PM -0400, Bruce Momjian wrote:
> Well, this is really embarassing.  I can't imagine why we would not set
> at least -O on all platforms.  Looking at the template files, I see
> these have no optimization set:

I think gcc _used_ to generate bad code on SPARC if you set any
optimisation.  We tested it on Sol7 with gcc 2.95 more than a year
ago, and tried various settings.  -O2 worked, but other items were
really bad.  Some of them would pass regression but cause strange
behaviour, random coredumps, &c.  A little digging demonstrated that
anything beyond -O2 just didn't work for gcc at the time.

A

--
----
Andrew Sullivan                         204-4141 Yonge Street
Afilias Canada                        Toronto, Ontario Canada
<andrew@libertyrms.info>                              M2P 2A8
                                         +1 416 646 3304 x110


Re: Sun performance - Major discovery!

From
Jeff
Date:
On Thu, 9 Oct 2003, Bruce Momjian wrote:

>
> So you want -fast added as default for non-gcc Solaris?  You mentioned
> there is a warning generated that we have to deal with?
>

 Yeah, suncc generates a warning for _every_ file that says:
Warning: -xarch=native has been explicitly specified, or implicitly
specified by a macro option, -xarch=native on this architecture implies
-xarch=v8plusa which generates code that does not run on pre-UltraSPARC
processors

And then I get various warnings here and there...

lots of "statement not reached" as in ecpg's type.c module
The offending code is a big switch statement like:
                case ECPGt_bool:
                        return ("ECPGt_bool");
                        break;

And then any functiont aht uses PG_RETURN_NULL generates " warning:
end-of-loop code not reached"

and a bunch of "constant promoted to unsigned long long"


And some places such as in fe-exec.c have code like this:
        buflen = strlen(strtext);       /* will shrink, also we discover
if

where strtext is an unsigned char * which generates warning: argument #1
is incompatible with prototype:

and then various other type mismatches here and there.

I skimmed through the manpage.. it doesn't look like we can supress
these..


Not sure we want it to look like we have bad code if someone uses cc.
perhaps issue a ./configure notice or something?

gcc compiles things fine.


--
Jeff Trout <jeff@jefftrout.com>
http://www.jefftrout.com/
http://www.stuarthamm.net/



Re: Sun performance - Major discovery!

From
Bruce Momjian
Date:
What is the performance win for the -fast flag again?

---------------------------------------------------------------------------

Jeff wrote:
> On Thu, 9 Oct 2003, Bruce Momjian wrote:
>
> >
> > So you want -fast added as default for non-gcc Solaris?  You mentioned
> > there is a warning generated that we have to deal with?
> >
>
>  Yeah, suncc generates a warning for _every_ file that says:
> Warning: -xarch=native has been explicitly specified, or implicitly
> specified by a macro option, -xarch=native on this architecture implies
> -xarch=v8plusa which generates code that does not run on pre-UltraSPARC
> processors
>
> And then I get various warnings here and there...
>
> lots of "statement not reached" as in ecpg's type.c module
> The offending code is a big switch statement like:
>                 case ECPGt_bool:
>                         return ("ECPGt_bool");
>                         break;
>
> And then any functiont aht uses PG_RETURN_NULL generates " warning:
> end-of-loop code not reached"
>
> and a bunch of "constant promoted to unsigned long long"
>
>
> And some places such as in fe-exec.c have code like this:
>         buflen = strlen(strtext);       /* will shrink, also we discover
> if
>
> where strtext is an unsigned char * which generates warning: argument #1
> is incompatible with prototype:
>
> and then various other type mismatches here and there.
>
> I skimmed through the manpage.. it doesn't look like we can supress
> these..
>
>
> Not sure we want it to look like we have bad code if someone uses cc.
> perhaps issue a ./configure notice or something?
>
> gcc compiles things fine.
>
>
> --
> Jeff Trout <jeff@jefftrout.com>
> http://www.jefftrout.com/
> http://www.stuarthamm.net/
>
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 5: Have you checked our extensive FAQ?
>
>                http://www.postgresql.org/docs/faqs/FAQ.html
>

--
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Re: Sun performance - Major discovery!

From
Jeff
Date:
On Thu, 9 Oct 2003, Bruce Momjian wrote:

>
> What is the performance win for the -fast flag again?
>
> ---------------------------------------------------------------------------
>
52 seconds to 19-20 seconds


--
Jeff Trout <jeff@jefftrout.com>
http://www.jefftrout.com/
http://www.stuarthamm.net/



Re: Sun performance - Major discovery!

From
Bruce Momjian
Date:
Jeff wrote:
> On Thu, 9 Oct 2003, Bruce Momjian wrote:
>
> >
> > What is the performance win for the -fast flag again?
> >
> > ---------------------------------------------------------------------------
> >
> 52 seconds to 19-20 seconds

Wow, that's dramatic.  Do you want to propose some flags for non-gcc
Solaris?  Is -fast the only one?  Is there one that suppresses those
warnings or are they OK?

--
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Re: Sun performance - Major discovery!

From
Jeff
Date:
On Thu, 9 Oct 2003, Bruce Momjian wrote:

> > 52 seconds to 19-20 seconds
>
> Wow, that's dramatic.  Do you want to propose some flags for non-gcc
> Solaris?  Is -fast the only one?  Is there one that suppresses those
> warnings or are they OK?
>

Well. As I said, I didn't see an obvious way to hide those warnings.
I'd love to make those warnings go away.  That is why I suggested perhaps
printing a message to ensure the user knows that warnings may be printed
when using sunsoft.

-fast should be all you need - it picks the "best settings" to use for the
platform that is doing the compile.


--
Jeff Trout <jeff@jefftrout.com>
http://www.jefftrout.com/
http://www.stuarthamm.net/



Re: Sun performance - Major discovery!

From
Jeff
Date:
On Thu, 9 Oct 2003, Kenneth Marshall wrote:

> Jeff,
>
> My first concern with the -fast option is that it makes an executable
> that is specific for the platform on which the compilation is run
> unless other flags are given. My second concern is the effect it has
> on IEEE floating point behavior w.r.t. rounding, error handling, ....
> And my third concern is that if you use -fast, all other code must
> be compiled and linked with the -fast option for correct operation,
> this includes any functional languages such as perl, python, R,...
> That is a pretty big requirement for a default compilation flag.
>
> Ken Marshall
>

So you think we should leave PG alone and let it run horrifically slowly?
Do you have a better idea of how to do this?

And do you have evidence apps compiled with -fast linked to non -fast
(or gcc compiled) have problems?


--
Jeff Trout <jeff@jefftrout.com>
http://www.jefftrout.com/
http://www.stuarthamm.net/



Re: Sun performance - Major discovery!

From
Bruce Momjian
Date:
Jeff wrote:
> On Thu, 9 Oct 2003, Kenneth Marshall wrote:
>
> > Jeff,
> >
> > My first concern with the -fast option is that it makes an executable
> > that is specific for the platform on which the compilation is run
> > unless other flags are given. My second concern is the effect it has
> > on IEEE floating point behavior w.r.t. rounding, error handling, ....
> > And my third concern is that if you use -fast, all other code must
> > be compiled and linked with the -fast option for correct operation,
> > this includes any functional languages such as perl, python, R,...
> > That is a pretty big requirement for a default compilation flag.
> >
> > Ken Marshall
> >
>
> So you think we should leave PG alone and let it run horrifically slowly?
> Do you have a better idea of how to do this?
>
> And do you have evidence apps compiled with -fast linked to non -fast
> (or gcc compiled) have problems?

I have updated the Solaris FAQ:


5) How can I compile for optimum performance?

Try using the "-fast" compile flag.  The binaries might not be portable to
other Solaris systems, and you might need to compile everything that links
to PostgreSQL with "-fast", but PostgreSQL will run significantly faster,
50% faster on some tests.

--
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Re: Sun performance - Major discovery!

From
Andrew Sullivan
Date:
On Thu, Oct 09, 2003 at 01:04:23PM -0400, Jeff wrote:
>
> So you think we should leave PG alone and let it run horrifically slowly?
> Do you have a better idea of how to do this?

Given the point in the release cycle, mightn't the FAQ_Solaris or
some other place be better for this for now?  I agree with the
concern.  I'd rather have slow'n'stable than fast-but-broken.

A

--
----
Andrew Sullivan                         204-4141 Yonge Street
Afilias Canada                        Toronto, Ontario Canada
<andrew@libertyrms.info>                              M2P 2A8
                                         +1 416 646 3304 x110


Re: Sun performance - Major discovery!

From
Bruce Momjian
Date:
Andrew Sullivan wrote:
> On Thu, Oct 09, 2003 at 01:04:23PM -0400, Jeff wrote:
> >
> > So you think we should leave PG alone and let it run horrifically slowly?
> > Do you have a better idea of how to do this?
>
> Given the point in the release cycle, mightn't the FAQ_Solaris or
> some other place be better for this for now?  I agree with the
> concern.  I'd rather have slow'n'stable than fast-but-broken.

FAQ added.

--
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Re: Sun performance - Major discovery!

From
Jeff
Date:
We're keeping the -O2 for gcc in the template and moving the mention of
-fast to the FAQ, correct?

--
Jeff Trout <jeff@jefftrout.com>
http://www.jefftrout.com/
http://www.stuarthamm.net/



Re: Sun performance - Major discovery!

From
Bruce Momjian
Date:
Jeff wrote:
> We're keeping the -O2 for gcc in the template and moving the mention of
> -fast to the FAQ, correct?

gcc gets -O2, non-gcc gets -O, and -fast is in the FAQ, yea.

--
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Re: Sun performance - Major discovery!

From
Christopher Browne
Date:
pgman@candle.pha.pa.us (Bruce Momjian) writes:
> 5) How can I compile for optimum performance?
>
> Try using the "-fast" compile flag.  The binaries might not be portable to
> other Solaris systems, and you might need to compile everything that links
> to PostgreSQL with "-fast", but PostgreSQL will run significantly faster,
> 50% faster on some tests.

You might also mention something like the following:

  If you are compiling using GCC, you will quite likely want to add in
  the "-O2" compile flag.
--
let name="cbbrowne" and tld="libertyrms.info" in String.concat "@" [name;tld];;
<http://dev6.int.libertyrms.com/>
Christopher Browne
(416) 646 3304 x124 (land)

Re: Sun performance - Major discovery!

From
Bruce Momjian
Date:
Christopher Browne wrote:
> pgman@candle.pha.pa.us (Bruce Momjian) writes:
> > 5) How can I compile for optimum performance?
> >
> > Try using the "-fast" compile flag.  The binaries might not be portable to
> > other Solaris systems, and you might need to compile everything that links
> > to PostgreSQL with "-fast", but PostgreSQL will run significantly faster,
> > 50% faster on some tests.
>
> You might also mention something like the following:
>
>   If you are compiling using GCC, you will quite likely want to add in
>   the "-O2" compile flag.

We already do that by default in current CVS for gcc, and -O for
non-gcc.

--
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Re: Sun performance - Major discovery!

From
Neil Conway
Date:
On Wed, 2003-10-08 at 21:44, Bruce Momjian wrote:
> Agreed.  Do we set them all to -O2, then remove it from the ones we
> don't get successful reports on?

I took the time to compile CVS tip with a few different machines from
HP's TestDrive program, to see if there were any regressions using the
new optimization flags:

(1) (my usual dev machine)

$ uname -a
Linux tokyo 2.4.19-xfs #1 Mon Jan 20 19:12:29 EST 2003 i686 GNU/Linux
$ gcc --version
gcc (GCC) 3.3.2 20031005 (Debian prerelease)

'make check' passes

(2)

$ uname -a
Linux spe161 2.4.18-smp #1 SMP Sat Apr 6 21:42:22 EST 2002 alpha unknown
$ gcc --version
gcc (GCC) 3.3.1

'make check' passes

(3)

$ uname -a
Linux spe170 2.4.17-64 #1 Sat Mar 16 17:31:44 MST 2002 parisc64 unknown
$ gcc --version
3.0.4

'make check' passes

BTW, this platform doesn't have any code written for native spinlocks.

(4)

$ uname -a
Linux spe156 2.4.18-mckinley-smp #1 SMP Thu Jul 11 12:51:02 MDT 2002
ia64 unknown
$ gcc --version

When you compile PostgreSQL without changing the CFLAGS configure picks,
the initdb required for 'make check' fails with:

[...]
initializing pg_depend... ok
creating system views... ok
loading pg_description... ok
creating conversions... ERROR:  could not identify operator 679

I tried to compile PostgreSQL with CFLAGS='-O0' to see if the above
resulted from an optimization-induced compiler error, but I got the
following error:

$ gcc -O0 -Wall -Wmissing-prototypes -Wmissing-declarations
-I../../../../src/include -D_GNU_SOURCE   -c -o xlog.o xlog.c
../../../../src/include/storage/s_lock.h: In function `tas':
../../../../src/include/storage/s_lock.h:125: error: inconsistent
operand constraints in an `asm'

Whereas this works fine:

$ gcc -O2 -Wall -Wmissing-prototypes -Wmissing-declarations
-I../../../../src/include -D_GNU_SOURCE   -c -o xlog.o xlog.c
$

BTW, line 138 of s_lock.h is:

#if defined(__arm__) || defined(__arm__)

That seems a little redundant.

Anyway, I tried running initdb after compiling all of pgsql with "-O0',
except for the files that included s_lock.h, but make check still
failed:

creating information schema... ok
vacuuming database template1...
/house/neilc/pgsql/src/test/regress/./tmp_check/install//usr/local/pgsql/bin/initdb: line 882: 22035 Segmentation fault
    (core dumped) "$PGPATH"/postgres $PGSQL_OPT template1 >/dev/null  <<EOF 
ANALYZE;
VACUUM FULL FREEZE;
EOF

The core file seems to indicate a stack overflow due to an infinitely
recursive function:

(gdb) bt 25
#0  0x4000000000645dc0 in hash_search ()
#1  0x4000000000616930 in RelationSysNameCacheGetRelation ()
#2  0x4000000000616db0 in RelationSysNameGetRelation ()
#3  0x4000000000082e40 in relation_openr ()
#4  0x4000000000083910 in heap_openr ()
#5  0x400000000060e6b0 in ScanPgRelation ()
#6  0x4000000000611d60 in RelationBuildDesc ()
#7  0x4000000000616e70 in RelationSysNameGetRelation ()
#8  0x4000000000082e40 in relation_openr ()
#9  0x4000000000083910 in heap_openr ()
#10 0x400000000060e6b0 in ScanPgRelation ()
#11 0x4000000000611d60 in RelationBuildDesc ()
#12 0x4000000000616e70 in RelationSysNameGetRelation ()
#13 0x4000000000082e40 in relation_openr ()
#14 0x4000000000083910 in heap_openr ()
#15 0x400000000060e6b0 in ScanPgRelation ()
#16 0x4000000000611d60 in RelationBuildDesc ()
#17 0x4000000000616e70 in RelationSysNameGetRelation ()
#18 0x4000000000082e40 in relation_openr ()
#19 0x4000000000083910 in heap_openr ()
#20 0x400000000060e6b0 in ScanPgRelation ()
#21 0x4000000000611d60 in RelationBuildDesc ()
#22 0x4000000000616e70 in RelationSysNameGetRelation ()
#23 0x4000000000082e40 in relation_openr ()
#24 0x4000000000083910 in heap_openr ()
(More stack frames follow...)

(It also dumps core in the same place during initdb if CFLAGS='-O' is
specified.)

So it looks like the Itanium port is a little broken. Does anyone have
an idea what needs to be done to fix it?

-Neil



Re: Sun performance - Major discovery!

From
Bruce Momjian
Date:
Isn't it great how you have the same directory on every host so you can
download once and run the same tests easily.


Neil Conway wrote:
> $ uname -a
> Linux spe170 2.4.17-64 #1 Sat Mar 16 17:31:44 MST 2002 parisc64 unknown
> $ gcc --version
> 3.0.4
>
> 'make check' passes

I didn't know there was a pa-risc-64 chip.

> BTW, this platform doesn't have any code written for native spinlocks.
>
> (4)
>
> $ uname -a
> Linux spe156 2.4.18-mckinley-smp #1 SMP Thu Jul 11 12:51:02 MDT 2002
> ia64 unknown
> $ gcc --version
>
> When you compile PostgreSQL without changing the CFLAGS configure picks,
> the initdb required for 'make check' fails with:
>
> [...]
> initializing pg_depend... ok
> creating system views... ok
> loading pg_description... ok
> creating conversions... ERROR:  could not identify operator 679
>
> I tried to compile PostgreSQL with CFLAGS='-O0' to see if the above
> resulted from an optimization-induced compiler error, but I got the
> following error:
>
> $ gcc -O0 -Wall -Wmissing-prototypes -Wmissing-declarations
> -I../../../../src/include -D_GNU_SOURCE   -c -o xlog.o xlog.c
> ../../../../src/include/storage/s_lock.h: In function `tas':
> ../../../../src/include/storage/s_lock.h:125: error: inconsistent
> operand constraints in an `asm'
>
> Whereas this works fine:
>
> $ gcc -O2 -Wall -Wmissing-prototypes -Wmissing-declarations
> -I../../../../src/include -D_GNU_SOURCE   -c -o xlog.o xlog.c
> $
>
> BTW, line 138 of s_lock.h is:
>
> #if defined(__arm__) || defined(__arm__)

Fix just committed.  Thanks.

> That seems a little redundant.
>
> Anyway, I tried running initdb after compiling all of pgsql with "-O0',
> except for the files that included s_lock.h, but make check still
> failed:
>
> creating information schema... ok
> vacuuming database template1...
> /house/neilc/pgsql/src/test/regress/./tmp_check/install//usr/local/pgsql/bin/initdb: line 882: 22035 Segmentation
fault     (core dumped) "$PGPATH"/postgres $PGSQL_OPT template1 >/dev/null  <<EOF 
> ANALYZE;
> VACUUM FULL FREEZE;
> EOF
>
> The core file seems to indicate a stack overflow due to an infinitely
> recursive function:
>
> (gdb) bt 25
> #0  0x4000000000645dc0 in hash_search ()
> #1  0x4000000000616930 in RelationSysNameCacheGetRelation ()
> #2  0x4000000000616db0 in RelationSysNameGetRelation ()
> #3  0x4000000000082e40 in relation_openr ()
> #4  0x4000000000083910 in heap_openr ()
> #5  0x400000000060e6b0 in ScanPgRelation ()
> #6  0x4000000000611d60 in RelationBuildDesc ()
> #7  0x4000000000616e70 in RelationSysNameGetRelation ()
> #8  0x4000000000082e40 in relation_openr ()
> #9  0x4000000000083910 in heap_openr ()
> #10 0x400000000060e6b0 in ScanPgRelation ()
> #11 0x4000000000611d60 in RelationBuildDesc ()
> #12 0x4000000000616e70 in RelationSysNameGetRelation ()
> #13 0x4000000000082e40 in relation_openr ()
> #14 0x4000000000083910 in heap_openr ()
> #15 0x400000000060e6b0 in ScanPgRelation ()
> #16 0x4000000000611d60 in RelationBuildDesc ()
> #17 0x4000000000616e70 in RelationSysNameGetRelation ()
> #18 0x4000000000082e40 in relation_openr ()
> #19 0x4000000000083910 in heap_openr ()
> #20 0x400000000060e6b0 in ScanPgRelation ()
> #21 0x4000000000611d60 in RelationBuildDesc ()
> #22 0x4000000000616e70 in RelationSysNameGetRelation ()
> #23 0x4000000000082e40 in relation_openr ()
> #24 0x4000000000083910 in heap_openr ()
> (More stack frames follow...)
>
> (It also dumps core in the same place during initdb if CFLAGS='-O' is
> specified.)
>
> So it looks like the Itanium port is a little broken. Does anyone have
> an idea what needs to be done to fix it?

My guess is that the compiler itself is broken --- what else could it
be?

--
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Re: [HACKERS] Sun performance - Major discovery!

From
Tom Lane
Date:
Bruce Momjian <pgman@candle.pha.pa.us> writes:
> OK, patch attached and applied.  It centralizes the optimization
> defaults into configure.in, rather than having CFLAGS= in the template
> files.

I think there's a problem here:

> + # configure sets CFLAGS to -O2 for gcc, so this is only for non-gcc
> + if test x"$CFLAGS" = x""; then
> +     CFLAGS="-O"
> + fi
>   if test "$enable_debug" = yes && test "$ac_cv_prog_cc_g" = yes; then
>     CFLAGS="$CFLAGS -g"
>   fi

since this will cause "configure --enable-debug" to default to selecting
CFLAGS="-O -g" for non-gcc compilers.  On a lot of compilers that
combination does not work, and will generate tons of useless warnings.
I think it might be better to do

  if test "$enable_debug" = yes && test "$ac_cv_prog_cc_g" = yes; then
    CFLAGS="$CFLAGS -g"
+ else
+   # configure sets CFLAGS to -O2 for gcc, so this is only for non-gcc
+   if test x"$CFLAGS" = x""; then
+     CFLAGS="-O"
+   fi
  fi

            regards, tom lane

Re: [HACKERS] Sun performance - Major discovery!

From
Bruce Momjian
Date:
Done as you suggested.

---------------------------------------------------------------------------

Tom Lane wrote:
> Bruce Momjian <pgman@candle.pha.pa.us> writes:
> > OK, patch attached and applied.  It centralizes the optimization
> > defaults into configure.in, rather than having CFLAGS= in the template
> > files.
>
> I think there's a problem here:
>
> > + # configure sets CFLAGS to -O2 for gcc, so this is only for non-gcc
> > + if test x"$CFLAGS" = x""; then
> > +     CFLAGS="-O"
> > + fi
> >   if test "$enable_debug" = yes && test "$ac_cv_prog_cc_g" = yes; then
> >     CFLAGS="$CFLAGS -g"
> >   fi
>
> since this will cause "configure --enable-debug" to default to selecting
> CFLAGS="-O -g" for non-gcc compilers.  On a lot of compilers that
> combination does not work, and will generate tons of useless warnings.
> I think it might be better to do
>
>   if test "$enable_debug" = yes && test "$ac_cv_prog_cc_g" = yes; then
>     CFLAGS="$CFLAGS -g"
> + else
> +   # configure sets CFLAGS to -O2 for gcc, so this is only for non-gcc
> +   if test x"$CFLAGS" = x""; then
> +     CFLAGS="-O"
> +   fi
>   fi
>
>             regards, tom lane
>

--
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Re: Sun performance - Major discovery!

From
Kenneth Marshall
Date:
Jeff,

My first concern with the -fast option is that it makes an executable
that is specific for the platform on which the compilation is run
unless other flags are given. My second concern is the effect it has
on IEEE floating point behavior w.r.t. rounding, error handling, ....
And my third concern is that if you use -fast, all other code must
be compiled and linked with the -fast option for correct operation,
this includes any functional languages such as perl, python, R,...
That is a pretty big requirement for a default compilation flag.

Ken Marshall

On Thu, Oct 09, 2003 at 12:07:20PM -0400, Jeff wrote:
> On Thu, 9 Oct 2003, Bruce Momjian wrote:
>
> > > 52 seconds to 19-20 seconds
> >
> > Wow, that's dramatic.  Do you want to propose some flags for non-gcc
> > Solaris?  Is -fast the only one?  Is there one that suppresses those
> > warnings or are they OK?
> >
>
> Well. As I said, I didn't see an obvious way to hide those warnings.
> I'd love to make those warnings go away.  That is why I suggested perhaps
> printing a message to ensure the user knows that warnings may be printed
> when using sunsoft.
>
> -fast should be all you need - it picks the "best settings" to use for the
> platform that is doing the compile.
>
>
> --
> Jeff Trout <jeff@jefftrout.com>
> http://www.jefftrout.com/
> http://www.stuarthamm.net/
>
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 7: don't forget to increase your free space map settings

Re: [HACKERS] Sun performance - Major discovery!

From
Marko Karppinen
Date:
On 8.10.2003, at 21:31, Bruce Momjian wrote:
> Well, this is really embarassing.  I can't imagine why we would not set
> at least -O on all platforms.  Looking at the template files, I see
> these have no optimization set:
>
>     darwin

Regarding Darwin optimizations, Apple has introduced a "-fast" flag in
their GCC 3.3 version that they recommend when compiling code for their
new G5 systems. Because of this, I foresee a lot of people defining
CFLAGS="-fast" on their systems.

This is problematic for PostgreSQL, however, since the -fast flag is
the equivalent of:

-O3 -falign-loops-max-skip=15 -falign-jumps-max-skip=15
-falign-loops=16 -falign-jumps=16 -falign-functions=16 -malign-natural
-ffast-math -fstrict-aliasing -frelax-aliasing -fgcse-mem-alias
-funroll-loops -floop-transpose -floop-to-memset -finline-floor
-mcpu=G5 -mpowerpc64 -mpowerpc-gpopt -mtune=G5 -fsched-interblock
-fload-after-store  --param max-gcse-passes=3  -fno-gcse-sm
-fgcse-loop-depth -funit-at-a-time  -fcallgraph-inlining
-fdisable-typechecking-for-spec

At least the --fast-math part causes problems, seeing that PostgreSQL
actually checks for the __FAST_MATH__ macro to make sure that it isn't
turned on. There might be other problems with Apple's flags, but I
think that the __FAST_MATH__ check should be altered.

As you know, setting --fast-math in GCC is the equivalent of setting
-fno-math-errno, -funsafe-math-optimizations, -fno-trapping-math,
-ffinite-math-only and -fno-signaling-nans. What really should be done,
I think, is adding the opposites of these flags (-fmath-errno,
-fno-unsafe-math-optimizations, -ftrapping_math, -fno-finite-math-only
and -fsignaling-nans) to the command line if __FAST_MATH__ is detected.
This would allow people to use CFLAGS="-fast" on their G5s, beat some
Xeon speed records, and not worry about esoteric IEEE math standards.
What do you guys think?

GCC sets __FAST_MATH__ even if you counter a -ffast-math with the
negating flags above. This means that it is not currently possible to
use the -fast flag when compiling PostgreSQL at all. Instead, you have
to go through all the flags Apple is setting and only pass on those
that don't break pg.

mk


Re: [HACKERS] Sun performance - Major discovery!

From
Peter Eisentraut
Date:
Bruce Momjian writes:

> Well, this is really embarassing.  I can't imagine why we would not set
> at least -O on all platforms.  Looking at the template files, I see
> these have no optimization set:

>     freebsd (non-alpha)

I'm wondering what that had in mind:

http://developer.postgresql.org/cvsweb.cgi/pgsql-server/src/template/freebsd.diff?r1=1.10&r2=1.11

--
Peter Eisentraut   peter_e@gmx.net


Re: Sun performance - Major discovery!

From
Kenneth Marshall
Date:
I would use a simple -xO2 or -xO3 instead as the default with
an -fsimple=2.

--Ken

-x02 -xbuiltin=%all
On Thu, Oct 09, 2003 at 01:04:23PM -0400, Jeff wrote:
> On Thu, 9 Oct 2003, Kenneth Marshall wrote:
>
> > Jeff,
> >
> > My first concern with the -fast option is that it makes an executable
> > that is specific for the platform on which the compilation is run
> > unless other flags are given. My second concern is the effect it has
> > on IEEE floating point behavior w.r.t. rounding, error handling, ....
> > And my third concern is that if you use -fast, all other code must
> > be compiled and linked with the -fast option for correct operation,
> > this includes any functional languages such as perl, python, R,...
> > That is a pretty big requirement for a default compilation flag.
> >
> > Ken Marshall
> >
>
> So you think we should leave PG alone and let it run horrifically slowly?
> Do you have a better idea of how to do this?
>
> And do you have evidence apps compiled with -fast linked to non -fast
> (or gcc compiled) have problems?
>
>
> --
> Jeff Trout <jeff@jefftrout.com>
> http://www.jefftrout.com/
> http://www.stuarthamm.net/
>
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 5: Have you checked our extensive FAQ?
>
>                http://www.postgresql.org/docs/faqs/FAQ.html

Re: [HACKERS] Sun performance - Major discovery!

From
Tom Lane
Date:
Marko Karppinen <marko@karppinen.fi> writes:
> At least the --fast-math part causes problems, seeing that PostgreSQL
> actually checks for the __FAST_MATH__ macro to make sure that it isn't
> turned on. There might be other problems with Apple's flags, but I
> think that the __FAST_MATH__ check should be altered.

Removing the check is not acceptable --- we spent far too much time
fighting bug reports that turned out to trace to -ffast-math.
See for example
http://archives.postgresql.org/pgsql-bugs/2002-09/msg00169.php

> As you know, setting --fast-math in GCC is the equivalent of setting
> -fno-math-errno, -funsafe-math-optimizations, -fno-trapping-math,
> -ffinite-math-only and -fno-signaling-nans.

I suspect that -funsafe-math-optimizations is the only one of those that
really affects the datetime code, but I would be quite worried about the
side-effects of any of them on the float8 arithmetic routines.  Also I
think the behavior of -ffast-math has changed over time; in the gcc
2.95.3 manual I see none of the above and only the description

`-ffast-math'
     This option allows GCC to violate some ANSI or IEEE rules and/or
     specifications in the interest of optimizing code for speed.  For
     example, it allows the compiler to assume arguments to the `sqrt'
     function are non-negative numbers and that no floating-point values
     are NaNs.

Since we certainly do use NaNs, it would be very bad to allow -ffast-math
in gcc 2.95.

gcc 3.2 has some but not all of the sub-flags you list above, so
apparently the behavior changed again as of gcc 3.3.

This means that relaxing the check would require (a) finding out which
of the sub-flags break our code and which don't; (b) finding out how the
answer to (a) has varied with gcc release; and (c) finding out how we
can test whether a given sub-flag is set --- are there #defines for each
of them in gcc 3?

This does not sound real practical to me...

> This would allow people to use CFLAGS="-fast" on their G5s, beat some
> Xeon speed records, and not worry about esoteric IEEE math standards.

In the words of the sage, "I can make this code *arbitrarily* fast ...
if it doesn't have to give the right answer."  Those "esoteric"
standards make the difference between printing 5:00:00 and printing
4:59:60.

            regards, tom lane