Thread: Solaris
We have an Ultra Enterprise 3000 with a Sparc Array 1010 lying around here acting as a coffee table in the server room. It was being used as a database server, but when a drive crumped, nobody knew how to get the array put back together, sosome $250 per hour consultant wiped out all the data. The solution was, of course, to switch to MS SQL Server on i386stuff and use the Sun machine as a coffee table. It seem like a bit of a waste. I am running my PostgreSQL database on a Dell PowerEdge 2400. What, advantages/disadvantages are there to learning Solarisand migrating my stuff over to the coffee table machine? Does the Dell fall under the heading of "cheap" (crummy)hardware that Tom alluded to causing corruption issues? Ian Harding Programmer/Analyst II Tacoma-Pierce County Health Department iharding@tpchd.org (253) 798-3549
On Wednesday 23 April 2003 20:33, Ian Harding wrote: > I am running my PostgreSQL database on a Dell PowerEdge 2400. What, > advantages/disadvantages are there to learning Solaris and migrating my > stuff over to the coffee table machine? Does the Dell fall under the > heading of "cheap" (crummy) hardware that Tom alluded to causing corruption > issues? Dell server might be good enough for most tasks but if the sun machine is 64 bit, then you have a significant advantage there. Besides they should be bit more robust(Sorry, this is absolute wild shot. Have no idea about specs of any of these two machines.) Just one real advice, from what I have heard on lists. If the sun machine supports, install linux rather than solaris. Apparently linux on sparc is far faster than solaris, at least at lower to middle end. Besides postgresql and solaris have had performance issues in past. Shridhar
On Wed, 23 Apr 2003, Shridhar Daithankar wrote: > On Wednesday 23 April 2003 20:33, Ian Harding wrote: > > I am running my PostgreSQL database on a Dell PowerEdge 2400. What, > > advantages/disadvantages are there to learning Solaris and migrating my > > stuff over to the coffee table machine? Does the Dell fall under the > > heading of "cheap" (crummy) hardware that Tom alluded to causing corruption > > issues? > > Dell server might be good enough for most tasks but if the sun machine is 64 > bit, then you have a significant advantage there. Besides they should be bit > more robust(Sorry, this is absolute wild shot. Have no idea about specs of > any of these two machines.) > > Just one real advice, from what I have heard on lists. If the sun machine > supports, install linux rather than solaris. Apparently linux on sparc is far > faster than solaris, at least at lower to middle end. Besides postgresql and > solaris have had performance issues in past. Even better, throw in an extra drive and configure it to dual boot. Then you can test the two against each other be sure which is faster for what you're doing.
On Wed, 23 Apr 2003 20:44:02 +0530 Shridhar Daithankar <shridhar_daithankar@persistent.co.in> wrote:
> Just one real advice, from what I have heard on lists. If the sun
> machine
> supports, install linux rather than solaris. Apparently linux on sparc
> is far
> faster than solaris, at least at lower to middle end. Besides postgresql
> and
> solaris have had performance issues in past.
investigate OpenBSD and NetBSD as well (although if the Sun has multiple
processors, which is likely with an E, OpenBSD is no longer a candidate due
to lack of SMP support.) the *BSD systems are generally very stable and
reliable, and have a couple of advantages over Linux in a number of
situations.
richard
--
Richard Welty rwelty@averillpark.net
Averill Park Networking 518-573-7592
Unix, Linux, IP Network Engineering, Security
You should at least have ECC RAM in the Solaris box, but then again, you should have ECC RAM in any "server" to avoid in memory corruption. If the disk "Array" blew it's toys, then it sounds like you may have yourself a RAID controller, you could easily buy a few new hard disks, and get yourself back up and running. RAID 5 needs a minimum of three disks, RAID 1 Needs two. Failing the RAID controller, the solaris will have a SCSI subsystem .... mmmmm, 15,000rpm, 16MB Cache disks.... Hadley. On Thu, 2003-04-24 at 03:34, scott.marlowe wrote: > On Wed, 23 Apr 2003, Shridhar Daithankar wrote: > > > On Wednesday 23 April 2003 20:33, Ian Harding wrote: > > > I am running my PostgreSQL database on a Dell PowerEdge 2400. What, > > > advantages/disadvantages are there to learning Solaris and migrating my > > > stuff over to the coffee table machine? Does the Dell fall under the > > > heading of "cheap" (crummy) hardware that Tom alluded to causing corruption > > > issues? > > > > Dell server might be good enough for most tasks but if the sun machine is 64 > > bit, then you have a significant advantage there. Besides they should be bit > > more robust(Sorry, this is absolute wild shot. Have no idea about specs of > > any of these two machines.) > > > > Just one real advice, from what I have heard on lists. If the sun machine > > supports, install linux rather than solaris. Apparently linux on sparc is far > > faster than solaris, at least at lower to middle end. Besides postgresql and > > solaris have had performance issues in past. > > Even better, throw in an extra drive and configure it to dual boot. Then > you can test the two against each other be sure which is faster for what > you're doing. > > > ---------------------------(end of broadcast)--------------------------- > TIP 6: Have you searched our list archives? > > http://archives.postgresql.org -- Hadley Willan > Systems Development > Deeper Design Limited. +64(7)377-3328 hadley.willan@deeperdesign.co.nz > www.deeperdesign.com > +64(21)-28-41-463 Level 1, 4 Tamamutu St, PO Box 90, TAUPO 2730, New Zealand.
Shridhar Daithankar wrote: >Just one real advice, from what I have heard on lists. If the sun machine >supports, install linux rather than solaris. Apparently linux on sparc is far >faster than solaris, at least at lower to middle end. > Any idea how much faster (and in what areas)? >Besides postgresql and >solaris have had performance issues in past. > > I believe this is all *sorted* now :-) The issue was the qsort library bundled with Solaris - Postgresql now supplies its own. So Solaris is ok for Postgresql (I think quite a few folk actually use this combination). best wishes Mark
On Fri, Apr 25, 2003 at 02:03:24PM +1200, Mark Kirkwood wrote:
> I believe this is all *sorted* now :-) The issue was the qsort library
> bundled with Solaris - Postgresql now supplies its own. So Solaris is ok
> for Postgresql (I think quite a few folk actually use this combination).
It's still not all that fast. Better, but not what you'd expect.
There's something fishy with the SYSV shared memory management, but
I'm darned if I can figure out what it is.
A
--
----
Andrew Sullivan 204-4141 Yonge Street
Liberty RMS Toronto, Ontario Canada
<andrew@libertyrms.info> M2P 2A8
+1 416 646 3304 x110
Andrew Sullivan <andrew@libertyrms.info> writes:
> There's something fishy with the SYSV shared memory management, but
> I'm darned if I can figure out what it is.
Hmm, you're still running 7.2.*, right? There's some code added in 7.3
to enable "intimate shared memory" on Solaris:
#if defined(solaris) && defined(__sparc__)
/* use intimate shared memory on SPARC Solaris */
memAddress = shmat(shmid, 0, SHM_SHARE_MMU);
#else
memAddress = shmat(shmid, 0, 0);
#endif
I disremember the details but we were told this would improve
performance. It'd be an easy enough patch in 7.2
(src/backend/storage/ipc/ipc.c about line 638) if you care to try it.
regards, tom lane
On Thu, Apr 24, 2003 at 11:08:11PM -0400, Tom Lane wrote:
> Hmm, you're still running 7.2.*, right? There's some code added in 7.3
> to enable "intimate shared memory" on Solaris:
Tried it. I was unable to show that it helped, and it seemed to make
things worse in some cases. As usual, the time I actually got to
spend doing this was (it felt) inversely proportionate to its value,
which means that any numbers I managed to produce are not real
reliable. But it sure didn't have the effect I was hoping for. The
best I can say is that the numbers were inconclusive for our
application. I have some more tests planned, if I can find the time.
A
--
----
Andrew Sullivan 204-4141 Yonge Street
Liberty RMS Toronto, Ontario Canada
<andrew@libertyrms.info> M2P 2A8
+1 416 646 3304 x110
Andrew Sullivan wrote: >It's still not all that fast. Better, but not what you'd expect. >There's something fishy with the SYSV shared memory management, but >I'm darned if I can figure out what it is. > > Just out of curiousity, where are you seeing the effect of this the most? (Running through my mind is the thought that VM management in Linux is not so hot either....) cheers Mark
On Fri, 25 Apr 2003, Mark Kirkwood wrote: > Shridhar Daithankar wrote: > > >Just one real advice, from what I have heard on lists. If the sun machine > >supports, install linux rather than solaris. Apparently linux on sparc is far > >faster than solaris, at least at lower to middle end. > > > Any idea how much faster (and in what areas)? My general experience from a year ago was that Linux was about 2 to 3 times faster on older 32 bit sparc hardware than Solaris, and just under 2 times as fast on 64 bit hardware. This is with one CPU. I'd expect a 64 CPU E10k to run postgresql faster running Solaris rather than Linux, but until I can come up with a spare $24k to buy a used one on Ebay I won't know. :-) > >Besides postgresql and > >solaris have had performance issues in past. > > > > > I believe this is all *sorted* now :-) The issue was the qsort library > bundled with Solaris - Postgresql now supplies its own. So Solaris is ok > for Postgresql (I think quite a few folk actually use this combination). Mostly. The issue in the past was mainly that Solaris had a brain damaged sort() call that was very slow when it had a lot of duplicate keys. That has since been updated by Sun. However, Solaris' heavy process / light thread design is probably sub optimal for Postgresql in an environment where you are forking the server all the time. If you have pooled connections, then Solaris should do fine. I'd love to get ahold of an older SMP Sparc box and test the two against each other.
scott.marlowe wrote: >My general experience from a year ago was that Linux was about 2 to 3 >times faster on older 32 bit sparc hardware than Solaris, and just under 2 >times as fast on 64 bit hardware. > Wow - 2 times faster is significant ! It would be interesting to try a midrange SMP box ( e.g E280/E480 ). > The issue in the past was mainly that Solaris had a brain damaged >sort() call that was very slow when it had a lot of duplicate keys. That >has since been updated by Sun. > > Hmm - I ran into that situation with qsort() and many equal keys (see Hackers thread "Solaris Performance"). Sadly Sun have not amended that situation at all (in Solaris 8 anyway). best wishes Mark
On Fri, Apr 25, 2003 at 11:19:47PM +1200, Mark Kirkwood wrote:
> Just out of curiousity, where are you seeing the effect of this the most?
> (Running through my mind is the thought that VM management in Linux is
> not so hot either....)
The problems seem to be somewhat related to load. With no load, of
course, everything is real fast. But with any sort of load, we get
occasional spikes of unexpectedly slow performance, even on things
like INSERT. Some of this I have been able to attribute to
maintenance issues, but not all of it. We see from truss, however,
that a significant amount of timee is spent in the SYSV system
calls, so it appears that there is some kind of inefficiency there.
Sorry I'm not able to provide more detail; as I say, I'm not real
happy with the extent of the tests I've been able to conduct, and in
any case, the problems aren't reproducable at will.
A
--
----
Andrew Sullivan 204-4141 Yonge Street
Liberty RMS Toronto, Ontario Canada
<andrew@libertyrms.info> M2P 2A8
+1 416 646 3304 x110
On Sat, 26 Apr 2003, Mark Kirkwood wrote: > scott.marlowe wrote: > > >My general experience from a year ago was that Linux was about 2 to 3 > >times faster on older 32 bit sparc hardware than Solaris, and just under 2 > >times as fast on 64 bit hardware. > > > Wow - 2 times faster is significant ! It would be interesting to try a > midrange SMP box ( e.g E280/E480 ). Keep in mind, this is running Postgresql that measures out at about twice as fast. I was testing with things like the 10k row database in the regression tests, a few of our own 1M row tables, and pgbench with parallel access up to a good amount (I think about 64 or so) The boxes were an Ultra 1, and a Sparc 20. The Sparc 20 / Linux was running Solaris and Linux in dual boot, and the linux performance on it was astounding, Solaris was painfully slow at everything. On the Ultra 1, the race was much closer, i.e. Linux was about 80% faster than solaris. I think we were running whatever the latest production quality versions of RedHat (6.2, the last Sparc version) and solaris at the time. note that we're talking about a year and a half or so ago, so things may have changed under both OSes. > > The issue in the past was mainly that Solaris had a brain damaged > >sort() call that was very slow when it had a lot of duplicate keys. That > >has since been updated by Sun. > > > > > Hmm - I ran into that situation with qsort() and many equal keys (see > Hackers thread "Solaris Performance"). Sadly Sun have not amended that > situation at all (in Solaris 8 anyway). Actually, I thought there was patch out there somewhere. Anyone else know?
scott.marlowe wrote: > note that we're talking about a year and a half or so ago, so things may > have changed under both OSes. > > > > The issue in the past was mainly that Solaris had a brain damaged > > >sort() call that was very slow when it had a lot of duplicate keys. That > > >has since been updated by Sun. > > > > > > > > Hmm - I ran into that situation with qsort() and many equal keys (see > > Hackers thread "Solaris Performance"). Sadly Sun have not amended that > > situation at all (in Solaris 8 anyway). > > Actually, I thought there was patch out there somewhere. Anyone else > know? qsort() patch is in PostgreSQL 7.3. We used the FreeBSD one for Solaris. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania 19073
scott.marlowe wrote: >Actually, I thought there was patch out there somewhere. Anyone else >know? > > > It looks like Solaris 9 has this included in a libc patch: http://sunsolve.sun.com/pub-cgi/retrieve.pl?doc=fpatches/112874 However I think the solution chosen by the Postgresql developers - using qsort.c from BSD is the best one, as it *sorts* all the Solaris releases (its in your Pg source : src/port/qsort.c). In fact I wonder if using the BSD qsort for *all* platforms might be a good idea - it would provide cross platform consistency and possibly better performance (e.g. It was several times quicker than glibc qsort, when I checked this on Linux last year.... ) best wishes Mark
Andrew Sullivan wrote: >We see from truss, however, >that a significant amount of timee is spent in the SYSV system >calls, so it appears that there is some kind of inefficiency there. > It would be interesting to run a profiling enabled postgres binary - altho that's not really nice on a production system :-). > >in any case, the problems aren't reproducable at will. > > The best ones never seem to be - means you can't try the profiling in a safe environment....:-( regards Mark
Mark Kirkwood wrote: > scott.marlowe wrote: > > >Actually, I thought there was patch out there somewhere. Anyone else > >know? > > > > > > > It looks like Solaris 9 has this included in a libc patch: > http://sunsolve.sun.com/pub-cgi/retrieve.pl?doc=fpatches/112874 > > However I think the solution chosen by the Postgresql developers - > using qsort.c from BSD is the best one, as it *sorts* all the Solaris > releases (its in your Pg source : src/port/qsort.c). > > In fact I wonder if using the BSD qsort for *all* platforms might be a > good idea - it would provide cross platform consistency and possibly > better performance (e.g. It was several times quicker than glibc qsort, > when I checked this on Linux last year.... ) Replacing qsort() on all platforms has a "we know better than the OS" feel to it that we try to avoid. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania 19073
Bruce Momjian <pgman@candle.pha.pa.us> writes:
> Mark Kirkwood wrote:
>> In fact I wonder if using the BSD qsort for *all* platforms might be a
>> good idea - it would provide cross platform consistency and possibly
>> better performance (e.g. It was several times quicker than glibc qsort,
>> when I checked this on Linux last year.... )
> Replacing qsort() on all platforms has a "we know better than the OS"
> feel to it that we try to avoid.
I agree on that --- but when it's provable that we do know better than a
*particular* OS, dropping in the BSD qsort seems like an easy win. Can
anyone back up Mark's finding that the BSD qsort is quicker than glibc's?
regards, tom lane
In article <26779.1051629468@sss.pgh.pa.us>,
Tom Lane <tgl@sss.pgh.pa.us> wrote:
>I agree on that --- but when it's provable that we do know better than a
>*particular* OS, dropping in the BSD qsort seems like an easy win. Can
>anyone back up Mark's finding that the BSD qsort is quicker than glibc's?
Better yet: Anyway of running performance tests from configure?
Would a simple counter in the compare function be sufficient to determine
the speed?
mrc
--
Mike Castle dalgoda@ix.netcom.com www.netcom.com/~dalgoda/
We are all of us living in the shadow of Manhattan. -- Watchmen
fatal ("You are in a maze of twisty compiler features, all different"); -- gcc
dalgoda@ix.netcom.com (Mike Castle) writes:
> Better yet: Anyway of running performance tests from configure?
Peter will object to that because of cross-compilation issues; and I'll
object because I run configure often enough that I don't want it to take
the time that would be needed for a reliable performance test ...
regards, tom lane
In article <4896.1051644271@sss.pgh.pa.us>,
Tom Lane <tgl@sss.pgh.pa.us> wrote:
>dalgoda@ix.netcom.com (Mike Castle) writes:
>> Better yet: Anyway of running performance tests from configure?
>
>Peter will object to that because of cross-compilation issues; and I'll
>object because I run configure often enough that I don't want it to take
>the time that would be needed for a reliable performance test ...
Understandable.
What about --with-pg-qsort (that defaults to use for currently known
systems) with a test program people could run if they want?
In that case, would counting the calls to the compare function be the
appropriate measurement (I'd think either wall or system time would vary
too widely).
mrc
--
Mike Castle dalgoda@ix.netcom.com www.netcom.com/~dalgoda/
We are all of us living in the shadow of Manhattan. -- Watchmen
fatal ("You are in a maze of twisty compiler features, all different"); -- gcc
Mike Castle wrote: > In article <4896.1051644271@sss.pgh.pa.us>, > Tom Lane <tgl@sss.pgh.pa.us> wrote: > >dalgoda@ix.netcom.com (Mike Castle) writes: > >> Better yet: Anyway of running performance tests from configure? > > > >Peter will object to that because of cross-compilation issues; and I'll > >object because I run configure often enough that I don't want it to take > >the time that would be needed for a reliable performance test ... > > > Understandable. > > What about --with-pg-qsort (that defaults to use for currently known > systems) with a test program people could run if they want? Let's have folks run a test program and get the results for some OS's. I would prefer to get some results before moving to a formalized option. > In that case, would counting the calls to the compare function be the > appropriate measurement (I'd think either wall or system time would vary > too widely). No. It is not calls the compare function, but total time in the qsort routine that has to be measured. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania 19073
Bruce Momjian wrote:
>Let's have folks run a test program and get the results for some OS's.
>I would prefer to get some results before moving to a formalized option.
>
>
>
I retested Linux 2.4.19 (Mandrake 9) using a simple "many equal keys"
program (see below) compiled with and without BSD qsort.c from Pg source.
$ time ./testbsdqsort
0.80user 0.02system 0:00.83elapsed 97%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (69major+991minor)pagefaults 0swaps
$ time ./testqsort
2.08user 0.04system 0:02.13elapsed 99%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (105major+1967minor)pagefaults 0swaps
So for the many equal keys, BSD qsort is significantly faster.
Clearly there are other test scenarios to examine, but the many equal
keys case is important.
cheers
Mark
-- the program...
#include <stdio.h>
#include <stdlib.h>
const int elements = 1000000;
const int mod = 100;
int sortfunc( const void* n1, const void* n2 ) {
int ret;
ret = (*(int*)n1) -(*(int*)n2) ;
return ret;
}
int main(int argc, char** argv ) {
int array[elements];
int i;
for ( i = 0; i < elements; i++ ) {
array[i] = i % mod;
}
qsort( array, elements, sizeof(int),sortfunc);
return 0;
}
-- end
On Wed, Apr 30, 2003 at 01:26:05PM +1200, Mark Kirkwood wrote: > So for the many equal keys, BSD qsort is significantly faster. > > Clearly there are other test scenarios to examine, but the many equal > keys case is important. I just whipped up a really simply test package (attached, 3KB). Just unpack it and run make. Anyway, on a Debian Woody machine here I get: 10^6 items, mod = 100 Value (i = index) BSD GLIBC (seconds) random() * mod 1.58 2.29 i % mod 0.52 1.69 i / (ITEMS / mod) 0.38 1.09 i ^ 0x5555555 1.18 1.67 It would seem to support the theory that the BSD qsort is faster than the Glibc one. The results seem quite repeatable (repeats give +/- 5%). -- Martijn van Oosterhout <kleptog@svana.org> http://svana.org/kleptog/ > "the West won the world not by the superiority of its ideas or values or > religion but rather by its superiority in applying organized violence. > Westerners often forget this fact, non-Westerners never do." > - Samuel P. Huntington
Attachment
dalgoda@ix.netcom.com (Mike Castle) writes:
> What about --with-pg-qsort (that defaults to use for currently known
> systems) with a test program people could run if they want?
I'd lean to "--with-pg-qsort" forcing the BSD qsort, "--without-pg-qsort"
forcing the native qsort, and if you don't say either then you get a
choice based on which OS you are running. (Not sure how hard this is
to do in the autoconf structure, but if we can do it, it seems like the
right user interface.)
> In that case, would counting the calls to the compare function be the
> appropriate measurement (I'd think either wall or system time would vary
> too widely).
I'd trust overall time measurements way more than call counts.
Obviously it's up to the user to hold overall system load and suchlike
outside factors constant while conducting the tests --- but measuring
any single component like comparison-function calls is just asking to be
misled.
regards, tom lane
Martijn van Oosterhout wrote: > Anyway, on a Debian Woody machine here I get: > >10^6 items, mod = 100 > >Value (i = index) BSD GLIBC (seconds) >random() * mod 1.58 2.29 >i % mod 0.52 1.69 >i / (ITEMS / mod) 0.38 1.09 >i ^ 0x5555555 1.18 1.67 > >It would seem to support the theory that the BSD qsort is faster than the >Glibc one. The results seem quite repeatable (repeats give +/- 5%). > Martin's test on two Linux machines here gives : 10^6 items, mod = 100 1) Mandrake 9 (x86 - Linux 2.4.19, glibc 2.2.5, gcc 3.2) Value (i = index) BSD GLIBC (seconds) random() * mod 2.20 2.70 i % mod 0.80 2.10 i / (ITEMS / mod) 0.53 1.28 i ^ 0x5555555 1.64 1.95 2) Yellowdog 2.3 (ppc - Linux 2.4.19, glibc 2.2.5. gcc 2.95.4) Value (i = index) BSD GLIBC (seconds) random() * mod 1.74 3.12 i % mod 0.69 2.65 i / (ITEMS / mod) 0.55 1.66 i ^ 0x5555555 1.42 2.45 Interestingly, the the ppc machine gets more improvement from the BSD qsort than the x86 one does. regards Mark
More test values :) On a RH 8.0 i686 box with a custom 2.4.20 kernel, with gcc 3.2 20020903 (Red Hat Linux 8.0 3.2-7) and with glibc 2.2.93 Value (i = index) BSD GLIBC random() * mod 0.53 0.78 i % mod 0.15 0.64 i / (ITEMS / mod) 0.10 0.35 i ^ 0x5555555 0.39 0.54 -- Jean-Christian Imbeault
I prevailed on some friends to try out a non Redhat distro plus HPUX 10.20 : 1 United Linux 1.0 (x86 Linux 2.4.19 ) Value (i = index) BSD GLIBC (seconds) random() * mod 0.35 0.56 i % mod 0.14 0.41 i / (ITEMS / mod) 0.10 0.27 i ^ 0x5555555 0.26 0.42 2 HPUX 10.20 (pa-risc1.1 D380 ) Value (i = index) BSD HPUX (seconds) random() * mod 8.3 14.0 i % mod 2.9 4.9 i / (ITEMS / mod) 2.4 2.9 i ^ 0x5555555 7.8 13.8 The HPUX results are interesting ! regards Mark
The trend has been approx:
On Linux
BSD qsort is 30-55% faster on average.
On Non Liux
BSD qsor it 50-90% on average.
This kind of says that Linux is doing something right in the kernal,
outdoing mainframe software in this sort of thing.
Mark Kirkwood wrote:
>
> I prevailed on some friends to try out a non Redhat distro plus HPUX 10.20 :
>
> 1 United Linux 1.0 (x86 Linux 2.4.19 )
>
> Value (i = index) BSD GLIBC (seconds)
> random() * mod 0.35 0.56
> i % mod 0.14 0.41
> i / (ITEMS / mod) 0.10 0.27
> i ^ 0x5555555 0.26 0.42
>
> 2 HPUX 10.20 (pa-risc1.1 D380 )
>
> Value (i = index) BSD HPUX (seconds)
> random() * mod 8.3 14.0
> i % mod 2.9 4.9
> i / (ITEMS / mod) 2.4 2.9
> i ^ 0x5555555 7.8 13.8
>
> The HPUX results are interesting !
>
> regards
>
> Mark
>
> ---------------------------(end of broadcast)---------------------------
> TIP 2: you can get off all lists at once with the unregister command
> (send "unregister YourEmailAddressHere" to majordomo@postgresql.org)
Mark Kirkwood <markir@paradise.net.nz> writes:
> The HPUX results are interesting !
I can confirm the results for the i % mod case on my HPUX 10.20 system
(I get about 2.65 sec vs 4.40 sec) and also the i ^ 0x5555555 case
(about 7.25 vs 12.85 sec). Didn't try the other cases...
regards, tom lane
Dennis Gearon wrote: >The trend has been approx: > > On Linux > BSD qsort is 30-55% faster on average. > On Non Liux > BSD qsort it 50-90% on average. > >This kind of says that Linux is doing something right in the kernal, >outdoing mainframe software in this sort of thing. > The difference may be due to GNU and HP using different qsort algorithm tweeks. Consider : + HPUX vs BSD - very different for random*mod, close for i / (ITEMS / mod). + Linux vs BSD - reasonably different for all 4 cases. (Only way to tell would be to eyeball HP's qsort code...I am not sure if thats possible) regards Mark
In article <19660.1051676560@sss.pgh.pa.us>,
Tom Lane <tgl@sss.pgh.pa.us> wrote:
>dalgoda@ix.netcom.com (Mike Castle) writes:
>> What about --with-pg-qsort (that defaults to use for currently known
>> systems) with a test program people could run if they want?
>
>I'd lean to "--with-pg-qsort" forcing the BSD qsort, "--without-pg-qsort"
>forcing the native qsort, and if you don't say either then you get a
>choice based on which OS you are running. (Not sure how hard this is
>to do in the autoconf structure, but if we can do it, it seems like the
>right user interface.)
It's fairly easy, actually.
Something like:
AC_ARG_WITH([pg-qsort],
AC_HELP_STRING([--with-pg-qsort],
[use internal qsort (default is system
dependent)]),
case $withval
yes) force internal;;
no) force system;;
*) tell user to don't do that (--with-pg-qsort=blah);;
esac,
case OS
*Solaris*) force internal;;
*) force system;;
esac
)
>> In that case, would counting the calls to the compare function be the
>> appropriate measurement (I'd think either wall or system time would vary
>> too widely).
>
>I'd trust overall time measurements way more than call counts.
>Obviously it's up to the user to hold overall system load and suchlike
>outside factors constant while conducting the tests --- but measuring
>any single component like comparison-function calls is just asking to be
>misled.
I'm not certain.
I ran the tests posted to this lists with a few mods and got some very
interesting results.
First I added a counter to the compare function, and the most cases, the
glibc implementation was called significantly less often than the BSD
compare function.
In a simple test function, like comparing two ints, then yes, the BSD
implementation was faster. But in a more complex function, say comparing
strings, often times the glibc version was faster. Why? Because the
time spent in the compare function became the overwhelming factor.
So, I ask you this: in places where qsort is used in PG, is it more likely
to be used for simple comparisons or for complex comparisons that involve a
mixture of strings and ints? And, is it more likely to be called with
datasets that are partially sorted or not?
BSD prevailed on trivial comparisons and on sets that had any type of
sorting to them. Glibc prevailed on random order with heavier comparisons.
mrc
--
Mike Castle dalgoda@ix.netcom.com www.netcom.com/~dalgoda/
We are all of us living in the shadow of Manhattan. -- Watchmen
fatal ("You are in a maze of twisty compiler features, all different"); -- gcc
dalgoda@ix.netcom.com (Mike Castle) writes:
> In a simple test function, like comparing two ints, then yes, the BSD
> implementation was faster. But in a more complex function, say comparing
> strings, often times the glibc version was faster. Why? Because the
> time spent in the compare function became the overwhelming factor.
Interesting.
> So, I ask you this: in places where qsort is used in PG, is it more likely
> to be used for simple comparisons or for complex comparisons that involve a
> mixture of strings and ints?
The only place that I think performance is likely to matter for is the
calls in tuplesort.c, which are expensive --- even if you end up calling
something as cheap as btint4cmp, there's a lot of overhead in between.
(I seem to recall hearing that Oracle expends effort trying to "compile"
code for tuple comparisons, but we don't ... yet ...)
And, is it more likely to be called with
> datasets that are partially sorted or not?
I think that one's unanswerable. It'd depend on the workload.
regards, tom lane
Mike Castle wrote: >First I added a counter to the compare function, and the most cases, the >glibc implementation was called significantly less often than the BSD >compare function. > > Interesting - all examples I tried had glibc compare count higher, do you want to post one of yours? >In a simple test function, like comparing two ints, then yes, the BSD >implementation was faster. But in a more complex function, say comparing >strings, often times the glibc version was faster. Why? Because the >time spent in the compare function became the overwhelming factor. > > Considering strings is a good point - I went away and tried some examples, unfortunately I only managed to see BSD faster, but the difference was not as large as in the integer tests. regards Mark
I assume we decided that BSD sort isn't fast enough to warrant replacing the native qsort. --------------------------------------------------------------------------- Mark Kirkwood wrote: > Mike Castle wrote: > > >First I added a counter to the compare function, and the most cases, the > >glibc implementation was called significantly less often than the BSD > >compare function. > > > > > Interesting - all examples I tried had glibc compare count higher, do > you want to post one of yours? > > >In a simple test function, like comparing two ints, then yes, the BSD > >implementation was faster. But in a more complex function, say comparing > >strings, often times the glibc version was faster. Why? Because the > >time spent in the compare function became the overwhelming factor. > > > > > Considering strings is a good point - I went away and tried some > examples, unfortunately I only managed to see BSD faster, but the > difference was not as large as in the integer tests. > > > regards > > Mark > > > ---------------------------(end of broadcast)--------------------------- > TIP 6: Have you searched our list archives? > > http://archives.postgresql.org > -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania 19073
That is probably the most reasonable conclusion to draw from the results ;-) cheers Mark Bruce Momjian wrote: >I assume we decided that BSD sort isn't fast enough to warrant replacing >the native qsort. > >--------------------------------------------------------------------------- > >Mark Kirkwood wrote: > > >>Mike Castle wrote: >> >> >> >>>First I added a counter to the compare function, and the most cases, the >>>glibc implementation was called significantly less often than the BSD >>>compare function. >>> >>> >>> >>> >>Interesting - all examples I tried had glibc compare count higher, do >>you want to post one of yours? >> >> >> >>>In a simple test function, like comparing two ints, then yes, the BSD >>>implementation was faster. But in a more complex function, say comparing >>>strings, often times the glibc version was faster. Why? Because the >>>time spent in the compare function became the overwhelming factor. >>> >>> >>> >>> >>Considering strings is a good point - I went away and tried some >>examples, unfortunately I only managed to see BSD faster, but the >>difference was not as large as in the integer tests. >> >> >>regards >> >>Mark >> >> >>---------------------------(end of broadcast)--------------------------- >>TIP 6: Have you searched our list archives? >> >>http://archives.postgresql.org >> >> >> > > >