Thread: Re: [GENERAL] Upgrade to dual processor machine?

Re: [GENERAL] Upgrade to dual processor machine?

From
"Henrik Steffen"
Date:
hi Ivan

to get the shared buffer memory I followed all the instructions
I gathered here on the list within the last two days. the kernel
settings SHMMAX etc. were important here in my opinion... you could
search the archives for all the other mails within this thread and
try yourself.

by the way: today we update to kernel 2.4.19 and we measured BIG
performance gains! however, since the upgrade 'top' doesn't show any
shared memory in the summary any longer... yet for every process
it lists a certain amount of shared mem... is this a kernel/top issue
or did I miss something here?

the kernel is much more performant!

--

Mit freundlichem Gruß

Henrik Steffen
Geschäftsführer

top concepts Internetmarketing GmbH
Am Steinkamp 7 - D-21684 Stade - Germany
--------------------------------------------------------
http://www.topconcepts.com          Tel. +49 4141 991230
mail: steffen@topconcepts.com       Fax. +49 4141 991233
--------------------------------------------------------
24h-Support Hotline:  +49 1908 34697 (EUR 1.86/Min,topc)
--------------------------------------------------------
Ihr SMS-Gateway: JETZT NEU unter: http://sms.city-map.de
System-Partner gesucht: http://www.franchise.city-map.de
--------------------------------------------------------
Handelsregister: AG Stade HRB 5811 - UstId: DE 213645563
--------------------------------------------------------

----- Original Message -----
From: "pginfo" <pginfo@t1.unisoftbg.com>
To: "Henrik Steffen" <steffen@city-map.de>
Sent: Thursday, November 14, 2002 3:15 PM
Subject: Re: [GENERAL] Upgrade to dual processor machine?


> Hi,
> Sorry for this question.
> I see you have 8520K shrd .
> How are you setup you linux box to use tis shared buffers? (any answer will be great).
>
>
>
> I have tryed it many times without success.
>
> Also it is courios that the sum of cpu loading is not 100% !
>
> We are using two pg servers:
> one single processor Intel 1 GHz, 1 GB RAM,
> and one dual Intel 1GHz, 1.5 GB RAM.
> It exist big diference in pg performance and I noticed many times when the system use
> all two processors.
>
> If I can help you with more info you are free to ask.
>
> regards,
> Ivan.
>
>
> Henrik Steffen wrote:
>
> > dear shridhar,
> >
> > > Yes. 2*max connection is minimum. Anything additional is always welcome as long
> > > as it does not starve the system.
> >
> > ok, I tried to set shared_buffers to 65535 now. but then restarting postgres
> > fails - it says:
> >
> > IpcMemoryCreate: shmget(key=5432001, size=545333248, 03600) failed: Invalid argument
> >
> > and a message telling me to either lower the shared_buffers or raise the
> > SHMMAX.
> >
> > > If you have a gig of memory and shared buffers are 536MB as you have indicated,
> > > who is taking rest of the RAM?
> >
> > well, I guess it's postgres... see the output of top below:
> >
> >  11:06am  up 1 day, 16:46,  1 user,  load average: 1,32, 1,12, 1,22
> > 53 processes: 52 sleeping, 1 running, 0 zombie, 0 stopped
> > CPU states: 24,5% user, 11,2% system,  0,0% nice,  5,6% idle
> > Mem:  1020808K av, 1006156K used,   14652K free,    8520K shrd,   37204K buff
> > Swap: 1028112K av,      60K used, 1028052K free                  849776K cached
> >
> >   PID USER     PRI  NI  SIZE  RSS SHARE STAT %CPU %MEM   TIME COMMAND
> > 10678 root      19   0  2184 2184  1584 S     2,9  0,2   0:00 sendmail
> >     1 root       8   0   520  520   452 S     0,0  0,0   0:03 init
> >     2 root       9   0     0    0     0 SW    0,0  0,0   0:00 keventd
> >     3 root       9   0     0    0     0 SW    0,0  0,0   0:00 kapm-idled
> >     4 root      19  19     0    0     0 SWN   0,0  0,0   0:00 ksoftirqd_CPU0
> >     5 root       9   0     0    0     0 SW    0,0  0,0   0:28 kswapd
> >     6 root       9   0     0    0     0 SW    0,0  0,0   0:00 kreclaimd
> >     7 root       9   0     0    0     0 SW    0,0  0,0   0:09 bdflush
> >     8 root       9   0     0    0     0 SW    0,0  0,0   0:00 kupdated
> >     9 root      -1 -20     0    0     0 SW<   0,0  0,0   0:00 mdrecoveryd
> >    13 root       9   0     0    0     0 SW    0,0  0,0   0:00 kjournald
> >   136 root       9   0     0    0     0 SW    0,0  0,0   0:00 kjournald
> >   137 root       9   0     0    0     0 SW    0,0  0,0   0:00 kjournald
> >   138 root       9   0     0    0     0 SW    0,0  0,0   0:00 kjournald
> >   139 root       9   0     0    0     0 SW    0,0  0,0   0:00 kjournald
> >   140 root       9   0     0    0     0 SW    0,0  0,0   2:16 kjournald
> >   378 root       9   0     0    0     0 SW    0,0  0,0   0:00 eth0
> >   454 root       9   0   572  572   476 S     0,0  0,0   0:00 syslogd
> >   459 root       9   0  1044 1044   392 S     0,0  0,1   0:00 klogd
> >   572 root       8   0  1128 1092   968 S     0,0  0,1   0:07 sshd
> >   584 root       9   0  1056 1056   848 S     0,0  0,1   0:02 nlservd
> >   611 root       8   0  1836 1820  1288 S     0,0  0,1   0:00 sendmail
> >   693 root       9   0   640  640   556 S     0,0  0,0   0:00 crond
> >   729 daemon     9   0   472  464   404 S     0,0  0,0   0:00 atd
> >   736 root       9   0   448  448   384 S     0,0  0,0   0:00 mingetty
> >   737 root       9   0   448  448   384 S     0,0  0,0   0:00 mingetty
> >   738 root       9   0   448  448   384 S     0,0  0,0   0:00 mingetty
> >   739 root       9   0   448  448   384 S     0,0  0,0   0:00 mingetty
> >   740 root       9   0   448  448   384 S     0,0  0,0   0:00 mingetty
> >   741 root       9   0   448  448   384 S     0,0  0,0   0:00 mingetty
> >  9800 root       9   0  1888 1864  1552 S     0,0  0,1   0:02 sshd
> >  9801 root      16   0  1368 1368  1016 S     0,0  0,1   0:00 bash
> > 10574 postgres   0   0  1448 1448  1380 S     0,0  0,1   0:00 postmaster
> > 10576 postgres   9   0  1436 1436  1388 S     0,0  0,1   0:00 postmaster
> > 10577 postgres   9   0  1480 1480  1388 S     0,0  0,1   0:00 postmaster
> > 10579 postgres  14   0 11500  11M 10324 S     0,0  1,1   0:08 postmaster
> > 10580 postgres   9   0 11672  11M 10328 S     0,0  1,1   0:03 postmaster
> > 10581 postgres  14   0 11620  11M 10352 S     0,0  1,1   0:08 postmaster
> > 10585 postgres  11   0 11560  11M 10304 S     0,0  1,1   0:08 postmaster
> > 10588 postgres   9   0 11520  11M 10316 S     0,0  1,1   0:14 postmaster
> > 10589 postgres   9   0 11632  11M 10324 S     0,0  1,1   0:06 postmaster
> > 10590 postgres  10   0 11620  11M 10320 S     0,0  1,1   0:06 postmaster
> > 10591 postgres   9   0 11536  11M 10320 S     0,0  1,1   0:08 postmaster
> > 10592 postgres  11   0 11508  11M 10316 S     0,0  1,1   0:04 postmaster
> > 10595 postgres   9   0 11644  11M 10324 S     0,0  1,1   0:03 postmaster
> > 10596 postgres  11   0 11664  11M 10328 S     0,0  1,1   0:08 postmaster
> > 10597 postgres   9   0 11736  11M 10340 S     0,0  1,1   0:24 postmaster
> > 10598 postgres   9   0 11500  11M 10312 S     0,0  1,1   0:10 postmaster
> > 10599 postgres  11   0 11676  11M 10324 S     0,0  1,1   0:13 postmaster
> > 10602 postgres   9   0 11476  11M 10308 S     0,0  1,1   0:09 postmaster
> > 10652 postgres   9   0  7840 7840  7020 S     0,0  0,7   0:00 postmaster
> > 10669 postgres   9   0  9076 9076  8224 S     0,0  0,8   0:00 postmaster
> > 10677 root      13   0  1032 1028   828 R     0,0  0,1   0:00 top
> >
> > I have now changed the SHMMAX settings to 545333248 and changed the
> > shared_buffers to 65535 again. now postgres starts up correctly.
> >
> > the top result changes to:
> >
> >  11:40am  up 1 day, 17:20,  1 user,  load average: 2,24, 2,51, 2,14
> > 57 processes: 55 sleeping, 2 running, 0 zombie, 0 stopped
> > CPU states: 24,7% user, 11,3% system,  0,0% nice,  6,2% idle
> > Mem:  1020808K av, 1015844K used,    4964K free,  531420K shrd,   24796K buff
> > Swap: 1028112K av,      60K used, 1028052K free                  338376K cached
> >
> >   PID USER     PRI  NI  SIZE  RSS SHARE STAT %CPU %MEM   TIME COMMAND
> > 11010 root      17   0  1036 1032   828 R    14,2  0,1   0:00 top
> > 11007 postgres  14   0 14268  13M 12668 R     9,7  1,3   0:00 postmaster
> > 11011 root       9   0  2184 2184  1584 S     3,0  0,2   0:00 sendmail
> >     1 root       8   0   520  520   452 S     0,0  0,0   0:03 init
> >     2 root       9   0     0    0     0 SW    0,0  0,0   0:00 keventd
> >     3 root       9   0     0    0     0 SW    0,0  0,0   0:00 kapm-idled
> >     4 root      19  19     0    0     0 SWN   0,0  0,0   0:00 ksoftirqd_CPU0
> >     5 root       9   0     0    0     0 SW    0,0  0,0   0:29 kswapd
> >     6 root       9   0     0    0     0 SW    0,0  0,0   0:00 kreclaimd
> >     7 root       9   0     0    0     0 SW    0,0  0,0   0:09 bdflush
> >     8 root       9   0     0    0     0 SW    0,0  0,0   0:00 kupdated
> >     9 root      -1 -20     0    0     0 SW<   0,0  0,0   0:00 mdrecoveryd
> >    13 root       9   0     0    0     0 SW    0,0  0,0   0:00 kjournald
> >   136 root       9   0     0    0     0 SW    0,0  0,0   0:00 kjournald
> >   137 root       9   0     0    0     0 SW    0,0  0,0   0:00 kjournald
> >   138 root       9   0     0    0     0 SW    0,0  0,0   0:00 kjournald
> >   139 root       9   0     0    0     0 SW    0,0  0,0   0:00 kjournald
> >   140 root       9   0     0    0     0 SW    0,0  0,0   2:18 kjournald
> >   378 root       9   0     0    0     0 SW    0,0  0,0   0:00 eth0
> >   454 root       9   0   572  572   476 S     0,0  0,0   0:00 syslogd
> >   459 root       9   0  1044 1044   392 S     0,0  0,1   0:00 klogd
> >   572 root       8   0  1128 1092   968 S     0,0  0,1   0:07 sshd
> >   584 root       9   0  1056 1056   848 S     0,0  0,1   0:02 nlservd
> >   611 root       9   0  1836 1820  1288 S     0,0  0,1   0:00 sendmail
> >   693 root       9   0   640  640   556 S     0,0  0,0   0:00 crond
> >   729 daemon     9   0   472  464   404 S     0,0  0,0   0:00 atd
> >   736 root       9   0   448  448   384 S     0,0  0,0   0:00 mingetty
> >   737 root       9   0   448  448   384 S     0,0  0,0   0:00 mingetty
> >   738 root       9   0   448  448   384 S     0,0  0,0   0:00 mingetty
> >   739 root       9   0   448  448   384 S     0,0  0,0   0:00 mingetty
> >   740 root       9   0   448  448   384 S     0,0  0,0   0:00 mingetty
> >   741 root       9   0   448  448   384 S     0,0  0,0   0:00 mingetty
> >  9800 root       9   0  1888 1864  1552 S     0,0  0,1   0:03 sshd
> >  9801 root      10   0  1368 1368  1016 S     0,0  0,1   0:00 bash
> > 10838 postgres   7   0  6992 6992  6924 S     0,0  0,6   0:00 postmaster
> > 10840 postgres   9   0  6984 6984  6932 S     0,0  0,6   0:00 postmaster
> > 10841 postgres   9   0  7024 7024  6932 S     0,0  0,6   0:00 postmaster
> > 10852 postgres   9   0  489M 489M  487M S     0,0 49,0   0:32 postmaster
> > 10869 postgres   9   0  357M 357M  356M S     0,0 35,8   0:21 postmaster
> > 10908 postgres   9   0  263M 263M  262M S     0,0 26,4   0:20 postmaster
> > 10909 postgres   9   0  283M 283M  281M S     0,0 28,4   0:19 postmaster
> > 10932 postgres   9   0  288M 288M  286M S     0,0 28,9   0:13 postmaster
> > 10946 postgres   9   0  213M 213M  211M S     0,0 21,4   0:06 postmaster
> > 10947 postgres   9   0  239M 239M  238M S     0,0 24,0   0:07 postmaster
> > 10948 postgres   9   0  292M 292M  290M S     0,0 29,2   0:09 postmaster
> > 10957 postgres   9   0  214M 214M  212M S     0,0 21,5   0:10 postmaster
> > 10964 postgres   9   0 58156  56M 56400 S     0,0  5,6   0:05 postmaster
> > 10974 postgres   9   0 50860  49M 49120 S     0,0  4,9   0:04 postmaster
> > 10975 postgres   9   0  209M 209M  207M S     0,0 21,0   0:04 postmaster
> > 10976 postgres   9   0  174M 174M  172M S     0,0 17,5   0:08 postmaster
> > 10977 postgres   9   0 52484  51M 50932 S     0,0  5,1   0:05 postmaster
> > 10990 postgres   9   0  199M 199M  197M S     0,0 19,9   0:06 postmaster
> > 10993 postgres   9   0  141M 141M  139M S     0,0 14,1   0:01 postmaster
> > 10998 postgres   9   0  181M 181M  180M S     0,0 18,2   0:04 postmaster
> > 10999 postgres   9   0  139M 139M  138M S     0,0 14,0   0:01 postmaster
> > 11001 postgres   9   0 45484  44M 43948 S     0,0  4,4   0:01 postmaster
> > 11006 postgres   9   0 15276  14M 13952 S     0,0  1,4   0:00 postmaster
> >
> > now, does this look better in your eyes?
> >
> > > What are your current settings? Could you please repost. I lost earlier
> > > thread(Sorry for that.. Had a HDD meltdown here couple of days back. Lost few
> > > mails..)
> >
> > do you need more information here?
> >
> > ---------------------------(end of broadcast)---------------------------
> > TIP 6: Have you searched our list archives?
> >
> > http://archives.postgresql.org
>
>
>


Re: [GENERAL] Upgrade to dual processor machine?

From
"Henrik Steffen"
Date:
hi,

this is what it look like right now... looks like 69 MB of shared memory...

------ Shared Memory Segments --------
key        shmid      owner      perms      Bytes      nattch     Status
0x0052e2c1 131072     postgres  600        69074944   19

------ Semaphore Arrays --------
key        semid      owner      perms      nsems      Status
0x0052e2c1 655360     postgres  600        17
0x0052e2c2 688129     postgres  600        17
0x0052e2c3 720898     postgres  600        17

------ Message Queues --------
key        msqid      owner      perms      used-bytes   messages

--

Mit freundlichem Gruß

Henrik Steffen
Geschäftsführer

top concepts Internetmarketing GmbH
Am Steinkamp 7 - D-21684 Stade - Germany
--------------------------------------------------------
http://www.topconcepts.com          Tel. +49 4141 991230
mail: steffen@topconcepts.com       Fax. +49 4141 991233
--------------------------------------------------------
24h-Support Hotline:  +49 1908 34697 (EUR 1.86/Min,topc)
--------------------------------------------------------
Ihr SMS-Gateway: JETZT NEU unter: http://sms.city-map.de
System-Partner gesucht: http://www.franchise.city-map.de
--------------------------------------------------------
Handelsregister: AG Stade HRB 5811 - UstId: DE 213645563
--------------------------------------------------------

----- Original Message -----
From: "Shridhar Daithankar" <shridhar_daithankar@persistent.co.in>
To: "Henrik Steffen" <steffen@city-map.de>
Sent: Thursday, November 14, 2002 4:25 PM
Subject: Re: [PERFORM] [GENERAL] Upgrade to dual processor machine?


> On Thursday 14 November 2002 08:50 pm, you wrote:
> > by the way: today we update to kernel 2.4.19 and we measured BIG
> > performance gains! however, since the upgrade 'top' doesn't show any
> > shared memory in the summary any longer... yet for every process
> > it lists a certain amount of shared mem... is this a kernel/top issue
> > or did I miss something here?
>
> No. The shared memory accounting is turned off because it is seemingly much
> complex. Process do share memory.  Check output of ipcs as root..
>
>  Shridhar


Re: [GENERAL] Upgrade to dual processor machine?

From
Shridhar Daithankar
Date:
On Thursday 14 November 2002 09:01 pm, you wrote:
> this is what it look like right now... looks like 69 MB of shared memory...
> ------ Shared Memory Segments --------
> key        shmid      owner      perms      Bytes      nattch     Status
> 0x0052e2c1 131072     postgres  600        69074944   19

Well, if you sample this figure for min/max/avg usage say for a day, you will
have sufficient idea as in what are your exact requirements are in terms of
shared buffers. I would say 5% more that would prove to be much more optimal
setting. IMO it's worth of experiement..

Just start out with pretty high to leave some room..

 Shridhar