Thread: PG 7.0.3 & RH 7 IPC problems?

PG 7.0.3 & RH 7 IPC problems?

From
DHSC Webmaster
Date:
I'm running 7.0.3 on a double pentium 733 RH 7 box with 512M ram.

We quickly ran up against the default 32 max. connection limit since it
is a backend DB for our web server. When trying to increase the buffers
and connections as advised in the docs, I got the error:

[postgres@atl01371 data]$ nohup postmaster -D /usr/local/pgsql/data -i
-B 128 -N 64 > /usr/local/pgsql/data/logs/log.txt
IpcMemoryCreate: shmget failed (Invalid argument) key=5432001,
size=2048000, permission=600
This type of error is usually caused by an improper
shared memory or System V IPC semaphore configuration.
For more information, see the FAQ and platform-specific
FAQ's in the source directory pgsql/doc or on our
web site at http://www.postgresql.org.
FATAL 1:  ShmemCreate: cannot create region
[postgres@atl01371 data]$

Since Redhat says that version 7 supports IPC and we have increased
shared memory to 128 meg without solving the problem, does anyone have a
positive experience with this scenario to share?

Here is a partial list from sysctl -a
vm.kswapd = 512 32      32
vm.pagecache = 2        15      75
vm.buffermem = 2        10      60
vm.overcommit_memory = 0
vm.freepages = 256      512     768
kernel.sysrq = 0
kernel.shmall = 33554432
kernel.shmmax = 134217728
kernel.rtsig-max = 1024
kernel.rtsig-nr = 0
kernel.acct = 4 2       30
kernel.modprobe = /sbin/modprobe
kernel.printk = 6       4       1       7
kernel.ctrl-alt-del = 0
kernel.real-root-dev = 2054
kernel.panic = 0
kernel.version = #1 SMP Mon Feb 5 14:36:22 EST 2001
kernel.osrelease = 2.2.17-14smp

--
Bill MacArthur
Webmaster
DHS Club

Re: PG 7.0.3 & RH 7 IPC problems?

From
Tom Lane
Date:
DHSC Webmaster <webmaster@dhs-club.com> writes:
> [postgres@atl01371 data]$ nohup postmaster -D /usr/local/pgsql/data -i
> -B 128 -N 64 > /usr/local/pgsql/data/logs/log.txt
> IpcMemoryCreate: shmget failed (Invalid argument) key=5432001,
> size=2048000, permission=600

Not sure about Linux, but on HPUX there are two possible reasons for
shmget to return EINVAL:

          [EINVAL]       size is less than the system-imposed minimum or
                         greater than the system-imposed maximum.

          [EINVAL]       A shared memory identifier exists for key but the
                         size of the segment associated with it is less
                         than size and size is not equal to zero.

In other words, you could get this if there is an old segment of smaller
size and the same key still laying about, which there might well be if
you were foolish enough to kill -9 the old postmaster.  Use ipcs and
ipcrm to check and clean up.

7.1 uses a slightly different approach that hopefully avoids this
particular failure scenario...

            regards, tom lane

Re: PG 7.0.3 & RH 7 IPC problems?

From
Stefan Huber
Date:
<kill -9 postmaster_pid snipped>
>7.1 uses a slightly different approach that hopefully avoids this
>particular failure scenario...

Is there any information availible _how_ this is avoided? I'd be interested, and if nothing/noone else helps, I'll
studythe source :) 
But I'd be nice to point me to a document explaining the novelties or a short description.

Thanks,
Stefan
--
Atheism is a non-prophet organization.


Re: PG 7.0.3 & RH 7 IPC problems?

From
DHSC Webmaster
Date:
It would appear that either my icps is not working or that everything is
alright.
(don't know enough about what to expect to know!)
bash-2.04$ ipcs

------ Shared Memory Segments --------
key       shmid     owner     perms     bytes     nattch    status

------ Semaphore Arrays --------
key       semid     owner     perms     nsems     status

------ Message Queues --------
key       msqid     owner     perms     used-bytes  messages

bash-2.04$

Tom Lane wrote:
>
> DHSC Webmaster <webmaster@dhs-club.com> writes:
> > [postgres@atl01371 data]$ nohup postmaster -D /usr/local/pgsql/data -i
> > -B 128 -N 64 > /usr/local/pgsql/data/logs/log.txt
> > IpcMemoryCreate: shmget failed (Invalid argument) key=5432001,
> > size=2048000, permission=600
>
> Not sure about Linux, but on HPUX there are two possible reasons for
> shmget to return EINVAL:
>
>           [EINVAL]       size is less than the system-imposed minimum or
>                          greater than the system-imposed maximum.
>
>           [EINVAL]       A shared memory identifier exists for key but the
>                          size of the segment associated with it is less
>                          than size and size is not equal to zero.
>
> In other words, you could get this if there is an old segment of smaller
> size and the same key still laying about, which there might well be if
> you were foolish enough to kill -9 the old postmaster.  Use ipcs and
> ipcrm to check and clean up.
>
> 7.1 uses a slightly different approach that hopefully avoids this
> particular failure scenario...
>
>                         regards, tom lane
>
> ---------------------------(end of broadcast)---------------------------
> TIP 5: Have you checked our extensive FAQ?
>
> http://www.postgresql.org/users-lounge/docs/faq.html

--
Bill MacArthur
Webmaster
DHS Club

Re: PG 7.0.3 & RH 7 IPC problems?

From
Tom Lane
Date:
Stefan Huber <schweinsaug@crosswinds.net> writes:
> <kill -9 postmaster_pid snipped>
>> 7.1 uses a slightly different approach that hopefully avoids this
>> particular failure scenario...

> Is there any information availible _how_ this is avoided?

We try to remove and recreate, rather than just reuse, an existing
Postgres-owned segment.  Reuse worked fine as long as the segment
size was >= what we were asking for, but otherwise not so fine...

            regards, tom lane

Re: PG 7.0.3 & RH 7 IPC problems?

From
Tom Lane
Date:
DHSC Webmaster <webmaster@dhs-club.com> writes:
> It would appear that either my icps is not working or that everything is
> alright.

Try it as root.  I think on Linux, ipcs won't show shm segments that you
do not have read access to (a mighty braindead way to do it but...)

            regards, tom lane

Re: PG 7.0.3 & RH 7 IPC problems?

From
DHSC Webmaster
Date:
Thank you Tom. Here are the results.
[root@atl01371 linux]# ipcs

------ Shared Memory Segments --------
key       shmid     owner     perms     bytes     nattch    status
0x0052e2ca 36864     postgres  700       144       5
0x0052e2c1 26625     postgres  600       1104896   5
0x0052e2c7 37890     postgres  600       66060     5

------ Semaphore Arrays --------
key       semid     owner     perms     nsems     status
0x0052e2ce 6144      postgres  600       16
0x0052e2cf 6145      postgres  600       16

------ Message Queues --------
key       msqid     owner     perms     used-bytes  messages

These values don't seem to change from one moment to the next although
there are of course varying numbers of backends running at any given
moment.
Since I am unfamiliar with this output I hesitate to use icprm to remove
something which ought not be removed.
I have not used kill -9 on anything. :)

Tom Lane wrote:
>
> DHSC Webmaster <webmaster@dhs-club.com> writes:
> > It would appear that either my icps is not working or that everything is
> > alright.
>
> Try it as root.  I think on Linux, ipcs won't show shm segments that you
> do not have read access to (a mighty braindead way to do it but...)
>
>                         regards, tom lane

--
Bill MacArthur
Webmaster
DHS Club

Re: PG 7.0.3 & RH 7 IPC problems?

From
Tom Lane
Date:
DHSC Webmaster <webmaster@dhs-club.com> writes:
> [root@atl01371 linux]# ipcs

> ------ Shared Memory Segments --------
> key       shmid     owner     perms     bytes     nattch    status
> 0x0052e2ca 36864     postgres  700       144       5
> 0x0052e2c1 26625     postgres  600       1104896   5
> 0x0052e2c7 37890     postgres  600       66060     5

> ------ Semaphore Arrays --------
> key       semid     owner     perms     nsems     status
> 0x0052e2ce 6144      postgres  600       16
> 0x0052e2cf 6145      postgres  600       16

> ------ Message Queues --------
> key       msqid     owner     perms     used-bytes  messages

> These values don't seem to change from one moment to the next although
> there are of course varying numbers of backends running at any given
> moment.

These objects are created by the postmaster and persist as long as the
postmaster does.  They *should* go away when you shut down the
postmaster, but perhaps they didn't for some reason.

Anyway, I suggest (a) shut down postmaster, (b) use ipcs to make sure
there are no postgres shmem or sema objects (delete 'em with ipcrm if
necessary), and (c) try to start postmaster with new -B/-N values.

If step (c) fails when there are no pre-existing objects, then it must
be that you have not managed to increase the kernel's limits on shared
memory.  The exact incantations needed to accomplish that trick vary
across systems, so I can't offer a lot of help there.

            regards, tom lane

Re: PG 7.0.3 & RH 7 IPC problems?

From
Stefan Huber
Date:
At 13:26 2001-03-29 -0500, DHSC Webmaster wrote:
>Thank you Tom. Here are the results.
>[root@atl01371 linux]# ipcs
<output snipped>
>These values don't seem to change from one moment to the next although
>there are of course varying numbers of backends running at any given
>moment.

Did you issue ipcs when running Postgres? Then that's ok (I cannot tell you, if the sizes are ok, but at leaste the
existenceof sharedmemory and semaphores is alright). 
If you still have some of those IPCs left, after shutting down the postmaster, there's probably something wrong.

>Since I am unfamiliar with this output I hesitate to use icprm to remove
>something which ought not be removed.

good idea ;)

>I have not used kill -9 on anything. :)

Maybe you used killall, although this should normally issue SIGTERM (15), but who knows? Or maybe some script kills the
postmasterwith SIGKILL. Or maybe, the system just hung once. 

Stefan

--
Atheism is a non-prophet organization.


Re: PG 7.0.3 & RH 7 IPC problems?

From
DHSC Webmaster
Date:
Gracias, merci' & Thank You Tom!
After shutting down the entries were still there.
I rebooted and restarted and voila!, we're in business.

Tom Lane wrote:
>
> DHSC Webmaster <webmaster@dhs-club.com> writes:
> > [root@atl01371 linux]# ipcs
>
> > ------ Shared Memory Segments --------
> > key       shmid     owner     perms     bytes     nattch    status
> > 0x0052e2ca 36864     postgres  700       144       5
> > 0x0052e2c1 26625     postgres  600       1104896   5
> > 0x0052e2c7 37890     postgres  600       66060     5
>
> > ------ Semaphore Arrays --------
> > key       semid     owner     perms     nsems     status
> > 0x0052e2ce 6144      postgres  600       16
> > 0x0052e2cf 6145      postgres  600       16
>
> > ------ Message Queues --------
> > key       msqid     owner     perms     used-bytes  messages
>
> > These values don't seem to change from one moment to the next although
> > there are of course varying numbers of backends running at any given
> > moment.
>
> These objects are created by the postmaster and persist as long as the
> postmaster does.  They *should* go away when you shut down the
> postmaster, but perhaps they didn't for some reason.
>
> Anyway, I suggest (a) shut down postmaster, (b) use ipcs to make sure
> there are no postgres shmem or sema objects (delete 'em with ipcrm if
> necessary), and (c) try to start postmaster with new -B/-N values.
>
> If step (c) fails when there are no pre-existing objects, then it must
> be that you have not managed to increase the kernel's limits on shared
> memory.  The exact incantations needed to accomplish that trick vary
> across systems, so I can't offer a lot of help there.
>
>                         regards, tom lane

--
Bill MacArthur
Webmaster
DHS Club

Re: PG 7.0.3 & RH 7 IPC problems?

From
Stefan Huber
Date:
Hi Bill!

>After shutting down the entries were still there.


Hmm, is this normal on your system? Because this would indicate a serious problem with your configuration (it's
certainlynot Postgres' fault). 

Try shutting down the postmaster again, and if the smhs are still there, then maybe you should doublecheck the
scripts/commandsyou use for shutting the postmaster down. 

Stefan
--
Atheism is a non-prophet organization.


Re: PG 7.0.3 & RH 7 IPC problems?

From
Lamar Owen
Date:
Stefan Huber wrote:
> >I have not used kill -9 on anything. :)

> Maybe you used killall, although this should normally issue SIGTERM (15), but who knows? Or maybe some script kills
thepostmaster with SIGKILL. Or maybe, the system just hung once. 

Sounds a little familiar.  Suffice to say the 7.1 initscript won't do a
kill -9.
--
Lamar Owen
WGCR Internet Radio
1 Peter 4:11