Thread: BUG #1632: Several jailed PostgreSQL instances.
The following bug has been logged online: Bug reference: 1632 Logged by: Alexander Rusinov Email address: boot@eurocom.od.ua PostgreSQL version: 7.4.7 Operating system: FreeBSD 5.4-PRERELEASE i386 Description: Several jailed PostgreSQL instances. Details: Only one instance of jailed PostgreSQL server is operating properly (the one which was started last). All of the others servers refuse to process connections. The error messages are as follows: Apr 26 10:16:10 www postgres[9063]: [2-1] FATAL: semctl(65536, 4, SETVAL, 0) failed: Invalid argument Apr 26 12:03:08 www postgres[15714]: [2-1] FATAL: semctl(1310720, 3, SETVAL, 0) failed: Invalid argument Apr 26 16:08:45 www postgres[27982]: [2-1] FATAL: semctl(1638400, 5, SETVAL, 0) failed: Invalid argument PostgreSQL servers were installed from FreeBSD ports collection and they are running in different FreeBSD jails. It's enough to run two servers (two jails) to reproduce the problem. Related kernel settings are: security.jail.sysvipc_allowed: 1 kern.ipc.shmmax: 100000000 kern.ipc.shmmin: 1 kern.ipc.shmmni: 192 kern.ipc.shmseg: 128 kern.ipc.shmall: 32768 kern.ipc.shm_use_phys: 0 kern.ipc.shm_allow_removed: 0 kern.ipc.semmap: 256 kern.ipc.semmni: 256 kern.ipc.semmns: 512 kern.ipc.semmnu: 256 kern.ipc.semmsl: 60 kern.ipc.semopm: 100 kern.ipc.semume: 10 kern.ipc.semusz: 92 kern.ipc.semvmx: 32767 kern.ipc.semaem: 16384 PostgreSQL servers are configured for 5 connections: max_connections = 5 shared_buffers = 100
O Alexander Rusinov Ýãñáøå óôéò Apr 27, 2005 : > > The following bug has been logged online: > > Bug reference: 1632 > Logged by: Alexander Rusinov > Email address: boot@eurocom.od.ua > PostgreSQL version: 7.4.7 > Operating system: FreeBSD 5.4-PRERELEASE i386 > Description: Several jailed PostgreSQL instances. > Details: Excuse me if i missed some episodes, but to the best of my knowledge, FreeBSD IPC is not jailified. There have been talks and talks on the matter on both lists, and it seems the only way to go is to start the jailed postgresql instances to listen to different ports. Tom and others, please correct me if situation now with FreeBSD 5.3+ has changed. > > Only one instance of jailed PostgreSQL server is operating properly (the one > which was started last). All of the others servers refuse to process > connections. The error messages are as follows: > > Apr 26 10:16:10 www postgres[9063]: [2-1] FATAL: semctl(65536, 4, SETVAL, > 0) failed: Invalid argument > Apr 26 12:03:08 www postgres[15714]: [2-1] FATAL: semctl(1310720, 3, > SETVAL, 0) failed: Invalid argument > Apr 26 16:08:45 www postgres[27982]: [2-1] FATAL: semctl(1638400, 5, > SETVAL, 0) failed: Invalid argument > > PostgreSQL servers were installed from FreeBSD ports collection and they are > running in different FreeBSD jails. It's enough to run two servers (two > jails) to reproduce the problem. > > Related kernel settings are: > > security.jail.sysvipc_allowed: 1 > > kern.ipc.shmmax: 100000000 > kern.ipc.shmmin: 1 > kern.ipc.shmmni: 192 > kern.ipc.shmseg: 128 > kern.ipc.shmall: 32768 > kern.ipc.shm_use_phys: 0 > kern.ipc.shm_allow_removed: 0 > > kern.ipc.semmap: 256 > kern.ipc.semmni: 256 > kern.ipc.semmns: 512 > kern.ipc.semmnu: 256 > kern.ipc.semmsl: 60 > kern.ipc.semopm: 100 > kern.ipc.semume: 10 > kern.ipc.semusz: 92 > kern.ipc.semvmx: 32767 > kern.ipc.semaem: 16384 > > PostgreSQL servers are configured for 5 connections: > max_connections = 5 > shared_buffers = 100 > > ---------------------------(end of broadcast)--------------------------- > TIP 6: Have you searched our list archives? > > http://archives.postgresql.org > -- -Achilleus
You are correct - freebsd have indeed a global shm space which I don't think is fixed in 5.x yet. We have run up to 4 postgresql in jails for our testing and it just-works(tm) if we choose a different port for each database instance. It might still be a good idea for postgresql to be able to detect this collision without crashing each others backend or doing other weird stuff. Maybe a dedicated bit in the shm space could be flipped by the new postmaster so it could see if it was flipped back again - this would allow it to abort gracefully with a "Other postmaster active in my shared memory" error. Any other ideas ? It should be trivally to implement something to handle it better. Nicolai Petri ----- Original Message ----- From: "Achilleus Mantzios" <achill@matrix.gatewaynet.com> To: "Alexander Rusinov" <boot@eurocom.od.ua> Cc: <pgsql-bugs@postgresql.org> Sent: Wednesday, April 27, 2005 3:53 PM Subject: Re: [BUGS] BUG #1632: Several jailed PostgreSQL instances. >O Alexander Rusinov Ýãñáøå óôéò Apr 27, 2005 : > >> >> The following bug has been logged online: >> >> Bug reference: 1632 >> Logged by: Alexander Rusinov >> Email address: boot@eurocom.od.ua >> PostgreSQL version: 7.4.7 >> Operating system: FreeBSD 5.4-PRERELEASE i386 >> Description: Several jailed PostgreSQL instances. >> Details: > > Excuse me if i missed some episodes, but to the best of my knowledge, > FreeBSD IPC is not jailified. > There have been talks and talks on the matter on both lists, > and it seems > the only way to go is to start the jailed postgresql instances > to listen to different ports. > > Tom and others, please correct me if situation now with FreeBSD 5.3+ has > changed. >
"Nicolai Petri (lists)" <lists@petri.cc> writes:
> It might still be a good idea for postgresql to be able to detect this
> collision without crashing each others backend or doing other weird stuff.
It tries --- see IpcSemaphoreCreate in src/backend/port/sysv_sema.c.
If the "jail" mechanism hides processes from each other but not
semaphore sets, then the attempted detection will fail. I don't think
that's a Postgres bug. The SysV IPC mechanism is defined to expose
process PIDs of processes accessing a shmem segment or sema set;
therefore you can't have a jail mechanism that separates PIDs but not
shmem/semas without fundamentally breaking the API.
Or this might just be a garden-variety kernel bug. In any case I think
you need to be complaining to FreeBSD kernel hackers, not us.
regards, tom lane
O Tom Lane Ýãñáøå óôéò Apr 30, 2005 : > "Nicolai Petri (lists)" <lists@petri.cc> writes: > > It might still be a good idea for postgresql to be able to detect this > > collision without crashing each others backend or doing other weird stuff. > > It tries --- see IpcSemaphoreCreate in src/backend/port/sysv_sema.c. > If the "jail" mechanism hides processes from each other but not > semaphore sets, then the attempted detection will fail. I don't think > that's a Postgres bug. The SysV IPC mechanism is defined to expose > process PIDs of processes accessing a shmem segment or sema set; > therefore you can't have a jail mechanism that separates PIDs but not > shmem/semas without fundamentally breaking the API. > > Or this might just be a garden-variety kernel bug. In any case I think Its obvious that FreeBSD says: DONT USE IPC in jails, so its far from a bug. > you need to be complaining to FreeBSD kernel hackers, not us. Now with XEN, i suspect jailifying IPC in FreeBSD seems even less attractive of a task, than it might have been back in 4.x days. Thats about now. But back then, if FreeBSD people had enuf demand for jailified IPC and finally implemented it, that would give postgresql a big boost in the hosting field (where FreeBSD is a very popular) vs mysql. > > regards, tom lane > > ---------------------------(end of broadcast)--------------------------- > TIP 3: if posting/reading through Usenet, please send an appropriate > subscribe-nomail command to majordomo@postgresql.org so that your > message can get through to the mailing list cleanly > -- -Achilleus
Achilleus Mantzios <achill@matrix.gatewaynet.com> writes:
> Its obvious that FreeBSD says: DONT USE IPC in jails, so its far from a
> bug.
Marc's been running PG inside FreeBSD jails for years, so it seems
like there's some disconnect here ... I don't have the knowledge
to resolve it though.
regards, tom lane
O Tom Lane Ýãñáøå óôéò May 3, 2005 : > Achilleus Mantzios <achill@matrix.gatewaynet.com> writes: > > Its obvious that FreeBSD says: DONT USE IPC in jails, so its far from a > > bug. > > Marc's been running PG inside FreeBSD jails for years, so it seems > like there's some disconnect here ... I don't have the knowledge > to resolve it though. FreeBSD's security.jail.sysvipc_allowed sysctl variable's intention was to allow a person to overcome FreeBSD security. Ofcource someone can run postgresql in jails, but it is not secure in public servers. Thats why security.jail.sysvipc_allowed defaults to -> 0, in other words thats why they say to not use IPC in jails. > > regards, tom lane > > ---------------------------(end of broadcast)--------------------------- > TIP 9: the planner will ignore your desire to choose an index scan if your > joining column's datatypes do not match > -- -Achilleus