Thread: running pgsql 7 under Jail'ed virtual machine on FreeBSD 4.2
Have a FreeBSD 4.2 box... have created two Virtual machines with the jail process. Apache and PHp configure and execute fine, but am having some configuration and execution problems with pgsql... not sure if I am missing something or if pgsql just cannot deal with being in the jail and sharing the sysv memory Attempting to manually start the postmaster produces the following: %postmaster -i IpcMemoryCreate: shmget failed (Function not implemented) key=5432010, size=144, permission=700 This type of error is usually caused by an improper shared memory or System V IPC semaphore configuration. For more information, see the FAQ and platform-specific FAQ's in the source directory pgsql/doc or on our web site at http://www.postgresql.org. IpcMemoryIdGet: shmget failed (Function not implemented) key=5432010, size=144, permission=0 IpcMemoryAttach: shmat failed (Function not implemented) id=-2 FATAL 1: AttachSLockMemory: could not attach segment % Have modified the sysctl.conf file to allow usage of sysv memory, but as it is already shared from the host machine, this may be causing pgsql to barf on it. Searching for jail faq's and posts regarding this matter produces some close, but not helpful posts regarding chroot'ed environments. Any insights, requests for details, and assistance are appreciated. Dave
On Thu, Jan 04, 2001 at 05:04:50PM -0500, Dave VanAuken wrote: > Have a FreeBSD 4.2 box... have created two Virtual machines with the > jail process. > Apache and PHp configure and execute fine, but am having some > configuration and execution problems with pgsql... not sure if I am > missing something or if pgsql just cannot deal with being in the jail > and sharing the sysv memory > > Attempting to manually start the postmaster produces the following: > > %postmaster -i > IpcMemoryCreate: shmget failed (Function not implemented) > key=5432010, Perhaps you could mail freebsd-questions@freebsd.org and find out if there are any issues with shared memory inside a jail. My first guess as to what the error message means is that it's not supported. -- Matt Beauregard Information Technology Operations, DesignScape Ph: +61 2 9361 4233 Fx: +61 2 9361 4633
read the jail man page:
jail.sysvipc_allowed
This MIB entry determines whether or not processes within a jail
have access to System V IPC primitives. In the current jail imple-
mentation, System V primitives share a single namespace across the
host and jail environments, meaning that processes within a jail
would be able to communicate with (and potentially interfere with)
processes outside of the jail, and in other jails. As such, this
functionality is disabled by default, but can be enabled by setting
this MIB entry to 1.
On Thu, 4 Jan 2001, Dave VanAuken wrote:
> Have a FreeBSD 4.2 box... have created two Virtual machines with the
> jail process.
> Apache and PHp configure and execute fine, but am having some
> configuration and execution problems with pgsql... not sure if I am
> missing something or if pgsql just cannot deal with being in the jail
> and sharing the sysv memory
>
> Attempting to manually start the postmaster produces the following:
>
> %postmaster -i
> IpcMemoryCreate: shmget failed (Function not implemented) key=5432010,
> size=144, permission=700
> This type of error is usually caused by an improper
> shared memory or System V IPC semaphore configuration.
> For more information, see the FAQ and platform-specific
> FAQ's in the source directory pgsql/doc or on our
> web site at http://www.postgresql.org.
> IpcMemoryIdGet: shmget failed (Function not implemented) key=5432010,
> size=144, permission=0
> IpcMemoryAttach: shmat failed (Function not implemented) id=-2
> FATAL 1: AttachSLockMemory: could not attach segment
> %
>
> Have modified the sysctl.conf file to allow usage of sysv memory, but
> as it is already shared from the host machine, this may be causing
> pgsql to barf on it.
>
> Searching for jail faq's and posts regarding this matter produces some
> close, but not helpful posts regarding chroot'ed environments.
>
> Any insights, requests for details, and assistance are appreciated.
>
> Dave
>
>
Marc G. Fournier ICQ#7615664 IRC Nick: Scrappy
Systems Administrator @ hub.org
primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org
As listed in the initial post immediately following the screen output,
I have already set the sharing in the sysctl.conf file, the same error
is being generated.
Dave
-----Original Message-----
From: pgsql-general-owner@postgresql.org
[mailto:pgsql-general-owner@postgresql.org]On Behalf Of The Hermit
Hacker
Sent: Thursday, January 04, 2001 11:30 PM
To: Dave VanAuken
Cc: pgsql-general@postgresql.org
Subject: Re: running pgsql 7 under Jail'ed virtual machine on FreeBSD
4.2
read the jail man page:
jail.sysvipc_allowed
This MIB entry determines whether or not processes within a
jail
have access to System V IPC primitives. In the current jail
imple-
mentation, System V primitives share a single namespace
across the
host and jail environments, meaning that processes within a
jail
would be able to communicate with (and potentially interfere
with)
processes outside of the jail, and in other jails. As such,
this
functionality is disabled by default, but can be enabled by
setting
this MIB entry to 1.
On Thu, 4 Jan 2001, Dave VanAuken wrote:
> Have a FreeBSD 4.2 box... have created two Virtual machines with
the
> jail process.
> Apache and PHp configure and execute fine, but am having some
> configuration and execution problems with pgsql... not sure if I am
> missing something or if pgsql just cannot deal with being in the
jail
> and sharing the sysv memory
>
> Attempting to manually start the postmaster produces the following:
>
> %postmaster -i
> IpcMemoryCreate: shmget failed (Function not implemented)
key=5432010,
> size=144, permission=700
> This type of error is usually caused by an improper
> shared memory or System V IPC semaphore configuration.
> For more information, see the FAQ and platform-specific
> FAQ's in the source directory pgsql/doc or on our
> web site at http://www.postgresql.org.
> IpcMemoryIdGet: shmget failed (Function not implemented)
key=5432010,
> size=144, permission=0
> IpcMemoryAttach: shmat failed (Function not implemented) id=-2
> FATAL 1: AttachSLockMemory: could not attach segment
> %
>
> Have modified the sysctl.conf file to allow usage of sysv memory,
but
> as it is already shared from the host machine, this may be causing
> pgsql to barf on it.
>
> Searching for jail faq's and posts regarding this matter produces
some
> close, but not helpful posts regarding chroot'ed environments.
>
> Any insights, requests for details, and assistance are appreciated.
>
> Dave
>
>
Marc G. Fournier ICQ#7615664 IRC Nick:
Scrappy
Systems Administrator @ hub.org
primary: scrappy@hub.org secondary:
scrappy@{freebsd|postgresql}.org
Here's the story about pgsql and jail: 1. FreeBSD shared memory is _really_ shared across all jail VMs. I think its not going to change, as this seems like a great way for processes to exchange data...But I could be wrong. :) 2. Postgres uses some funky math to determine which shmid to use for different memory allocations it needs. The problem with that it has no idea about jails and will not generate a different shmid for different processes, so postgres processes end up stepping on each other's toes. See include/storage/ipc.h, definition of SystemPortAddressGetIPCKey, and you'll understand what I mean. To properly fix this, I think this should be done: Another (optional) backend parameter which should determine offset (hardcoded now to 17491) for allocation of different shmids... comments? TODO? On Thu, 4 Jan 2001, Dave VanAuken wrote: > Have a FreeBSD 4.2 box... have created two Virtual machines with the > jail process. > Apache and PHp configure and execute fine, but am having some > configuration and execution problems with pgsql... not sure if I am > missing something or if pgsql just cannot deal with being in the jail > and sharing the sysv memory > > Attempting to manually start the postmaster produces the following: > > %postmaster -i > IpcMemoryCreate: shmget failed (Function not implemented) key=5432010, > size=144, permission=700 > This type of error is usually caused by an improper > shared memory or System V IPC semaphore configuration. > For more information, see the FAQ and platform-specific > FAQ's in the source directory pgsql/doc or on our > web site at http://www.postgresql.org. > IpcMemoryIdGet: shmget failed (Function not implemented) key=5432010, > size=144, permission=0 > IpcMemoryAttach: shmat failed (Function not implemented) id=-2 > FATAL 1: AttachSLockMemory: could not attach segment > % > > Have modified the sysctl.conf file to allow usage of sysv memory, but > as it is already shared from the host machine, this may be causing > pgsql to barf on it. > > Searching for jail faq's and posts regarding this matter produces some > close, but not helpful posts regarding chroot'ed environments. > > Any insights, requests for details, and assistance are appreciated. > > Dave > >
Knew about #1... and I'm not holding my breath for change gicing where the development on that angle is heading. In the installation directory for the jail i don't have the /storage directory, or for that matter the ipc.h file anywhere in the pgsql tree (both in the jail and host installations). The version installed was 7.0.2 What about hacking supplimentary installations and hardcoding new offsets for each subsequent installation of pgsql (can't even look into that until i detemine where the ipc.h code now exists). Appreciate the input... the best description so far from the pgsql pov regarding the situation... have been working almost entirely from the fbsd pov up until not and running out of options. May be a wash for using pgsql in this situation. Dave -----Original Message----- From: pgsql-general-owner@postgresql.org [mailto:pgsql-general-owner@postgresql.org]On Behalf Of Alex Pilosov Sent: Friday, January 05, 2001 1:50 PM To: Dave VanAuken Cc: pgsql-general@postgresql.org Subject: Re: running pgsql 7 under Jail'ed virtual machine on FreeBSD 4.2 Here's the story about pgsql and jail: 1. FreeBSD shared memory is _really_ shared across all jail VMs. I think its not going to change, as this seems like a great way for processes to exchange data...But I could be wrong. :) 2. Postgres uses some funky math to determine which shmid to use for different memory allocations it needs. The problem with that it has no idea about jails and will not generate a different shmid for different processes, so postgres processes end up stepping on each other's toes. See include/storage/ipc.h, definition of SystemPortAddressGetIPCKey, and you'll understand what I mean. To properly fix this, I think this should be done: Another (optional) backend parameter which should determine offset (hardcoded now to 17491) for allocation of different shmids... comments? TODO? On Thu, 4 Jan 2001, Dave VanAuken wrote: > Have a FreeBSD 4.2 box... have created two Virtual machines with the > jail process. > Apache and PHp configure and execute fine, but am having some > configuration and execution problems with pgsql... not sure if I am > missing something or if pgsql just cannot deal with being in the jail > and sharing the sysv memory > > Attempting to manually start the postmaster produces the following: > > %postmaster -i > IpcMemoryCreate: shmget failed (Function not implemented) key=5432010, > size=144, permission=700 > This type of error is usually caused by an improper > shared memory or System V IPC semaphore configuration. > For more information, see the FAQ and platform-specific > FAQ's in the source directory pgsql/doc or on our > web site at http://www.postgresql.org. > IpcMemoryIdGet: shmget failed (Function not implemented) key=5432010, > size=144, permission=0 > IpcMemoryAttach: shmat failed (Function not implemented) id=-2 > FATAL 1: AttachSLockMemory: could not attach segment > % > > Have modified the sysctl.conf file to allow usage of sysv memory, but > as it is already shared from the host machine, this may be causing > pgsql to barf on it. > > Searching for jail faq's and posts regarding this matter produces some > close, but not helpful posts regarding chroot'ed environments. > > Any insights, requests for details, and assistance are appreciated. > > Dave > >
Yes, obviously /storage/* isn't in the install dir, its the source file. If you are using a port to compile postgres, it'd be in /usr/ports/databases/postgresql7/work/postgresql-7.0.2/src/include I'd suggest you do this: add a global backend_shmid_offset in ipc.c, initialized to current default, and an option to postgres to put a value there. I'm not sure whether this option should be in pg_options or postmaster.opts, and what's the name of it should be...Comments? After you are done, send us a patch :) On Fri, 5 Jan 2001, Dave VanAuken wrote: > Knew about #1... and I'm not holding my breath for change gicing where > the development on that angle is heading. > > In the installation directory for the jail i don't have the /storage > directory, or for that matter the ipc.h file anywhere in the pgsql > tree (both in the jail and host installations). The version installed > was 7.0.2 > > What about hacking supplimentary installations and hardcoding new > offsets for each subsequent installation of pgsql (can't even look > into that until i detemine where the ipc.h code now exists). > > Appreciate the input... the best description so far from the pgsql > pov regarding the situation... have been working almost entirely from > the fbsd pov up until not and running out of options. May be a wash > for using pgsql in this situation. > > Dave > > -----Original Message----- > From: pgsql-general-owner@postgresql.org > [mailto:pgsql-general-owner@postgresql.org]On Behalf Of Alex Pilosov > Sent: Friday, January 05, 2001 1:50 PM > To: Dave VanAuken > Cc: pgsql-general@postgresql.org > Subject: Re: running pgsql 7 under Jail'ed virtual machine on FreeBSD > 4.2 > > > Here's the story about pgsql and jail: > > 1. FreeBSD shared memory is _really_ shared across all jail VMs. I > think > its not going to change, as this seems like a great way for processes > to > exchange data...But I could be wrong. :) > > 2. Postgres uses some funky math to determine which shmid to use for > different memory allocations it needs. The problem with that it has no > idea about jails and will not generate a different shmid for different > processes, so postgres processes end up stepping on each other's toes. > > See include/storage/ipc.h, definition of SystemPortAddressGetIPCKey, > and > you'll understand what I mean. > > To properly fix this, I think this should be done: > > Another (optional) backend parameter which should determine offset > (hardcoded now to 17491) for allocation of different shmids... > > comments? TODO? > > > On Thu, 4 Jan 2001, Dave VanAuken wrote: > > > Have a FreeBSD 4.2 box... have created two Virtual machines with > the > > jail process. > > Apache and PHp configure and execute fine, but am having some > > configuration and execution problems with pgsql... not sure if I am > > missing something or if pgsql just cannot deal with being in the > jail > > and sharing the sysv memory > > > > Attempting to manually start the postmaster produces the following: > > > > %postmaster -i > > IpcMemoryCreate: shmget failed (Function not implemented) > key=5432010, > > size=144, permission=700 > > This type of error is usually caused by an improper > > shared memory or System V IPC semaphore configuration. > > For more information, see the FAQ and platform-specific > > FAQ's in the source directory pgsql/doc or on our > > web site at http://www.postgresql.org. > > IpcMemoryIdGet: shmget failed (Function not implemented) > key=5432010, > > size=144, permission=0 > > IpcMemoryAttach: shmat failed (Function not implemented) id=-2 > > FATAL 1: AttachSLockMemory: could not attach segment > > % > > > > Have modified the sysctl.conf file to allow usage of sysv memory, > but > > as it is already shared from the host machine, this may be causing > > pgsql to barf on it. > > > > Searching for jail faq's and posts regarding this matter produces > some > > close, but not helpful posts regarding chroot'ed environments. > > > > Any insights, requests for details, and assistance are appreciated. > > > > Dave > > > > > > >
Alex Pilosov <alex@pilosoft.com> writes:
> I'd suggest you do this: add a global backend_shmid_offset in ipc.c,
> initialized to current default, and an option to postgres to put a value
> there.
Don't waste your time. This issue is gone in current sources. See
IpcMemoryCreate and IpcSemaphoreCreate in src/backend/storage/ipc/ipc.c.
regards, tom lane
Re: SHM ids (was running pgsql 7 under Jail'ed virtual machine on FreeBSD 4.2)
From
Alex Pilosov
Date:
On Fri, 5 Jan 2001, Tom Lane wrote: > Alex Pilosov <alex@pilosoft.com> writes: > > I'd suggest you do this: add a global backend_shmid_offset in ipc.c, > > initialized to current default, and an option to postgres to put a value > > there. > > Don't waste your time. This issue is gone in current sources. See > IpcMemoryCreate and IpcSemaphoreCreate in src/backend/storage/ipc/ipc.c. Damn. One of these days I'm going to go and get myself up on postgresql 7.1, so I won't bother you and others with silly things that are fixed in -current :) Thanks Tom -alex
I said:
> Alex Pilosov <alex@pilosoft.com> writes:
>> I'd suggest you do this: add a global backend_shmid_offset in ipc.c,
>> initialized to current default, and an option to postgres to put a value
>> there.
> Don't waste your time. This issue is gone in current sources. See
> IpcMemoryCreate and IpcSemaphoreCreate in src/backend/storage/ipc/ipc.c.
Actually, now that I think about it, you might still have a problem if
a "jail" is what I think it is. The current code will step past an
existing shmem segment if it appears to be a non-Postgres memory segment
or if it appears to belong to another running postmaster. However, the
test to see if the segment owner appears to be alive is
/*
* If the creator PID is my own PID or does not belong to any
* extant process, it's safe to zap it.
*/
if (hdr->creatorPID != getpid())
{
if (kill(hdr->creatorPID, 0) == 0 ||
errno != ESRCH)
{
// segment belongs to a live postmaster, so continue
}
}
So the question for you is, what will happen if kill() is given a PID
belonging to a process that's in a different "jail"? Will it return
some kind of permission failure, or will it claim that there is no
such PID (ie, return ESRCH)? If the latter, we still have a problem ...
regards, tom lane
Anybody knows where I can find the windows - ODBC driver for the latest version of PostgreSQL (7.0.3)? Thanx
Re: SHM ids (was running pgsql 7 under Jail'ed virtual machine on FreeBSD 4.2)
From
Alex Pilosov
Date:
What'd happen is: Operation not permitted (EPERM, -1), so its not a
problem...
-alex
On Fri, 5 Jan 2001, Tom Lane wrote:
> I said:
> > Alex Pilosov <alex@pilosoft.com> writes:
> >> I'd suggest you do this: add a global backend_shmid_offset in ipc.c,
> >> initialized to current default, and an option to postgres to put a value
> >> there.
>
> > Don't waste your time. This issue is gone in current sources. See
> > IpcMemoryCreate and IpcSemaphoreCreate in src/backend/storage/ipc/ipc.c.
>
> Actually, now that I think about it, you might still have a problem if
> a "jail" is what I think it is. The current code will step past an
> existing shmem segment if it appears to be a non-Postgres memory segment
> or if it appears to belong to another running postmaster. However, the
> test to see if the segment owner appears to be alive is
>
> /*
> * If the creator PID is my own PID or does not belong to any
> * extant process, it's safe to zap it.
> */
> if (hdr->creatorPID != getpid())
> {
> if (kill(hdr->creatorPID, 0) == 0 ||
> errno != ESRCH)
> {
> // segment belongs to a live postmaster, so continue
> }
> }
>
> So the question for you is, what will happen if kill() is given a PID
> belonging to a process that's in a different "jail"? Will it return
> some kind of permission failure, or will it claim that there is no
> such PID (ie, return ESRCH)? If the latter, we still have a problem ...
>
> regards, tom lane
>
>
RE: SHM ids (was running pgsql 7 under Jail'ed virtual machine on FreeBSD 4.2)
From
"Dave VanAuken"
Date:
the jail can only address the PID's that operate within itself... in
effect allowing it to f'up its own enviroment only... thus the
problem with pg trying to allocate dedicated portions of the
host/system wide memory.
will up the 7.0.2 to 7.1 and give her another go around. Really would
like to work this within a jail'ed environment, but have much to much
code completed based on postgres to consider dumping it just because
of the functionality of the jail.
Dave
-----Original Message-----
From: pgsql-general-owner@postgresql.org
[mailto:pgsql-general-owner@postgresql.org]On Behalf Of Tom Lane
Sent: Friday, January 05, 2001 6:45 PM
To: Alex Pilosov
Cc: Dave VanAuken; pgsql-general@postgresql.org
Subject: Re: SHM ids (was running pgsql 7 under Jail'ed virtual
machine
on FreeBSD 4.2)
I said:
> Alex Pilosov <alex@pilosoft.com> writes:
>> I'd suggest you do this: add a global backend_shmid_offset in
ipc.c,
>> initialized to current default, and an option to postgres to put a
value
>> there.
> Don't waste your time. This issue is gone in current sources. See
> IpcMemoryCreate and IpcSemaphoreCreate in
src/backend/storage/ipc/ipc.c.
Actually, now that I think about it, you might still have a problem if
a "jail" is what I think it is. The current code will step past an
existing shmem segment if it appears to be a non-Postgres memory
segment
or if it appears to belong to another running postmaster. However,
the
test to see if the segment owner appears to be alive is
/*
* If the creator PID is my own PID or does not belong to any
* extant process, it's safe to zap it.
*/
if (hdr->creatorPID != getpid())
{
if (kill(hdr->creatorPID, 0) == 0 ||
errno != ESRCH)
{
// segment belongs to a live postmaster, so continue
}
}
So the question for you is, what will happen if kill() is given a PID
belonging to a process that's in a different "jail"? Will it return
some kind of permission failure, or will it claim that there is no
such PID (ie, return ESRCH)? If the latter, we still have a problem
...
regards, tom lane
Re: RE: SHM ids (was running pgsql 7 under Jail'ed virtual machine on FreeBSD 4.2)
From
Tom Lane
Date:
"Dave VanAuken" <dave@hawk-systems.com> writes:
> the jail can only address the PID's that operate within itself... in
> effect allowing it to f'up its own enviroment only...
Certainly a jailed process should not be allowed to actually kill()
a process outside the jail. The question is what error code does it
get?
regards, tom lane
Is there an option to log just queries that fail or output some warning in 7.1? I'd like to use this to check for incompatibilities of various software with pgsql 7.1. Currently I get these errors: ERROR: parser: parse error at or near "." ERROR: parser: parse error at or near "." ERROR: parser: parse error at or near "." etc Would like to catch the real queries. Btw: this is from phorum 3.2.11. It seems to work OK anyway, but still want to solve this. Tomaz
Hello,
I'm converting a mysql db to postgres and ran into this line of code:
$SQL = "select username from users where user_id = ? AND session = ?
AND ADDDATE(last_login, INTERVAL 30 MINUTE)>CURRENT_TIMESTAMP()
How do I implement adddate to get this kind of funcionality?
Thanks,
Steve Leibel
Assuming adddate does what seems obvious, you should just be able to add intervals to timestamps ... last_login + '30 minutes' > now() On Thu, 25 Jan 2001, Steve Leibel wrote: > Hello, > > I'm converting a mysql db to postgres and ran into this line of code: > > $SQL = "select username from users where user_id = ? AND session = ? > AND ADDDATE(last_login, INTERVAL 30 MINUTE)>CURRENT_TIMESTAMP() > > How do I implement adddate to get this kind of funcionality?