Thread: flock patch breaks things here

backend/libpq/pgcomm.c no longer compiles on my system.  The cvs log sez

Massimo Dal Zotto <dz@cs.unitn.it>
> socket-flock.patch
>       use advisory locks to check if the unix socket can be deleted.
>       A running postmaster keeps a lock on that file. A starting
>       postmaster exits if the file exists and is locked, otherwise
>       it deletes the sockets and proceeds.
>       This avoid the need to remove manually the file after a postmaster
>       or system crash.
>       I don't know if flock is available on any system. If not we could
>       define a HAVE_FLOCK set by configure.

flock is *VERY* far from portable.  I am aware of three or four
different, mutually incompatible file locking syscalls on different
Unix flavors.  flock is just one of the contestants.  Even if the
call syntax were uniform, the semantics are not portable enough to
be safe (advisory locks don't work on NFS-mounted files, for example).

Massimo has a good idea in the long run, but I have strong doubts that
we want to start working the bugs out two days before a beta release
cycle.  Portable file locking in Unix is a very nasty can of worms,
and I recommend not opening it at this particular point.

In short: I'd like to see this patch backed out until after 6.4.

            regards, tom lane

On Sun, 30 Aug 1998, Tom Lane wrote:

> Massimo Dal Zotto <dz@cs.unitn.it>
> > socket-flock.patch
> >       use advisory locks to check if the unix socket can be deleted.
> >       A running postmaster keeps a lock on that file. A starting
> >       postmaster exits if the file exists and is locked, otherwise
> >       it deletes the sockets and proceeds.
> >       This avoid the need to remove manually the file after a postmaster
> >       or system crash.
> >       I don't know if flock is available on any system. If not we could
> >       define a HAVE_FLOCK set by configure.
>
> flock is *VERY* far from portable.  I am aware of three or four
> different, mutually incompatible file locking syscalls on different
> Unix flavors.  flock is just one of the contestants.  Even if the
> call syntax were uniform, the semantics are not portable enough to
> be safe (advisory locks don't work on NFS-mounted files, for example).

  You can't create unix domain sockets on NFS files systems (well, you
might be allowed to, it just might not be very useful).

  The flock() call syntax is very consistant.  The only other option is
fcntl() which is very consistant too, and is described in POSIX.1

  The only other option is lockf().  No one uses this anymore.  It was
only ever supported on SVR2,3,4 anyhow.  It also is just a subset of
fcntl()

> Massimo has a good idea in the long run, but I have strong doubts that
> we want to start working the bugs out two days before a beta release
> cycle.  Portable file locking in Unix is a very nasty can of worms,
> and I recommend not opening it at this particular point.

  Not really.  The biggest problem is NFS issues, which don't need to be
considered for unix domain sockets.  The various sematics issues don't
really apply for locking is being used for.

> In short: I'd like to see this patch backed out until after 6.4.
>
>             regards, tom lane

Tom

>
> backend/libpq/pgcomm.c no longer compiles on my system.  The cvs log sez
>
> Massimo Dal Zotto <dz@cs.unitn.it>
> > socket-flock.patch
> >       use advisory locks to check if the unix socket can be deleted.
> >       A running postmaster keeps a lock on that file. A starting
> >       postmaster exits if the file exists and is locked, otherwise
> >       it deletes the sockets and proceeds.
> >       This avoid the need to remove manually the file after a postmaster
> >       or system crash.
> >       I don't know if flock is available on any system. If not we could
> >       define a HAVE_FLOCK set by configure.
>
> flock is *VERY* far from portable.  I am aware of three or four
> different, mutually incompatible file locking syscalls on different
> Unix flavors.  flock is just one of the contestants.  Even if the
> call syntax were uniform, the semantics are not portable enough to
> be safe (advisory locks don't work on NFS-mounted files, for example).
>
> Massimo has a good idea in the long run, but I have strong doubts that
> we want to start working the bugs out two days before a beta release
> cycle.  Portable file locking in Unix is a very nasty can of worms,
> and I recommend not opening it at this particular point.
>
> In short: I'd like to see this patch backed out until after 6.4.
>

Yes, I'm aware of this. For the moment I suggest we put a #ifdef linux
around the code until a more portable solution is found.

--
Massimo Dal Zotto

+----------------------------------------------------------------------+
|  Massimo Dal Zotto                email:  dz@cs.unitn.it             |
|  Via Marconi, 141                 phone:  ++39-461-534251            |
|  38057 Pergine Valsugana (TN)     www:  http://www.cs.unitn.it/~dz/  |
|  Italy                            pgp:  finger dz@tango.cs.unitn.it  |
+----------------------------------------------------------------------+

[Charset iso-8859-1 unsupported, filtering to ASCII...]
> >
> > backend/libpq/pgcomm.c no longer compiles on my system.  The cvs log sez
> >
> > Massimo Dal Zotto <dz@cs.unitn.it>
> > > socket-flock.patch
> > >       use advisory locks to check if the unix socket can be deleted.
> > >       A running postmaster keeps a lock on that file. A starting
> > >       postmaster exits if the file exists and is locked, otherwise
> > >       it deletes the sockets and proceeds.
> > >       This avoid the need to remove manually the file after a postmaster
> > >       or system crash.
> > >       I don't know if flock is available on any system. If not we could
> > >       define a HAVE_FLOCK set by configure.
> >
> > flock is *VERY* far from portable.  I am aware of three or four
> > different, mutually incompatible file locking syscalls on different
> > Unix flavors.  flock is just one of the contestants.  Even if the
> > call syntax were uniform, the semantics are not portable enough to
> > be safe (advisory locks don't work on NFS-mounted files, for example).
> >
> > Massimo has a good idea in the long run, but I have strong doubts that
> > we want to start working the bugs out two days before a beta release
> > cycle.  Portable file locking in Unix is a very nasty can of worms,
> > and I recommend not opening it at this particular point.
> >
> > In short: I'd like to see this patch backed out until after 6.4.
> >
>
> Yes, I'm aware of this. For the moment I suggest we put a #ifdef linux
> around the code until a more portable solution is found.


Can't we just have configure check for flock().  Another idea is to
create a 'pid' file in the pgsql/data/base directory, and do a kill -0
to see if it is stil running before removing the lock.

--
Bruce Momjian                          |  830 Blythe Avenue
maillist@candle.pha.pa.us              |  Drexel Hill, Pennsylvania 19026
  +  If your life is a hard drive,     |  (610) 353-9879(w)
  +  Christ can be your backup.        |  (610) 853-3000(h)

Bruce Momjian <maillist@candle.pha.pa.us> writes:
> Can't we just have configure check for flock().  Another idea is to
> create a 'pid' file in the pgsql/data/base directory, and do a kill -0
> to see if it is stil running before removing the lock.

The latter approach is what I was going to suggest.  Writing a pid file
would be a fine idea anyway --- for one thing, it makes it a lot easier
to write a "kill the postmaster" script.  Given that the postmaster
should write a pid file, a new postmaster should look for an existing
pid file, and try to do a kill(pid, 0) on the number contained therein.
If this doesn't return an error, then you figure there is already a
postmaster running, complain, and exit.  Otherwise you figure you is it,
(re)write the pid file and away you go.  Then pqcomm.c can just
unconditionally delete any old file that's in the way of making the
pipe.

The pidfile checking and creation probably ought to go in postmaster.c,
not down inside pqcomm.c.  I never liked the fact that a critical
interlock function was being done by a low-level library that one might
not even want to invoke (if all your clients are using TCP, opening up
the Unix-domain socket is a waste of time, no?).

BTW, there is another problem with relying on flock on the socket file
for this purpose: it opens up a hole for a denial-of-service attack.
Anyone who can write the file can flock it.  (We already had a problem
with DOS via creating a dummy file at /tmp/.s.PGSQL.5432, but it would
be harder to spot the culprit with an flock-based interference.)

            regards, tom lane

>
> Bruce Momjian <maillist@candle.pha.pa.us> writes:
> > Can't we just have configure check for flock().  Another idea is to
> > create a 'pid' file in the pgsql/data/base directory, and do a kill -0
> > to see if it is stil running before removing the lock.
>
> The latter approach is what I was going to suggest.  Writing a pid file
> would be a fine idea anyway --- for one thing, it makes it a lot easier
> to write a "kill the postmaster" script.  Given that the postmaster
> should write a pid file, a new postmaster should look for an existing
> pid file, and try to do a kill(pid, 0) on the number contained therein.
> If this doesn't return an error, then you figure there is already a
> postmaster running, complain, and exit.  Otherwise you figure you is it,
> (re)write the pid file and away you go.  Then pqcomm.c can just
> unconditionally delete any old file that's in the way of making the
> pipe.
>
> The pidfile checking and creation probably ought to go in postmaster.c,
> not down inside pqcomm.c.  I never liked the fact that a critical
> interlock function was being done by a low-level library that one might
> not even want to invoke (if all your clients are using TCP, opening up
> the Unix-domain socket is a waste of time, no?).
>
> BTW, there is another problem with relying on flock on the socket file
> for this purpose: it opens up a hole for a denial-of-service attack.
> Anyone who can write the file can flock it.  (We already had a problem
> with DOS via creating a dummy file at /tmp/.s.PGSQL.5432, but it would
> be harder to spot the culprit with an flock-based interference.)

This came to my mind, but I didn't think this would have happened so
quickly. In my opinion the socket and the pidfile should be created in a
directory owned by postgres, for example /tmp/.Pgsql-unix, like does X.

--
Massimo Dal Zotto

+----------------------------------------------------------------------+
|  Massimo Dal Zotto                email:  dz@cs.unitn.it             |
|  Via Marconi, 141                 phone:  ++39-461-534251            |
|  38057 Pergine Valsugana (TN)     www:  http://www.cs.unitn.it/~dz/  |
|  Italy                            pgp:  finger dz@tango.cs.unitn.it  |
+----------------------------------------------------------------------+

Massimo Dal Zotto <dz@cs.unitn.it> writes:
> In my opinion the socket and the pidfile should be created in a
> directory owned by postgres, for example /tmp/.Pgsql-unix, like does X.

The pidfile belongs at the top level of the database directory (eg,
/usr/local/pgsql/data/postmaster.pid), because what it actually
represents is that there is a postmaster running *for that database
group*.

If you want to support multiple database sets on one machine (which I
do), then the interlock has to be per database directory.  Putting the
pidfile into a common directory would mean we'd have to invent some
kind of pidfile naming convention to keep multiple postmasters from
tromping on each other.  This is unnecessarily complex.

I agree with you that putting the socket file into a less easily munged
directory than /tmp would be a good idea for security.  But that's a
separate issue.  On machines that understand stickybits for directories,
the security hole is not really very big.

At this point, the fact that /tmp/.s.PGSQL.port# is the socket path is
effectively a version-independent aspect of the FE/BE protocol, and so
we can't change it without breaking old applications.  I'm not sure that
that's worth the security improvement.

What I'd like to see someday is a postmaster command line switch to tell
it to use *only* TCP connections and not create a Unix socket at all.
That hasn't been possible so far, because we were relying on the socket
file to provide a safety interlock against starting multiple
postmasters.  But an interlock using a pidfile would be much better.
(Look around; *every* other Unix daemon I know of that wants to ensure
that there's only one of it uses a pidfile interlock.  Not file locking.
There's a reason why that's the well-trodden path.)

            regards, tom lane

On Sun, 30 Aug 1998, Tom Lane wrote:

> Massimo Dal Zotto <dz@cs.unitn.it> writes:
> > In my opinion the socket and the pidfile should be created in a
> > directory owned by postgres, for example /tmp/.Pgsql-unix, like does X.
>
> The pidfile belongs at the top level of the database directory (eg,
> /usr/local/pgsql/data/postmaster.pid), because what it actually
> represents is that there is a postmaster running *for that database
> group*.

    I have to agree with this one...but then it also negates the
argument about the flock() DoS...*grin*

    BTW...I like the kill(pid,0) solution myself, primarily because it
is, i think, the most portable solution.

    I would not consider a patch to remove the flock() solution and
replace it with the kill(pid,0) solution a new feature, just an
improvement of an existing one...either way, moving the pid file (or
socket, for that matter) from /tmp should be listed as a security related
requirement for v6.4 :)

Marc G. Fournier
Systems Administrator @ hub.org
primary: scrappy@hub.org           secondary: scrappy@{freebsd|postgresql}.org

The Hermit Hacker <scrappy@hub.org> writes:
> either way, moving the pid file (or
> socket, for that matter) from /tmp should be listed as a security related
> requirement for v6.4 :)

Huh?  There is no pid file being generated in /tmp (or anywhere else)
at the moment.  If we do add one, it should not go into /tmp for the
reasons I gave before.

Where the Unix-domain socket file lives is an entirely separate issue.

If we move the socket out of /tmp then we have just kicked away all the
work we did to preserve backwards compatibility of the FE/BE protocol
with existing clients.  Being able to talk to a 1.0 client isn't much
good if you aren't listening where he's going to try to contact you.
So I think I have to vote in favor of leaving the socket where it is.

            regards, tom lane

> The Hermit Hacker <scrappy@hub.org> writes:
> > either way, moving the pid file (or
> > socket, for that matter) from /tmp should be listed as a security related
> > requirement for v6.4 :)
>
> Huh?  There is no pid file being generated in /tmp (or anywhere else)
> at the moment.  If we do add one, it should not go into /tmp for the
> reasons I gave before.
>
> Where the Unix-domain socket file lives is an entirely separate issue.
>
> If we move the socket out of /tmp then we have just kicked away all the
> work we did to preserve backwards compatibility of the FE/BE protocol
> with existing clients.  Being able to talk to a 1.0 client isn't much
> good if you aren't listening where he's going to try to contact you.
> So I think I have to vote in favor of leaving the socket where it is.

I have been thinking about this.  First, we can easily use fopen(r+) to
check to see if the file exists, and if it does read the pid and do a
kill -0 to see if it is running.  If no one else does it, I will take it
on.

Second, where to put the pid file.  There is reason to put in /tmp,
because it will get cleared in a reboot, and because it is locking the
port number 5432.  There is also reason to put it in /data because you
can't have more than one postmaster running on a single data directory.

So, we really want to lock both places.  If this is going to make it
easier for people to run more than one postmaster, because it will
prevent/warn administrators when they try and put two postmasters in the
same data dir or port, I say create the pid lock files both places, and
give the admin a clear description of what he is doing wrong in each
case.


--
Bruce Momjian                          |  830 Blythe Avenue
maillist@candle.pha.pa.us              |  Drexel Hill, Pennsylvania 19026
  +  If your life is a hard drive,     |  (610) 353-9879(w)
  +  Christ can be your backup.        |  (610) 853-3000(h)

>
> > The Hermit Hacker <scrappy@hub.org> writes:
> > > either way, moving the pid file (or
> > > socket, for that matter) from /tmp should be listed as a security related
> > > requirement for v6.4 :)
> >
> > Huh?  There is no pid file being generated in /tmp (or anywhere else)
> > at the moment.  If we do add one, it should not go into /tmp for the
> > reasons I gave before.
> >
> > Where the Unix-domain socket file lives is an entirely separate issue.
> >
> > If we move the socket out of /tmp then we have just kicked away all the
> > work we did to preserve backwards compatibility of the FE/BE protocol
> > with existing clients.  Being able to talk to a 1.0 client isn't much
> > good if you aren't listening where he's going to try to contact you.
> > So I think I have to vote in favor of leaving the socket where it is.
>
> I have been thinking about this.  First, we can easily use fopen(r+) to
> check to see if the file exists, and if it does read the pid and do a
> kill -0 to see if it is running.  If no one else does it, I will take it
> on.
>
> Second, where to put the pid file.  There is reason to put in /tmp,
> because it will get cleared in a reboot, and because it is locking the
> port number 5432.  There is also reason to put it in /data because you
> can't have more than one postmaster running on a single data directory.
>
> So, we really want to lock both places.  If this is going to make it
> easier for people to run more than one postmaster, because it will
> prevent/warn administrators when they try and put two postmasters in the
> same data dir or port, I say create the pid lock files both places, and
> give the admin a clear description of what he is doing wrong in each
> case.
>


If the traffic on bugtraq is any indication, writing in /tmp is a
security exposure for daemons. Typical attack is:

 ln -s /etc/passwd /tmp/dumbrootprog.tmpfile

when dumprootprog runs it writes on /etc/passwd. Cool huh? Not so serious
in our case, as only pgsql owned files are at risk, but why take a chance.
This argues for the private pid file.

Also, before sprinkling files all over it is good to try to conform
to the FHS (File Hierarchy Standard) (see http://www.pathname.com/fhs/)
which is pretty easy to do and likely to make life easier later.

-dg

David Gould            dg@informix.com           510.628.3783 or 510.305.9468
Informix Software  (No, really)         300 Lakeside Drive  Oakland, CA 94612
 - If simplicity worked, the world would be overrun with insects. -

On Mon, 31 Aug 1998, David Gould wrote:

> Also, before sprinkling files all over it is good to try to conform
> to the FHS (File Hierarchy Standard) (see http://www.pathname.com/fhs/)
> which is pretty easy to do and likely to make life easier later.

    I just downloaded and skim'd the fhs notes, and it looks
reasonable...but, other then the current socket in /tmp, we don't
"splinkle files all over"...do we? *raised eyebrow*

dg@informix.com (David Gould) writes:
> If the traffic on bugtraq is any indication, writing in /tmp is a
> security exposure for daemons. Typical attack is:
>  ln -s /etc/passwd /tmp/dumbrootprog.tmpfile

A partial answer to this is to unlink /tmp/pidfile before trying to
create/write it.  There's still a window for the attack, but it's
mighty tiny.  (You do have to check that the unlink either succeeds
or fails with ENOENT --- in particular, an EPERM failure is trouble
for obvious reasons.)

I finally understand where Bruce is coming from on this point: he wants
pid lock files in *both* the data dir (to lock the database) and /tmp
(to lock the Unix-socket port number).  This makes sense to me, and I
agree that it'd become a lot safer to run multiple postmasters/databases
that way.  The data and the port are independent resources and each one
needs a lock.

I just came up with an idea that might help alleviate the /tmp security
exposure without creating a backwards-compatibility problem.  It works
like this:

1. During installation, create a subdirectory of /tmp to hold Postgres'
socket files and associated pid lockfiles.  This subdirectory should be
owned by the Postgres superuser and have permissions 755
(world-readable, writable only by Postgres superuser).  Maybe call it
/tmp/.pgsql --- the name should start with a dot to keep it out of the
way.  (Bruce points out that some systems clear /tmp during reboot, so
it might be that a postmaster will have to be prepared to recreate this
directory at startup --- anyone know if subdirectories of /tmp are
zapped too?  My system doesn't do that...)

2. When a postmaster fires up, it checks for/creates a pid lockfile in
the subdirectory, and also creates the socket file in the subdirectory.

3. For backwards compatibility with 1.0 clients, the socket file is also
hard-linked to /tmp/.s.PGSQL.port# (use an unlink() and link()).

This way, there's no security risk of overwriting someone else's file,
since we never create or write on a file in a directory we don't own, we
only try to link an already-existing socket file into /tmp.

I'd suggest that the pid and socket files be given names in /tmp/.pgsql
that don't start with dots --- no need to make them hard to see in that
directory.

We can change libpq to find the socket at /tmp/.pgsql/whatever, and
eventually we could perhaps drop the backwards-compatibility feature
of creating a link in /tmp.

> Also, before sprinkling files all over it is good to try to conform
> to the FHS (File Hierarchy Standard) (see http://www.pathname.com/fhs/)
> which is pretty easy to do and likely to make life easier later.

I notice that on my system, the X11 socket files in /tmp/.X11-unix are
actually symlinks to socket files in /usr/spool/sockets/X11.  I dunno if
it's worth our trouble to get into putting our sockets under /usr/spool
or /var/spool or whatever --- seems like another configuration choice to
mess up.  It'd be nice if the socket directory lived somewhere where the
parent dirs weren't world-writable, but this would mean one more thing
that you have to have root permissions for in order to install pgsql.

            regards, tom lane

   I just came up with an idea that might help alleviate the /tmp security
   exposure without creating a backwards-compatibility problem.  It works
   like this:

   1. During installation, create a subdirectory of /tmp to hold Postgres'
   socket files and associated pid lockfiles.  This subdirectory should be
   owned by the Postgres superuser and have permissions 755
   (world-readable, writable only by Postgres superuser).  Maybe call it
   /tmp/.pgsql --- the name should start with a dot to keep it out of the
   way.  (Bruce points out that some systems clear /tmp during reboot, so
   it might be that a postmaster will have to be prepared to recreate this
   directory at startup --- anyone know if subdirectories of /tmp are
   zapped too?  My system doesn't do that...)

   ...

   I notice that on my system, the X11 socket files in /tmp/.X11-unix are
   actually symlinks to socket files in /usr/spool/sockets/X11.  I dunno if
   it's worth our trouble to get into putting our sockets under /usr/spool
   or /var/spool or whatever --- seems like another configuration choice to
   mess up.  It'd be nice if the socket directory lived somewhere where the
   parent dirs weren't world-writable, but this would mean one more thing
   that you have to have root permissions for in order to install pgsql.

It seems like we need a directory for locks (= pid files) and one for
sockets (perhaps the same one).  I strongly suggest that the location
for these be configurable.  By default, it might make sense to put
them in ~pgsql/locks and ~pgsql/sockets.  It is easy (i.e., I'll be
glad to do it) to modify configure.in to take options like

         --lock-dir=/var/spool/lock
         --socket-dir=/var/spool/sockets

that set cc defines and have the code respond accordingly.  This way,
those who don't care (or don't have root access) can use the defaults,
whereas those with root access who like to keep locks and sockets in a
common place can do so easily.  Either way, multiple postmasters (all
compiled with the same options of course) can check the appropriate
locks in the well-known places.  Finally, drop the link into /tmp for
the old socket and document that it will be disappearing at some
point, and all is fine.

If someone wants to give me some guidance on what preprocessor
variables should be set in response to the above options (or something
like them), I'll do the configure stuff.

Cheers,
Brook

>    I just came up with an idea that might help alleviate the /tmp security
>    exposure without creating a backwards-compatibility problem.  It works
>    like this:
>
>    1. During installation, create a subdirectory of /tmp to hold Postgres'
>    socket files and associated pid lockfiles.  This subdirectory should be
>    owned by the Postgres superuser and have permissions 755
>    (world-readable, writable only by Postgres superuser).  Maybe call it
>    /tmp/.pgsql --- the name should start with a dot to keep it out of the
>    way.  (Bruce points out that some systems clear /tmp during reboot, so
>    it might be that a postmaster will have to be prepared to recreate this
>    directory at startup --- anyone know if subdirectories of /tmp are
>    zapped too?  My system doesn't do that...)
>
>    ...
>
>    I notice that on my system, the X11 socket files in /tmp/.X11-unix are
>    actually symlinks to socket files in /usr/spool/sockets/X11.  I dunno if
>    it's worth our trouble to get into putting our sockets under /usr/spool
>    or /var/spool or whatever --- seems like another configuration choice to
>    mess up.  It'd be nice if the socket directory lived somewhere where the
>    parent dirs weren't world-writable, but this would mean one more thing
>    that you have to have root permissions for in order to install pgsql.
>
> It seems like we need a directory for locks (= pid files) and one for
> sockets (perhaps the same one).  I strongly suggest that the location
> for these be configurable.  By default, it might make sense to put
> them in ~pgsql/locks and ~pgsql/sockets.  It is easy (i.e., I'll be
> glad to do it) to modify configure.in to take options like
>
>          --lock-dir=/var/spool/lock
>          --socket-dir=/var/spool/sockets
>
> that set cc defines and have the code respond accordingly.  This way,
> those who don't care (or don't have root access) can use the defaults,
> whereas those with root access who like to keep locks and sockets in a
> common place can do so easily.  Either way, multiple postmasters (all
> compiled with the same options of course) can check the appropriate
> locks in the well-known places.  Finally, drop the link into /tmp for
> the old socket and document that it will be disappearing at some
> point, and all is fine.
>
> If someone wants to give me some guidance on what preprocessor
> variables should be set in response to the above options (or something
> like them), I'll do the configure stuff.

I think we need to keep stuff in /tmp.  No reason to add more
configuration just for the sake of it.

If Tom says we can use symbolic links to sockets, why not just do that
for backward compatability.

On my system, all of /tmp is wiped out on reboot because it is a memory
file system.

Let each postmaster start up and create it's own directory as
/tmp/.pgsql.5432 or whatever port number they use.  In the directory, it
can put its socket and pid file.  We also put a pid file in the
postmaster's data directory, and give admins errors if they try anything
funny.

I don't think we can use a main /tmp/.pgsql directory because one system
can have multiple postmasters run by different users.  If we start
putting it in /usr/local/pgsql, we need the socket in a common place,
because the actual pgsql directory can be anywhere, so we need /tmp
anyway.  We can't have clients being configured for different pgsql
locations.

We still have a denial of service attach if someone creates a
.pgsql.5432 directory before we do, but the unix domain socket has to be
in a common place, and we can't just kick people out common territory
because we are not root.

Again, we could put it all in pgsql, but we still need that unix domain
socket, and that can't be configured into the client easily, so I think
we just have to live with /tmp.

Even if we put all the sockets in /tmp/.pgsql, we have the problem of
someone creating /tmp/.pgsql before we start, or after a reboot.  No way
around it that I can see.

In fact, I can't really see a reason for the separate directory in /tmp
vs. what we do now, except that we assume /tmp has the sticky bit, while
creation of our own directory with our own permissions fixes that, which
may be a win, especially because X and others do it that way.

--
Bruce Momjian                          |  830 Blythe Avenue
maillist@candle.pha.pa.us              |  Drexel Hill, Pennsylvania 19026
  +  If your life is a hard drive,     |  (610) 353-9879(w)
  +  Christ can be your backup.        |  (610) 853-3000(h)

> dg@informix.com (David Gould) writes:
> > If the traffic on bugtraq is any indication, writing in /tmp is a
> > security exposure for daemons. Typical attack is:
> >  ln -s /etc/passwd /tmp/dumbrootprog.tmpfile
>
> A partial answer to this is to unlink /tmp/pidfile before trying to
> create/write it.  There's still a window for the attack, but it's
> mighty tiny.  (You do have to check that the unlink either succeeds
> or fails with ENOENT --- in particular, an EPERM failure is trouble
> for obvious reasons.)

The real fix for this is to do an open(O_CREAT) to force creation at the
time of the open.  If it fails, try removing it and try again.

With our own directory, it is even easier.  If no pid locks,
unconditionally drop the directory, and recreate it with our
permissions, and if the create fails, try again.  We don't continue with
the postmaster until we drop and create the file or directory.  Can't
symlink hack around that because we force creation.


> 1. During installation, create a subdirectory of /tmp to hold Postgres'
> socket files and associated pid lockfiles.  This subdirectory should be
> owned by the Postgres superuser and have permissions 755
> (world-readable, writable only by Postgres superuser).  Maybe call it
> /tmp/.pgsql --- the name should start with a dot to keep it out of the

Problem is multiple postmasters with different users.

> way.  (Bruce points out that some systems clear /tmp during reboot, so
> it might be that a postmaster will have to be prepared to recreate this
> directory at startup --- anyone know if subdirectories of /tmp are
> zapped too?  My system doesn't do that...)

Get's wiped out.  Has to be done only by the postmaster.  Assuming
initdb is going to create something that is going to stay around in /tmp
is going to break.

>
> 2. When a postmaster fires up, it checks for/creates a pid lockfile in
> the subdirectory, and also creates the socket file in the subdirectory.

Forces creation.

>
> 3. For backwards compatibility with 1.0 clients, the socket file is also
> hard-linked to /tmp/.s.PGSQL.port# (use an unlink() and link()).

Good idea.

>
> This way, there's no security risk of overwriting someone else's file,
> since we never create or write on a file in a directory we don't own, we
> only try to link an already-existing socket file into /tmp.
>
> I'd suggest that the pid and socket files be given names in /tmp/.pgsql
> that don't start with dots --- no need to make them hard to see in that
> directory.

Yes, stuff in the directory doesn't need dots.

> We can change libpq to find the socket at /tmp/.pgsql/whatever, and
> eventually we could perhaps drop the backwards-compatibility feature
> of creating a link in /tmp.

OK

>
> > Also, before sprinkling files all over it is good to try to conform
> > to the FHS (File Hierarchy Standard) (see http://www.pathname.com/fhs/)
> > which is pretty easy to do and likely to make life easier later.
>
> I notice that on my system, the X11 socket files in /tmp/.X11-unix are
> actually symlinks to socket files in /usr/spool/sockets/X11.  I dunno if
> it's worth our trouble to get into putting our sockets under /usr/spool
> or /var/spool or whatever --- seems like another configuration choice to
> mess up.  It'd be nice if the socket directory lived somewhere where the
> parent dirs weren't world-writable, but this would mean one more thing
> that you have to have root permissions for in order to install pgsql.

Yes, too compilicated.

--
Bruce Momjian                          |  830 Blythe Avenue
maillist@candle.pha.pa.us              |  Drexel Hill, Pennsylvania 19026
  +  If your life is a hard drive,     |  (610) 353-9879(w)
  +  Christ can be your backup.        |  (610) 853-3000(h)

>
> On Mon, 31 Aug 1998, David Gould wrote:
>
> > Also, before sprinkling files all over it is good to try to conform
> > to the FHS (File Hierarchy Standard) (see http://www.pathname.com/fhs/)
> > which is pretty easy to do and likely to make life easier later.
>
>     I just downloaded and skim'd the fhs notes, and it looks
> reasonable...but, other then the current socket in /tmp, we don't
> "splinkle files all over"...do we? *raised eyebrow*
>

Well, perhaps not, thay might have been hyperbole, but I wanted to head it
off before it happened.

Also, what about moving the socket to the PG_DATA dir and then creating a
symlink to it in /tmp for older clients. New installations could optionally
not create (or remove) the symlink...

-dg

David Gould            dg@informix.com           510.628.3783 or 510.305.9468
Informix Software  (No, really)         300 Lakeside Drive  Oakland, CA 94612
 - If simplicity worked, the world would be overrun with insects. -

> >
> > On Mon, 31 Aug 1998, David Gould wrote:
> >
> > > Also, before sprinkling files all over it is good to try to conform
> > > to the FHS (File Hierarchy Standard) (see http://www.pathname.com/fhs/)
> > > which is pretty easy to do and likely to make life easier later.
> >
> >     I just downloaded and skim'd the fhs notes, and it looks
> > reasonable...but, other then the current socket in /tmp, we don't
> > "splinkle files all over"...do we? *raised eyebrow*
> >
>
> Well, perhaps not, thay might have been hyperbole, but I wanted to head it
> off before it happened.
>
> Also, what about moving the socket to the PG_DATA dir and then creating a
> symlink to it in /tmp for older clients. New installations could optionally
> not create (or remove) the symlink...

But then we have to compile the data directory into the client, or add
an option to specify the unix domain socket AND the data directory.  Not
worth it, I think.

--
Bruce Momjian                          |  830 Blythe Avenue
maillist@candle.pha.pa.us              |  Drexel Hill, Pennsylvania 19026
  +  If your life is a hard drive,     |  (610) 353-9879(w)
  +  Christ can be your backup.        |  (610) 853-3000(h)

>
> >
> > On Mon, 31 Aug 1998, David Gould wrote:
> >
> > > Also, before sprinkling files all over it is good to try to conform
> > > to the FHS (File Hierarchy Standard) (see http://www.pathname.com/fhs/)
> > > which is pretty easy to do and likely to make life easier later.
> >
> >     I just downloaded and skim'd the fhs notes, and it looks
> > reasonable...but, other then the current socket in /tmp, we don't
> > "splinkle files all over"...do we? *raised eyebrow*
> >
>
> Well, perhaps not, thay might have been hyperbole, but I wanted to head it
> off before it happened.
>
> Also, what about moving the socket to the PG_DATA dir and then creating a
> symlink to it in /tmp for older clients. New installations could optionally
> not create (or remove) the symlink...
>
> -dg
>
> David Gould            dg@informix.com           510.628.3783 or 510.305.9468
> Informix Software  (No, really)         300 Lakeside Drive  Oakland, CA 94612
>  - If simplicity worked, the world would be overrun with insects. -

In the fhs notes I read:


------------------------------------------------------------------------
5.9  /var/run : Run-time variable files

This directory contains system information files describing the system
since it was booted.  Files in this directory should be cleared (removed
or truncated as appropriate) at the beginning of the boot process.

Process identifier (PID) files, which were originally placed in /etc,
should be placed in /var/run.  The naming convention for PID files is
<program-name>.pid.  For example, the crond PID file is named
/var/run/crond.pid.

The internal format of PID files remains unchanged.  The file should
consist of the process identifier in ASCII-encoded decimal, followed by
a newline character.  For example, if crond was process number 25,
/var/run/crond.pid would contain three characters: two, five, and
newline.

Programs that read PID files should be somewhat flexible in what they
accept; i.e., they should ignore extra whitespace, leading zeroes,
absence of the trailing newline, or additional lines in the PID file.
Programs that create PID files should use the simple specification
located in the above paragraph.

The utmp file, which stores information about who is currently using the
system, is located in this directory.

Programs that maintain transient UNIX-domain sockets should place them
in this directory.
------------------------------------------------------------------------


It seems that if we want to follow this document there is no much to
discuss. But on my linux installation this directory is owned by root
so I don't see how pgsql could create a socket in it. What standards
are in use on other systems ?

--
Massimo Dal Zotto

+----------------------------------------------------------------------+
|  Massimo Dal Zotto                email:  dz@cs.unitn.it             |
|  Via Marconi, 141                 phone:  ++39-461-534251            |
|  38057 Pergine Valsugana (TN)     www:  http://www.cs.unitn.it/~dz/  |
|  Italy                            pgp:  finger dz@tango.cs.unitn.it  |
+----------------------------------------------------------------------+

> Also, what about moving the socket to the PG_DATA dir and then
> creating a symlink to it in /tmp for older clients.

Clients don't have visibility into $PG_DATA, do they? Just ran into this
working on the ODBC interface, trying to find a place for a system-wide
configuration file. Ended up putting it in $POSTGRESDIR by default.

The /var/run option (or something similar) seems to be a good way to
head, if we can get enough support on the different platforms. Actually,
this could be an autoconf test, couldn't it?

                    - Thomas

Bruce Momjian writes:

> I have been thinking about this.  First, we can easily use fopen(r+) to
> check to see if the file exists, and if it does read the pid and do a
> kill -0 to see if it is running.  If no one else does it, I will take it
> on.

It is better to use open with the O_CREAT and O_EXCL set.  If the file does not
exist it will be created and the PID can be written to it.  If the file exists
then the call will fail, at which point it can be opened with fread, and the
PID it contains can be checked to see if it still exists with kill.  The open
call has the added advantage that 'The check for the existence of the file and
the creation of the file if it does not exist is atomic with respect to other
processes executing open naming the same filename in the same directory with
O_EXCL and O_CREAT set.' [from the UnixAWare 7 man page, open(2)].

Also, you can't just delete the file, create it and write the your PID to it
and assume that you have the lock, you need to close the file, sleep some
small amount of time and then open and read the file to see if you still have
the lock.  If you like, I can take this task on.

Oh, the postmaster must clear the PID when it exits.

>
> Second, where to put the pid file.  There is reason to put in /tmp,
> because it will get cleared in a reboot, and because it is locking the
> port number 5432.  There is also reason to put it in /data because you
> can't have more than one postmaster running on a single data directory.
>
> So, we really want to lock both places.  If this is going to make it
> easier for people to run more than one postmaster, because it will
> prevent/warn administrators when they try and put two postmasters in the
> same data dir or port, I say create the pid lock files both places, and
> give the admin a clear description of what he is doing wrong in each
> case.

IHMO, the pid should be put in the data directory.  The reasoning that it will get cleared in a reboot is not sufficent
sincethe logic used to create the PID file will delete it if the PID it contains is not a running process.  Besides, I
haveused systems where /tmp was not cleared out on a re-boot (for various reasons).  Also, I would rather have a script
thatexplicitly removes the PID locking file at system statup (if it exists), in which case, it doesn't matter where it
resides.
--
____       | Billy G. Allie    | Domain....: Bill.Allie@mug.org
|  /|      | 7436 Hartwell     | Compuserve: 76337,2061
|-/-|----- | Dearborn, MI 48126| MSN.......: B_G_Allie@email.msn.com
|/  |LLIE  | (313) 582-1540    |

Massimo, do we have all the documentation change for the new features
you added.  I am thinking particularly about the new -d debug levels,
but there may be more.



--
  Bruce Momjian                        |  http://www.op.net/~candle
  maillist@candle.pha.pa.us            |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

On Sun, 30 Aug 1998, Tom Lane wrote:

> The Hermit Hacker <scrappy@hub.org> writes:
> > either way, moving the pid file (or
> > socket, for that matter) from /tmp should be listed as a security related
> > requirement for v6.4 :)
> 
> Huh?  There is no pid file being generated in /tmp (or anywhere else)
> at the moment.  If we do add one, it should not go into /tmp for the
> reasons I gave before.
> 
> Where the Unix-domain socket file lives is an entirely separate issue.
> 
> If we move the socket out of /tmp then we have just kicked away all the
> work we did to preserve backwards compatibility of the FE/BE protocol
> with existing clients.  Being able to talk to a 1.0 client isn't much
> good if you aren't listening where he's going to try to contact you.
> So I think I have to vote in favor of leaving the socket where it is.

Let me put my vote in...: $PGSQLHOME/run/{pgsql.pid,pgsql.pid.socket}

Screw backwards compatibility...v6.5 becomes v7.0 *shrug*

Marc G. Fournier                                
Systems Administrator @ hub.org 
primary: scrappy@hub.org           secondary: scrappy@{freebsd|postgresql}.org 

The Hermit Hacker <scrappy@hub.org> writes:
> On Sun, 30 Aug 1998, Tom Lane wrote:
>> So I think I have to vote in favor of leaving the socket where it is.

> Let me put my vote in...: $PGSQLHOME/run/{pgsql.pid,pgsql.pid.socket}
> Screw backwards compatibility...v6.5 becomes v7.0 *shrug*

Catching up on back email, Marc?

I think the end consensus of that thread was that we should move the
socket file to someplace safer than /tmp (exactly where being a
configure-time choice), *and* optionally put a softlink to it in /tmp
for backwards compatibility with old clients.  Use of the /tmp link
would be deprecated because of possible security risks, but it could
continue to work for as long as a particular installation needed to
keep that option enabled.

Actually getting it done doesn't seem to have happened :-(.  I think
we were all viewing this as part & parcel of fixing the lockfile issues,
which no one got exercised enough to do.  Too many higher-priority
tasks...
        regards, tom lane

On Thu, 29 Oct 1998, Tom Lane wrote:

> The Hermit Hacker <scrappy@hub.org> writes:
> > On Sun, 30 Aug 1998, Tom Lane wrote:
> >> So I think I have to vote in favor of leaving the socket where it is.
> 
> > Let me put my vote in...: $PGSQLHOME/run/{pgsql.pid,pgsql.pid.socket}
> > Screw backwards compatibility...v6.5 becomes v7.0 *shrug*
> 
> Catching up on back email, Marc?
I have a cold and am partially delirious? :)  The cold part is
accurate, at least :)

Marc G. Fournier                                
Systems Administrator @ hub.org 
primary: scrappy@hub.org           secondary: scrappy@{freebsd|postgresql}.org 

> The Hermit Hacker <scrappy@hub.org> writes:
> > On Sun, 30 Aug 1998, Tom Lane wrote:
> >> So I think I have to vote in favor of leaving the socket where it is.
> 
> > Let me put my vote in...: $PGSQLHOME/run/{pgsql.pid,pgsql.pid.socket}
> > Screw backwards compatibility...v6.5 becomes v7.0 *shrug*
> 
> Catching up on back email, Marc?
> 
> I think the end consensus of that thread was that we should move the
> socket file to someplace safer than /tmp (exactly where being a
> configure-time choice), *and* optionally put a softlink to it in /tmp
> for backwards compatibility with old clients.  Use of the /tmp link
> would be deprecated because of possible security risks, but it could
> continue to work for as long as a particular installation needed to
> keep that option enabled.

Just a reminder that if you move it, all clients must know the new
location, or somehow the location must be accessable by the clients.


--  Bruce Momjian                        |  http://www.op.net/~candle maillist@candle.pha.pa.us            |  (610)
853-3000+  If your life is a hard drive,     |  830 Blythe Avenue +  Christ can be your backup.        |  Drexel Hill,
Pennsylvania19026
 

Bruce Momjian <maillist@candle.pha.pa.us> writes:
>> I think the end consensus of that thread was that we should move the
>> socket file to someplace safer than /tmp (exactly where being a
>> configure-time choice), *and* optionally put a softlink to it in /tmp
>> for backwards compatibility with old clients.

> Just a reminder that if you move it, all clients must know the new
> location, or somehow the location must be accessable by the clients.

That's why I said *configure* time choice.  The socket location has to
be compiled into both frontends and backends.  But there's no reason
why the location couldn't be set during site configure, rather than
being hard-wired as /tmp/.s.PGSQL.####.

Supporting a softlink in /tmp would be helpful if an installation
doesn't want to rebuild all their existing frontend apps right away
(or at least all the ones that are on the server's local machine and use
Unix-socket connections).  Fortunately, cross-machine connections don't
use the socket file, or it wouldn't really be practical to make the
socket location site-dependent.
        regards, tom lane

On Fri, 30 Oct 1998, Bruce Momjian wrote:

> > The Hermit Hacker <scrappy@hub.org> writes:
> > > On Sun, 30 Aug 1998, Tom Lane wrote:
> > >> So I think I have to vote in favor of leaving the socket where it is.
> > 
> > > Let me put my vote in...: $PGSQLHOME/run/{pgsql.pid,pgsql.pid.socket}
> > > Screw backwards compatibility...v6.5 becomes v7.0 *shrug*
> > 
> > Catching up on back email, Marc?
> > 
> > I think the end consensus of that thread was that we should move the
> > socket file to someplace safer than /tmp (exactly where being a
> > configure-time choice), *and* optionally put a softlink to it in /tmp
> > for backwards compatibility with old clients.  Use of the /tmp link
> > would be deprecated because of possible security risks, but it could
> > continue to work for as long as a particular installation needed to
> > keep that option enabled.
> 
> Just a reminder that if you move it, all clients must know the new
> location, or somehow the location must be accessable by the clients.
Which is only relevant if using Unix-domain sockets...

Marc G. Fournier                                
Systems Administrator @ hub.org 
primary: scrappy@hub.org           secondary: scrappy@{freebsd|postgresql}.org