Thread: abstract Unix-domain sockets

During the discussion on Unix-domain sockets on Windows, someone pointed 
out[0] abstract Unix-domain sockets.  This is a variant of the normal 
Unix-domain sockets that don't use the file system but a separate 
"abstract" namespace.  At the user interface, such sockets are 
represented by names starting with "@".  I took a look at this and it 
wasn't hard to get working, so here is a patch.  It's supposed to be 
supported on Linux and Windows right now, but I haven't tested on Windows.

I figure, there are so many different deployment options nowadays, this 
could be useful somewhere.  It relieves you from dealing with the file 
system, you don't have to set up /tmp or something under /var/run, you 
don't need to make sure file system permissions are right.  Also, there 
is no need for a lock file or file cleanup.  (Unlike file-system 
namespace sockets, abstract namespace sockets give an EADDRINUSE when 
trying to bind to a name already in use.)  Conversely, of course, you 
don't get to use file-system permissions to manage access to the socket, 
but that isn't essential functionality, so it's a trade-off users can 
make on their own.

And then some extra patches for surrounding cleanup.  During testing I 
noticed that the bind() failure hint "Is another postmaster already 
running ..." was shown in inappropriate situations, so I changed that to 
only show for EADDRINUSE errors.  (Maybe other error codes could be 
appropriate, but I couldn't find any more.)

And then looking for other uses of EADDRINUSE I found some dead 
Windows-related code that can be cleaned up.


[0]: 
https://www.postgresql.org/message-id/20191218142419.fvv4ikm4wq4gnkco@isc.upenn.edu

-- 
Peter Eisentraut              http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Updated patch set after some conflicts had emerged.

On 2020-10-09 09:28, Peter Eisentraut wrote:
> During the discussion on Unix-domain sockets on Windows, someone pointed
> out[0] abstract Unix-domain sockets.  This is a variant of the normal
> Unix-domain sockets that don't use the file system but a separate
> "abstract" namespace.  At the user interface, such sockets are
> represented by names starting with "@".  I took a look at this and it
> wasn't hard to get working, so here is a patch.  It's supposed to be
> supported on Linux and Windows right now, but I haven't tested on Windows.
> 
> I figure, there are so many different deployment options nowadays, this
> could be useful somewhere.  It relieves you from dealing with the file
> system, you don't have to set up /tmp or something under /var/run, you
> don't need to make sure file system permissions are right.  Also, there
> is no need for a lock file or file cleanup.  (Unlike file-system
> namespace sockets, abstract namespace sockets give an EADDRINUSE when
> trying to bind to a name already in use.)  Conversely, of course, you
> don't get to use file-system permissions to manage access to the socket,
> but that isn't essential functionality, so it's a trade-off users can
> make on their own.
> 
> And then some extra patches for surrounding cleanup.  During testing I
> noticed that the bind() failure hint "Is another postmaster already
> running ..." was shown in inappropriate situations, so I changed that to
> only show for EADDRINUSE errors.  (Maybe other error codes could be
> appropriate, but I couldn't find any more.)
> 
> And then looking for other uses of EADDRINUSE I found some dead
> Windows-related code that can be cleaned up.

This last piece has been committed.

> 
> 
> [0]:
> https://www.postgresql.org/message-id/20191218142419.fvv4ikm4wq4gnkco@isc.upenn.edu

-- 
Peter Eisentraut              http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On Thu, Oct 22, 2020 at 09:03:49AM +0200, Peter Eisentraut wrote:
> On 2020-10-09 09:28, Peter Eisentraut wrote:
>> During the discussion on Unix-domain sockets on Windows, someone pointed
>> out[0] abstract Unix-domain sockets.  This is a variant of the normal
>> Unix-domain sockets that don't use the file system but a separate
>> "abstract" namespace.  At the user interface, such sockets are
>> represented by names starting with "@".  I took a look at this and it
>> wasn't hard to get working, so here is a patch.  It's supposed to be
>> supported on Linux and Windows right now, but I haven't tested on Windows.

Yeah, peaking at the Windows docs, what you are trying to do here
should be supported (please note that I have not tested ).  One
reference:
https://devblogs.microsoft.com/commandline/af_unix-comes-to-windows/

>> And then some extra patches for surrounding cleanup.  During testing I
>> noticed that the bind() failure hint "Is another postmaster already
>> running ..." was shown in inappropriate situations, so I changed that to
>> only show for EADDRINUSE errors.  (Maybe other error codes could be
>> appropriate, but I couldn't find any more.)
>>
>> And then looking for other uses of EADDRINUSE I found some dead
>> Windows-related code that can be cleaned up.
>
> This last piece has been committed.

+       <para>
+        A value that starts with <literal>@</literal> specifies that a
+        Unix-domain socket in the abstract namespace should be created
+        (currently supported on Linux and Windows).  In that case, this value
+        does not specify a <quote>directory</quote> but a prefix from which
+        the actual socket name is computed in the same manner as for the
+        file-system namespace.  While the abstract socket name prefix can be
+        chosen freely, since it is not a file-system location, the convention
+        is to nonetheless use file-system-like values such as
+        <literal>@/tmp</literal>.
+       </para>

As abstract namespaces don't have permissions, anyone knowing the name
of the path, which should be unique, can have an access to the server.
Do you think that the documentation should warn the user about that?
This feature is about easing the management part of the socket paths
while throwing away the security aspect of it.

When attempting to start a server that listens to the same port and
uses the same abstract path, the second server started still shows
a hint referring to a file that does not exist:
LOG: could not bind Unix address "@tmp/.s.PGSQL.5432": Address already
in use
HINT: Is another postmaster already running on port 5432? If not,
remove socket file "@tmp/.s.PGSQL.5432" and retry.

Instead of showing paths with at signs, wouldn't it be better to
mention it is an abstract socket address?

I am not sure that 0002 is an improvement.  It would be more readable
to move the part choosing what hint is adapted into a first block that
selects the hint string rather than have the whole thing in a single
elog() call.
--
Michael

On 2020-11-09 07:08, Michael Paquier wrote:
> As abstract namespaces don't have permissions, anyone knowing the name
> of the path, which should be unique, can have an access to the server.
> Do you think that the documentation should warn the user about that?
> This feature is about easing the management part of the socket paths
> while throwing away the security aspect of it.

We could modify the documentation further.  But note that the 
traditional way of putting the socket into /tmp has the same properties, 
so this shouldn't be a huge shock.

> When attempting to start a server that listens to the same port and
> uses the same abstract path, the second server started still shows
> a hint referring to a file that does not exist:
> LOG: could not bind Unix address "@tmp/.s.PGSQL.5432": Address already
> in use
> HINT: Is another postmaster already running on port 5432? If not,
> remove socket file "@tmp/.s.PGSQL.5432" and retry.
> 
> Instead of showing paths with at signs, wouldn't it be better to
> mention it is an abstract socket address?

The @ is the standard way of representing this in the user interface and 
the configuration, so it seems sensible to me that way.

> I am not sure that 0002 is an improvement.  It would be more readable
> to move the part choosing what hint is adapted into a first block that
> selects the hint string rather than have the whole thing in a single
> elog() call.

Can you sketch how you would structure this?  I realize it's not very 
elegant, but I couldn't come up with a better way that didn't involve 
having to duplicate some of the error messages into multiple branches.

-- 
Peter Eisentraut
2ndQuadrant, an EDB company
https://www.2ndquadrant.com/

On 11/9/20 9:04 AM, Peter Eisentraut wrote:
> On 2020-11-09 07:08, Michael Paquier wrote:
>> As abstract namespaces don't have permissions, anyone knowing the name
>> of the path, which should be unique, can have an access to the server.
>> Do you think that the documentation should warn the user about that?
>> This feature is about easing the management part of the socket paths
>> while throwing away the security aspect of it.
> 
> We could modify the documentation further.  But note that the 
> traditional way of putting the socket into /tmp has the same properties, 
> so this shouldn't be a huge shock.

One issue with them is that they interact differently with kernel 
namespaces than normal unix sockets do. Abstract sockets are handled by 
the network namespaces, and not the file system namespaces. But I am not 
sure that this is our job to document.

Andreas

On Mon, Nov 09, 2020 at 09:04:24AM +0100, Peter Eisentraut wrote:
> On 2020-11-09 07:08, Michael Paquier wrote:
> The @ is the standard way of representing this in the user interface and the
> configuration, so it seems sensible to me that way.

Ok.

> Can you sketch how you would structure this?  I realize it's not very
> elegant, but I couldn't come up with a better way that didn't involve having
> to duplicate some of the error messages into multiple branches.

I think that I would use a StringInfo to build each sentence of the
hint separately.  The first sentence, "Is another postmaster already
running on port %d?" is already known.  Then the second sentence could
be built depending on the two other conditions.  FWIW, I think that it
is confusing to mention in the hint to remove a socket file that
cannot be removed.
--
Michael

On 2020-11-10 07:24, Michael Paquier wrote:
>> Can you sketch how you would structure this?  I realize it's not very
>> elegant, but I couldn't come up with a better way that didn't involve having
>> to duplicate some of the error messages into multiple branches.
> 
> I think that I would use a StringInfo to build each sentence of the
> hint separately.  The first sentence, "Is another postmaster already
> running on port %d?" is already known.  Then the second sentence could
> be built depending on the two other conditions.

I'm not sure concatenating sentences like that is okay for translatability.

> FWIW, I think that it
> is confusing to mention in the hint to remove a socket file that
> cannot be removed.

Thinking about it further, I think the hint in the Unix-domain socket 
case is bogus.  A socket in the file-system namespace never reports 
EADDRINUSE anyway, it just overwrites the file.  For sockets in the 
abstract namespace, you can get this error, but of course there is no 
file to remove.

Perhaps we should change the hint in both the Unix and the IP cases to:

"Is another postmaster already running at this address?"

(This also resolves the confusing reference to "port" in the Unix case.)

Or we just drop the hint in the Unix case.  The primary error message is 
clear enough.

-- 
Peter Eisentraut
2ndQuadrant, an EDB company
https://www.2ndquadrant.com/

On Wed, Nov 11, 2020 at 01:39:17PM +0100, Peter Eisentraut wrote:
> Thinking about it further, I think the hint in the Unix-domain socket case
> is bogus.  A socket in the file-system namespace never reports EADDRINUSE
> anyway, it just overwrites the file.  For sockets in the abstract namespace,
> you can get this error, but of course there is no file to remove.
>
> Perhaps we should change the hint in both the Unix and the IP cases to:
>
> "Is another postmaster already running at this address?"
> (This also resolves the confusing reference to "port" in the Unix case.)

Er, it is perfectly possible for two postmasters to use the same unix
socket path, abstract or not, as long as they listen to different
ports (all nodes in a single TAP test do that for example).  So we
should keep a reference to the port used in the log message, no?

> Or we just drop the hint in the Unix case.  The primary error message is
> clear enough.

Dropping the hint for the abstract case sounds fine to me.
--
Michael

On 2020-11-12 08:12, Michael Paquier wrote:
> On Wed, Nov 11, 2020 at 01:39:17PM +0100, Peter Eisentraut wrote:
>> Thinking about it further, I think the hint in the Unix-domain socket case
>> is bogus.  A socket in the file-system namespace never reports EADDRINUSE
>> anyway, it just overwrites the file.  For sockets in the abstract namespace,
>> you can get this error, but of course there is no file to remove.
>>
>> Perhaps we should change the hint in both the Unix and the IP cases to:
>>
>> "Is another postmaster already running at this address?"
>> (This also resolves the confusing reference to "port" in the Unix case.)
> Er, it is perfectly possible for two postmasters to use the same unix
> socket path, abstract or not, as long as they listen to different
> ports (all nodes in a single TAP test do that for example).  So we
> should keep a reference to the port used in the log message, no?

"Port" is not a real thing for Unix-domain sockets, it's just something 
we use internally and append to the socket file.  The error message is 
currently something like

ERROR:  could not bind Unix address "/tmp/.s.PGSQL.5432": Address 
already in use
HINT:  Is another postmaster already running on port 5432? If not, 
remove socket file "/tmp/.s.PGSQL.5432" and retry.

So the mention of the "port" doesn't really add any information here and 
just introduces new terminology that isn't really relevant.

My idea is to change the message to:

ERROR:  could not bind Unix address "/tmp/.s.PGSQL.5432": Address 
already in use
HINT:  Is another postmaster already running at this address?

-- 
Peter Eisentraut
2ndQuadrant, an EDB company
https://www.2ndquadrant.com/

On Tue, Nov 17, 2020 at 11:18:12PM +0100, Peter Eisentraut wrote:
> So the mention of the "port" doesn't really add any information here and
> just introduces new terminology that isn't really relevant.
>
> My idea is to change the message to:
>
> ERROR:  could not bind Unix address "/tmp/.s.PGSQL.5432": Address already in
> use
> HINT:  Is another postmaster already running at this address?

Are you saying that you would remove the hint telling to remove the
socket file even for the case of non-abstract files?  For abstract
paths, this makes sense.  For both, removing the "port" part is indeed
a good idea as long as you keep around the full socket file name.
--
Michael

On 2020-11-18 04:35, David G. Johnston wrote:
> Given that "port" is a postgresql.conf setting its use here (and 
> elsewhere) should be taken to mean the value of that specific variable.  
> To that end, I find the current description of port to be lacking - it 
> should mention its usage as a qualifier when dealing with unix socket 
> files (in addition to the existing wording under unix_socket_directories).
> 
> If we are going to debate semantics here "bind unix address" doesn't 
> seem correct.  could not create Unix socket file /tmp/.s.PGSQL.5432, it 
> already exists.
> 
> The hint would be better written: Is another postmaster running with 
> unix_socket_directories = /tmp and port = 5432?  If not, remove the unix 
> socket file /tmp/.s.PGSQL.5432 and retry.
> 
> I don't see much benefit in trying to share logic/wording between the 
> various messages and hints for the different ways the server can 
> establish communication points.
> 
> I agree that there isn't a useful hint for the abstract case as it 
> shouldn't happen unless there is indeed another running instance with 
> the same configuration.  Though a hint similar to the above, but without 
> the "remove and retry" bit, probably wouldn't hurt.

I think we are getting a bit sidetracked here with the message wording. 
The reason I looked at this was that "remove socket file and retry" is 
never an appropriate action with abstract sockets.  And on further 
analysis, it is never an appropriate action with any Unix-domain socket 
(because with file system namespace sockets, you never get an 
EADDRINUSE, so it's dead code).  So my proposal here is to just delete 
that line from the hint and leave the rest the same.  There could be 
value in further refining and rephrasing this, but it ought to be a 
separate thread.

-- 
Peter Eisentraut
2ndQuadrant, an EDB company
https://www.2ndquadrant.com/

On 2020-11-18 04:35, David G. Johnston wrote:

I agree that there isn't a useful hint for the abstract case as it shouldn't happen unless there is indeed another running instance with the same configuration.  Though a hint similar to the above, but without the "remove and retry" bit, probably wouldn't hurt.

I think we are getting a bit sidetracked here with the message wording. The reason I looked at this was that "remove socket file and retry" is never an appropriate action with abstract sockets.  And on further analysis, it is never an appropriate action with any Unix-domain socket (because with file system namespace sockets, you never get an EADDRINUSE, so it's dead code).  So my proposal here is to just delete that line from the hint and leave the rest the same.  There could be value in further refining and rephrasing this, but it ought to be a separate thread.

On 2020-11-20 18:23, David G. Johnston wrote:
> If there is dead code there is an underlying problem to 
> address/discover, not just removing the dead code.  In this case are we 
> saying that a new server won’t ever fail to start because the socket 
> file exists but instead will just clobber the file with its own?  

Yes.  (In practice, there will be an error with respect to the lock file 
before you even get to that question, but that is different code elsewhere.)

> Because given that error, and a server process that failed to clean up 
> after itself, the correction to take would indeed seem to be to manually 
> remove the file as the hint says.  IOW, fix the code, not the message?

I don't understand that.

-- 
Peter Eisentraut
2ndQuadrant, an EDB company
https://www.2ndquadrant.com/

On Fri, Nov 20, 2020 at 04:06:43PM +0100, Peter Eisentraut wrote:
> I think we are getting a bit sidetracked here with the message wording. The
> reason I looked at this was that "remove socket file and retry" is never an
> appropriate action with abstract sockets.  And on further analysis, it is
> never an appropriate action with any Unix-domain socket (because with file
> system namespace sockets, you never get an EADDRINUSE, so it's dead code).
> So my proposal here is to just delete that line from the hint and leave the
> rest the same.

Reading again this thread, +1 on that.
--
Michael

On 2020-11-23 17:00, David G. Johnston wrote:
> So presently there is no functioning code to prevent two PostgreSQL 
> instances from using the same socket so long as they do not also use the 
> same data directory?  We only handle the case of an unclean crash - 
> where the pid and socket are both left behind - having the system tell 
> the user to remove the pid lock file but then auto-replacing the socket 
> (I was conflating the behavior with the pid lock file and the socket file).
> 
> I would expect that we handle port misconfiguration also, by not 
> auto-replacing the socket and instead have the existing error message 
> (with modified hint) remain behind.  This provides behavior consistent 
> with TCP port binding.  Or is it the case that we always attempt to bind 
> the TCP/IP port, regardless of the presence of a socket file, in which 
> case the failure for port binding does cover the socket situation as 
> well?  If this is the case, pointing that out in [1] and a code comment, 
> while removing that particular error as "dead code", would work.

We're subject to whatever the kernel behavior is.  If the kernel doesn't 
report address conflicts for Unix-domain sockets, then we can't do 
anything about that.  Having an error message ready in case the kernel 
does report such an error is not useful if it never does.

-- 
Peter Eisentraut
2ndQuadrant, an EDB company
https://www.2ndquadrant.com/

On 2020-11-24 02:57, Michael Paquier wrote:
> On Fri, Nov 20, 2020 at 04:06:43PM +0100, Peter Eisentraut wrote:
>> I think we are getting a bit sidetracked here with the message wording. The
>> reason I looked at this was that "remove socket file and retry" is never an
>> appropriate action with abstract sockets.  And on further analysis, it is
>> never an appropriate action with any Unix-domain socket (because with file
>> system namespace sockets, you never get an EADDRINUSE, so it's dead code).
>> So my proposal here is to just delete that line from the hint and leave the
>> rest the same.
> 
> Reading again this thread, +1 on that.

committed, thanks

-- 
Peter Eisentraut
2ndQuadrant, an EDB company
https://www.2ndquadrant.com/

On 2020-11-24 16:49, David G. Johnston wrote:
> On Tue, Nov 24, 2020 at 8:45 AM Peter Eisentraut 
> <peter.eisentraut@2ndquadrant.com 
> <mailto:peter.eisentraut@2ndquadrant.com>> wrote:
> 
>     We're subject to whatever the kernel behavior is.  If the kernel
>     doesn't
>     report address conflicts for Unix-domain sockets, then we can't do
>     anything about that.  Having an error message ready in case the kernel
>     does report such an error is not useful if it never does.
> 
> 
> It's a file, we can check for its existence in user-space.

But not without race conditions.  That's why we have the separate lock 
file, so we can do this properly.

Also, even if one were to add code to check the file existence first, 
this would be separate code and would not affect the behavior of the 
bind() call that we are discussing here.

-- 
Peter Eisentraut
2ndQuadrant, an EDB company
https://www.2ndquadrant.com/