Thread: Hostnames in pg_hba.conf

Hi there,<br /><br />I've been working on a patch to add hostname support to pg_hba.conf. It's not ready for public
displayyet, but I would just like to run a couple of issues / discussion points past everybody.<br /><br /> ISSUE #1:
Performance/ caching<br /><br />At present, I've simply not added caching. The reasoning for this is as follows:<br
/>(a)getaddrinfo doesn't tell us about expiry, so when do you refresh?<br />(b) If you put the cache in the postmaster,
itwill not work for exec-based backends as opposed to fork-based backends, since those read pg_hba.conf every time they
areexec'ed.<br /> (c) If you put this in the postmaster, the postmaster will have to update the cache every once in a
while,which may be slow and which may prevent new connections while the cache update takes place.<br />(d) Outdated
cacheentries may inexplicably and without any logging choose the wrong rule for some clients. Big aargh: people will
startusing this to specify 'deny' rules based on host names.<br /><br />If you COULD get expiry info out of getaddrinfo
youcould potentially store this info in a table or something like that, and have it updated by the backends? But that's
wayover my head for now. ISTM that this stuff may better be handled by a locally-running caching DNS server, if people
haveperformance issues with the lack of caching. These local caching DNS servers can also handle expiry correctly,
etcetera.<br/><br />We should of course still take care to look up a given hostname only once for each connection
request.<br/><br />ISSUE #2: Reverse lookup?<br /><br />There was a suggestion on the TODO list on the wiki, which
basicallysaid that maybe we could use reverse lookup to find "the" hostname and then check for that hostname in the
list.I think that won't work, since IPs can go by many names and may not support reverse lookup for some hostnames
(/etc/hostsanybody?). Furthermore, due to the top-to-bottom processing of pg_hba.conf, you CANNOT SKIP entries that
mightpossibly match. For instance, if the third line is for host "<a href="http://foo.example.com">foo.example.com</a>"
andthe fifth line is for "<a href="http://bar.example.com">bar.example.com</a>", both lines may apply to the same IP,
andyou still HAVE to check the first one, even if reverse lookup turns up the second host name. So it doesn't save you
anylookups, it just costs an extra one.<br /><br />ISSUE #3: Multiple hostnames?<br /><br />Currently, a pg_hba entry
listsan IP / netmask combination. I would suggest allowing lists of hostnames in the entries, so that you can at least
mimicthe "match multiple hosts by a single rule". Any reason not to do this?<br /><br />Comments / bright ideas are
welcome,especially regarding issue #1.<br /><br />Cheers,<br />Bart<br /> 

Bart Samwel <bart@samwel.tk> wrote:
> I've been working on a patch to add hostname support to
> pg_hba.conf.
> At present, I've simply not added caching.
Perhaps you could just recommend using nscd (or similar).
> There was a suggestion on the TODO list on the wiki, which
> basically said that maybe we could use reverse lookup to find
> "the" hostname and then check for that hostname in the list. I
> think that won't work, since IPs can go by many names and may not
> support reverse lookup for some hostnames (/etc/hosts anybody?).
Right.  Any reverse lookup should be, at best, for display in error
messages or logs.  There can be zero to many names for an IP
address.
> Currently, a pg_hba entry lists an IP / netmask combination. I
> would suggest allowing lists of hostnames in the entries, so that
> you can at least mimic the "match multiple hosts by a single
> rule". Any reason not to do this?
I can't see any reason other than code complexity.
-Kevin

On 02/11/2010 08:13 AM, Bart Samwel wrote:<br /><blockquote
cite="mid:ded01eb21002110513n296d60b7me5255820a69a4bff@mail.gmail.com"type="cite">ISSUE #1: Performance / caching<br
/><br/> At present, I've simply not added caching. The reasoning for this is as follows:<br /> (a) getaddrinfo doesn't
tellus about expiry, so when do you refresh?<br /> (b) If you put the cache in the postmaster, it will not work for
exec-basedbackends as opposed to fork-based backends, since those read pg_hba.conf every time they are exec'ed.<br />
(c)If you put this in the postmaster, the postmaster will have to update the cache every once in a while, which may be
slowand which may prevent new connections while the cache update takes place.<br /> (d) Outdated cache entries may
inexplicablyand without any logging choose the wrong rule for some clients. Big aargh: people will start using this to
specify'deny' rules based on host names.<br /><br /> If you COULD get expiry info out of getaddrinfo you could
potentiallystore this info in a table or something like that, and have it updated by the backends? But that's way over
myhead for now. ISTM that this stuff may better be handled by a locally-running caching DNS server, if people have
performanceissues with the lack of caching. These local caching DNS servers can also handle expiry correctly,
etcetera.<br/><br /> We should of course still take care to look up a given hostname only once for each connection
request.<br/></blockquote><br /> You should cache for some minimal amount of time or some minimal number of records -
evenif it's just one minute, and even if it's a fixed length LRU sorted list. This would deal with situations where a
newconnection is raised several times a second (some types of load). For connections raised once a minute or less, the
benefitof caching is far less. But, this can be a feature tagged on later if necessary and doesn't need to gate the
feature.<br/><br /> Many UNIX/Linux boxes have some sort of built-in cache, sometimes persistent, sometimes shared. On
myLinux box, I have nscd - "name server caching daemon" - which should be able to cache these sorts of lookups. I
believeit is used for things as common as mapping uid to username in output of "/bin/ls -l", so it does need to be
prettyfast.<br /><br /> The difference between in process cache and something like "nscd" is the inter-process
communicationrequired to use "nscd".<br /><br /><br /><blockquote
cite="mid:ded01eb21002110513n296d60b7me5255820a69a4bff@mail.gmail.com"type="cite">ISSUE #2: Reverse lookup?<br /><br />
Therewas a suggestion on the TODO list on the wiki, which basically said that maybe we could use reverse lookup to find
"the"hostname and then check for that hostname in the list. I think that won't work, since IPs can go by many names and
maynot support reverse lookup for some hostnames (/etc/hosts anybody?). Furthermore, due to the top-to-bottom
processingof pg_hba.conf, you CANNOT SKIP entries that might possibly match. For instance, if the third line is for
host"<a href="http://foo.example.com" moz-do-not-send="true">foo.example.com</a>" and the fifth line is for "<a
href="http://bar.example.com"moz-do-not-send="true">bar.example.com</a>", both lines may apply to the same IP, and you
stillHAVE to check the first one, even if reverse lookup turns up the second host name. So it doesn't save you any
lookups,it just costs an extra one.<br /></blockquote><br /> I don't see a need to do a reverse lookup. Reverse lookups
aresometimes done as a verification check, in the sense that it's cheap to get a map from NAME -> IP, but sometimes
itis much harder to get the reverse map from IP -> NAME. However, it's not a reliable check as many legitimate users
havetrouble getting a reverse map from IP -> NAME. It also doesn't same anything as IP -> NAME lookups are a
completelydifferent set of name servers, and these name servers are not always optimized for speed as IP -> NAME
lookupsare less common than NAME -> IP. Finally, if one finds a map from IP -> NAME, that doesn't prove that a
mapfrom NAME -> IP exists, so using *any* results from IP -> NAME is questionable.<br /><br /> I think reverse
lookupsare unnecessary and undesirable.<br /><br /><blockquote
cite="mid:ded01eb21002110513n296d60b7me5255820a69a4bff@mail.gmail.com"type="cite">ISSUE #3: Multiple hostnames?<br
/><br/> Currently, a pg_hba entry lists an IP / netmask combination. I would suggest allowing lists of hostnames in the
entries,so that you can at least mimic the "match multiple hosts by a single rule". Any reason not to do this?<br
/></blockquote><br/> I'm mixed. In some situations, I've wanted to put multiple IP/netmask. I would say that if
multiplenames are supported, then multiple IP/netmask should be supported. But, this does make the lines unwieldy
beyondtwo or three. This direction leans towards the capability to define "host classes", where the rules allows the
hostclass, and the host class can have a list of hostnames.<br /><br /> Two other aspects I don't see mentioned:<br
/><br/> 1) What will you do for hostnames that have multiple IP addresses? Will you accept all IP addresses as being
valid?<br/> 2) What will you do if they specify a hostname and a netmask? This seems like a convenient way of saying
"everybodyon the same subnet as NAME."<br /><br /> Cheers,<br /> mark<br /><br /><pre class="moz-signature"
cols="72">--
 
Mark Mielke <a class="moz-txt-link-rfc2396E" href="mailto:mark@mielke.cc"><mark@mielke.cc></a>
</pre>

Bart Samwel <bart@samwel.tk> writes:
> I've been working on a patch to add hostname support to pg_hba.conf.

Have you read the previous discussions about that?
        regards, tom lane

Not supported. Either an IP address / netmask combo, or a hostname, but not both. I wouldn't want to recommend hardcoding something such as netmasks (which are definitely subnet dependent) in combination with something as volatile as a host name -- move it to a different subnet, and you might allow a whole bigger subnet than you intended. If they want to specify a netmask, then they should just use hardcoded IPs as well.

Cheers,
Bart

On 02/11/2010 04:54 PM, Bart Samwel wrote:

On Thu, Feb 11, 2010 at 16:36, Mark Mielke <mark@mark.mielke.cc> wrote:

ISSUE #3: Multiple hostnames?

Currently, a pg_hba entry lists an IP / netmask combination. I would suggest allowing lists of hostnames in the entries, so that you can at least mimic the "match multiple hosts by a single rule". Any reason not to do this?

I'm mixed. In some situations, I've wanted to put multiple IP/netmask. I would say that if multiple names are supported, then multiple IP/netmask should be supported. But, this does make the lines unwieldy beyond two or three. This direction leans towards the capability to define "host classes", where the rules allows the host class, and the host class can have a list of hostnames.

 
Yes, but before you know it people will ask for being able to specify multiple host classes. :-) Quite simply put, with a single subnet you can allow multiple hosts in. Allowing only a single hostname is a step backward from that, so adding support for multiple hostnames could be useful if somebody is replacing subnets with hostname-based configuration.

This implies two aspects which may not be true:

    1) All hosts that I want to allow belong to the same subnet.
    2) If I trust one host on the subnet, then I trust all hosts on the subnet.

While the above two points are often true, they are not universally true.

 

2) What will you do if they specify a hostname and a netmask? This seems like a convenient way of saying "everybody on the same subnet as NAME."

Not supported. Either an IP address / netmask combo, or a hostname, but not both. I wouldn't want to recommend hardcoding something such as netmasks (which are definitely subnet dependent) in combination with something as volatile as a host name -- move it to a different subnet, and you might allow a whole bigger subnet than you intended. If they want to specify a netmask, then they should just use hardcoded IPs as well.

Ah yes, I recall this from a previous thread. I think I also disagreed on the other thread. :-)

I thought of a use for reverse lookup - it would allow wild card hostnames. Still, that's an advanced feature that might be for later... :-)

Cheers,
mark

-- 
Mark Mielke <mark@mielke.cc>

<div class="gmail_quote">On Thu, Feb 11, 2010 at 17:21, Tom Lane <span dir="ltr"><<a
href="mailto:tgl@sss.pgh.pa.us">tgl@sss.pgh.pa.us</a>></span>wrote:<br /><blockquote class="gmail_quote"
style="margin:0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;"><div class="im">Bart
Samwel<<a href="mailto:bart@samwel.tk">bart@samwel.tk</a>> writes:<br /> > I've been working on a patch to add
hostnamesupport to pg_hba.conf.<br /><br /></div>Have you read the previous discussions about that?<br
/></blockquote></div><br/>Yes, mostly.<br /><br />The previous discussions included all sorts of complex stuff such as
wildcards.Personally, I'd think that in the cases where you'd want wildcards, then you should use IP / netmask
configuration,because that's a way better indicator of "something that comes from the same source network entity". For
instance,wildcards are nice for "all our own servers", except that you'd normally use IP / netmaks  to indicate your
ownserver subnet.<br /><br />The way I see it, hostname based configuration should be plain and simple. You suggested
inone of the earlier discussions that it should not be much more than removing the AI_NUMERICHOST hint in the lookup.
Mycurrent solution is slightly more involved, since it performs the by-hostname lookup at check time, not at
pg_hba.confread time -- but there is not much more complexity involved. If there is a case for more complexity, then we
willhear the actual use cases after this basic support is added, I guess.<br /><br />Cheers,<br />Bart<br /> 

I think wildcards are interesting, but I have yet to see an actual use case other than "it's cool and very generalized". In my mind (tell me if I'm wrong), the most common type of PostgreSQL authentication setup is within a local network within an organization. There, you either authorize an entire subnet ("the entire server park" or "all client PCs") or you authorize specific hosts (single IP address). The wildcard case is for replacing the first case, but for that case, subnets are usually just fine. I'm trying to target the second case here.

Cheers,
Bart

On 02/11/2010 05:12 PM, Bart Samwel wrote:

On Thu, Feb 11, 2010 at 23:01, Mark Mielke <mark@mark.mielke.cc> wrote:

On 02/11/2010 04:54 PM, Bart Samwel wrote:

ISSUE #3: Multiple hostnames?

Currently, a pg_hba entry lists an IP / netmask combination. I would suggest allowing lists of hostnames in the entries, so that you can at least mimic the "match multiple hosts by a single rule". Any reason not to do this?

I'm mixed. In some situations, I've wanted to put multiple IP/netmask. I would say that if multiple names are supported, then multiple IP/netmask should be supported. But, this does make the lines unwieldy beyond two or three. This direction leans towards the capability to define "host classes", where the rules allows the host class, and the host class can have a list of hostnames.

 
Yes, but before you know it people will ask for being able to specify multiple host classes. :-) Quite simply put, with a single subnet you can allow multiple hosts in. Allowing only a single hostname is a step backward from that, so adding support for multiple hostnames could be useful if somebody is replacing subnets with hostname-based configuration.

This implies two aspects which may not be true:

    1) All hosts that I want to allow belong to the same subnet.
    2) If I trust one host on the subnet, then I trust all hosts on the subnet.

While the above two points are often true, they are not universally true.

 
I don't think we're talking about the same thing here. I wasn't suggesting doing hostname-plus-netmask. NO! I was suggesting that where a lazy sysadmin would previously configure by subnet, they might switch to more fine-grained hostname-based configuration ONLY IF it doesn't require duplicating every line in pg_hba.conf for every host in the subnet.

Ah yes. You are focusing on allowing a netmask to expand to hostnames. I'm focusing on how netmasks were never that great on their own.

You want to allow multiple hosts - I want you to allow multiple netmasks. I think the requirement is the same. I also think that "same line" has always been an annoying restriction. I have many duplicated lines today just for:

host DATABASE USER 127.0.0.1/32 md5
host DATABASE USER ::1/128 md5

Isn't that a big silly? If you think it's acceptable to allow multiple hostname, I'm pointing out that your requirement is not limited to hostnames only. Why not?

host DATABASE USER 127.0.0.1/32,::1/128 md5

Same requirements, same syntax (assuming you were suggesting ','), same documentation. Why not?

But once there, it seems clear that packing hostnames or netmasks onto one line is just ugly and hard to manage. I'd like to see this extended to any of the many ways to allow hostnames to be specified one per line. For example:

set tool_servers {
    127.0.0.1/32
    ::1/128
    1.2.3.4/32
    1.2.3.5/32
}

host DATABASE USER $tool_servers md5

The above features easy parsing capability.

Of course, then I'll ask for the ability to simplify specifying multiple databases:

set databases {
    db1
    db2
}

set users {
    user1
    user2
}

host $databases $users $tool_servers md5

Sorry... :-)

2) What will you do if they specify a hostname and a netmask? This seems like a convenient way of saying "everybody on the same subnet as NAME."

Not supported. Either an IP address / netmask combo, or a hostname, but not both. I wouldn't want to recommend hardcoding something such as netmasks (which are definitely subnet dependent) in combination with something as volatile as a host name -- move it to a different subnet, and you might allow a whole bigger subnet than you intended. If they want to specify a netmask, then they should just use hardcoded IPs as well.

Ah yes, I recall this from a previous thread. I think I also disagreed on the other thread. :-)

I thought of a use for reverse lookup - it would allow wild card hostnames. Still, that's an advanced feature that might be for later... :-)

I think wildcards are interesting, but I have yet to see an actual use case other than "it's cool and very generalized". In my mind (tell me if I'm wrong), the most common type of PostgreSQL authentication setup is within a local network within an organization. There, you either authorize an entire subnet ("the entire server park" or "all client PCs") or you authorize specific hosts (single IP address). The wildcard case is for replacing the first case, but for that case, subnets are usually just fine. I'm trying to target the second case here.

The user case would be an organization with nodes all over the IP space, that wants to manage configuration from a single place. DNS would be that single place of choice. If moves trust from "trust the netmasks to be kept up-to-date" to "trust that DNS will be kept up-to-date". Since DNS has important reasons to be up-to-date, it's a pretty safe bet that DNS is equal or more up-to-date than pg_hba.conf hard coded netmasks. It makes sense, but it can be a later use case. It doesn't have to be in version 1.

Cheers,
mark

-- 
Mark Mielke <mark@mielke.cc>

Mark Mielke escreveu:
> Of course, then I'll ask for the ability to simplify specifying multiple
> databases:
> 
We already support multiple users and/or databases for a single pg_hba.conf
line ...


--  Euler Taveira de Oliveira http://www.timbira.com/

On 02/11/2010 09:38 PM, Euler Taveira de Oliveira wrote:
> Mark Mielke escreveu:
>    
>> Of course, then I'll ask for the ability to simplify specifying multiple
>> databases:
>>
>>      
> We already support multiple users and/or databases for a single pg_hba.conf
> line ...
>    

Is there a reason you trimmed out my points about specifying "classes" 
as a list of data (host, db, user) and using it in host lines? :-)

"Simplifying specifying multiple databases" in the sense of if a 
particular line has to list 10 databases, 10 users, and 10 hostnames, it 
would be most convenient to specify 3 "classes" with 10 records each, 
followed by one host line, instead of have 10 lines with 20 entries on 
it or 1000 lines... :-)

Cheers,
mark

-- 
Mark Mielke<mark@mielke.cc>

DNS is preferred to subnets in that regard, definitely. But again, that points to the per-hostname route, and it's not a use case for the wildcard route (unless people explicitly choose to organize their DNS hierarchy so that they can use it for PostgreSQL authorization -- doubtful.)

Cheers,
Bart

On tor, 2010-02-11 at 14:13 +0100, Bart Samwel wrote:
> I've been working on a patch to add hostname support to pg_hba.conf.
> It's not ready for public display yet, but I would just like to run a
> couple of issues / discussion points past everybody.

It might be good to review Apache's hostname-based access control:
http://httpd.apache.org/docs/2.2/mod/mod_authz_host.html#allow

On tor, 2010-02-11 at 14:13 +0100, Bart Samwel wrote:
> I've been working on a patch to add hostname support to pg_hba.conf.
> It's not ready for public display yet, but I would just like to run a
> couple of issues / discussion points past everybody.

What's the status of this?