Thread: Insecure DNS servers on PG infrastructure
I just noted that cvs.postgresql.org and svr1.postgresql.org are not running the latest bind release, which means that they are vulnerable to the DNS cache poisoning attack recently discovered by Dan Kaminsky. Vixie and co think this is a pretty big deal, so folks might want to update sooner rather than later.http://www.kb.cert.org/vuls/id/800113 BTW, there is an excellent end-to-end test available for whether the security fix (port randomization) is actually working for you: dig @server-to-test porttest.dns-oarc.net in txt This takes a few seconds (they've arranged it to force multiple queries from the tested server) and gives you back a readout of how many ports those queries arrived from and the spread in the port addresses. A good result looks about like this: ;; ANSWER SECTION: porttest.dns-oarc.net. 60 IN CNAME z.y.x.w.v.u.t.s.r.q.p.o.n.m.l.k.j.i.h.g.f.e.d.c.b.a.pt.dns-oarc.net. z.y.x.w.v.u.t.s.r.q.p.o.n.m.l.k.j.i.h.g.f.e.d.c.b.a.pt.dns-oarc.net. 60 IN TXT "66.207.139.134 is GOOD: 26 queries in 2.3seconds from 26 ports with std dev 17102.06" If it says FAIR or POOR then you have an unpatched server or there is something interfering with the port randomization. If the server is behind a NAT firewall then the latter is entirely likely. regards, tom lane
On Fri, 2008-07-25 at 11:02 -0400, Tom Lane wrote: > I just noted that cvs.postgresql.org and svr1.postgresql.org are not > running the latest bind release, which means that they are vulnerable to > the DNS cache poisoning attack recently discovered by Dan Kaminsky. > Vixie and co think this is a pretty big deal, so folks might want to > update sooner rather than later. > http://www.kb.cert.org/vuls/id/800113 Dave and Magnus are on vacation. I believe the only other people that would have access to those boxes are Stefan and Marc. I have pinged Stefan. Joshua D. Drake -- The PostgreSQL Company since 1997: http://www.commandprompt.com/ PostgreSQL Community Conference: http://www.postgresqlconference.org/ United States PostgreSQL Association: http://www.postgresql.us/ Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate
On Fri, Jul 25, 2008 at 11:02:03AM -0400, Tom Lane wrote: > I just noted that cvs.postgresql.org and svr1.postgresql.org are not > running the latest bind release, which means that they are vulnerable to > the DNS cache poisoning attack recently discovered by Dan Kaminsky. > Vixie and co think this is a pretty big deal, so folks might want to > update sooner rather than later. This is an extremely big deal. The numbers I've seen suggest windows somewhere around 10 minutes. If the systems above are doing recursion, then they need to be patched right away. (If they're running both authority and recursive services in the same BIND instance, I suggest that the practice be abandoned immediately.) > BTW, there is an excellent end-to-end test available for whether the > security fix (port randomization) is actually working for you: > > dig @server-to-test porttest.dns-oarc.net in txt > > This takes a few seconds (they've arranged it to force multiple queries > from the tested server) and gives you back a readout of how many ports > those queries arrived from and the spread in the port addresses. Note the TTL, BTW, so that running it twice in a row gives exactly the same answer. You need to wait for the cache to expire to repeat your tests. > If it says FAIR or POOR then you have an unpatched server or there > is something interfering with the port randomization. If the server > is behind a NAT firewall then the latter is entirely likely. There's no reason that a NAT should do that, if the device is competently built: if you randomise source ports on the inside, the NAT device could just use the same port on the outside. Much worse is the bad habit many got into recently of nailing the source port to 53, and then only opening that on the firewall on the way out. Some packages have had this behaviour turned on by default. Setting the source port this way will foil the patch that is included in BIND. You need to make your recursive queries from a random port, and the port needs to be unpredictable by someone watching your resolver. If you want to try another recursive-only resolver (one that is much more recent and so was never vulnerable to this attack), have a look at unbound (find it at unbound.net). It can do DNSSEC validation for you too. BTW, the strategies that BIND and others are using to foil the Kaminsky attack merely reduce the liklihood, and don't prevent it completely. There's not a lot of source ports over which to randomize. The real solution to all this is DNSSEC, but deploying it isn't trivial, and the root isn't signed yet. (.ORG will be soon, though!) A -- Andrew Sullivan ajs@commandprompt.com +1 503 667 4564 x104 http://www.commandprompt.com/
Andrew Sullivan <ajs@commandprompt.com> writes:
> On Fri, Jul 25, 2008 at 11:02:03AM -0400, Tom Lane wrote:
>> If it says FAIR or POOR then you have an unpatched server or there
>> is something interfering with the port randomization. If the server
>> is behind a NAT firewall then the latter is entirely likely.
> There's no reason that a NAT should do that, if the device is
> competently built: if you randomise source ports on the inside, the
> NAT device could just use the same port on the outside.
I'm not convinced that that's true. If the router is trying to forward
UDP messages arriving from several "inside" IP addresses using only one
"outside" address, it has to deal with the possibility of collisions,
ie two "inside" addresses using the same port number at about the same
time. So it doesn't surprise me that it rewrites the port numbers.
If it assigned randomly-generated substitute numbers there'd be no
problem, but with no prior knowledge that would be a good idea you can
hardly blame the router authors for not indulging in extra complexity.
What I do know is that my own firewall hardware (a Netopia T1 router
that's two or three years old) *was* rewriting UDP port numbers on
requests from a machine that was sharing a NAT address with others.
After remapping to give that machine its own "outside" IP address,
it stopped doing so. BTW the porttest.dns-oarc.net service was
invaluable in testing this; I'd probably have thought that just
installing the new BIND made me safe, if I hadn't had a way to test it.
regards, tom lane
On Fri, Jul 25, 2008 at 04:44:32PM -0400, Tom Lane wrote:
> I'm not convinced that that's true. If the router is trying to forward
> UDP messages arriving from several "inside" IP addresses using only one
> "outside" address, it has to deal with the possibility of collisions,
> ie two "inside" addresses using the same port number at about the same
> time.
This is true. They can't arrive at exactly the same time, though,
which means that different strategies can be used. It's certainly
true, however, that one of the strategies may well be to rewrite port
numbers.
In some sense, rewriting to the same port number makes things quite a
bit worse for the router, because rather than just remembering "oh,
port O1 was port I1 and port O2 was port I2", the router has to
remember which {staticport,Iport} pair belongs with which inside
address. So more state is needed. (Now everyone can be amazed at
just how fast a hand can be made to wave. But this is the gist of the
argument.)
> What I do know is that my own firewall hardware (a Netopia T1 router
> that's two or three years old) *was* rewriting UDP port numbers on
> requests from a machine that was sharing a NAT address with others.
It is a problem, for sure, and the OARC test is a big help. Yay
OARC (full disclosure: my former employer isa major OARC sponsor).
A
--
Andrew Sullivan
ajs@commandprompt.com
+1 503 667 4564 x104
http://www.commandprompt.com/
Tom Lane wrote: > Andrew Sullivan <ajs@commandprompt.com> writes: >> On Fri, Jul 25, 2008 at 11:02:03AM -0400, Tom Lane wrote: >>> If it says FAIR or POOR then you have an unpatched server or there >>> is something interfering with the port randomization. If the server >>> is behind a NAT firewall then the latter is entirely likely. > >> There's no reason that a NAT should do that, if the device is >> competently built: if you randomise source ports on the inside, the >> NAT device could just use the same port on the outside. Tom can you check if this has been resolved? If not I am going to start paging people. Joshua D. Drake
Andrew Sullivan wrote: > On Fri, Jul 25, 2008 at 11:02:03AM -0400, Tom Lane wrote: >> I just noted that cvs.postgresql.org and svr1.postgresql.org are not >> running the latest bind release, which means that they are vulnerable to >> the DNS cache poisoning attack recently discovered by Dan Kaminsky. >> Vixie and co think this is a pretty big deal, so folks might want to >> update sooner rather than later. > > This is an extremely big deal. The numbers I've seen suggest windows > somewhere around 10 minutes. If the systems above are doing > recursion, then they need to be patched right away. (If they're > running both authority and recursive services in the same BIND > instance, I suggest that the practice be abandoned immediately.) cvs.postgresql.org is not running bind at all - what it is using are two (purely) recursive resolvers upstream. One of them is only going to get upgraded tomorrow(some changes need to be rolled out in a staged fashion) the other one was done a while ago - I have simply removed that one from the resolv.conf for the time being. Stefan
"Joshua D. Drake" <jd@commandprompt.com> writes:
> Tom can you check if this has been resolved? If not I am going to start
> paging people.
The machines I have access to (cvs and svr1) are still unpatched
according to the "dig porttest" test. Personally, though, I'd be
more interested in getting someone to work on fixing the mailing
lists. I've received nothing since about midnight EDT, and I know
that there has been traffic.
regards, tom lane
Tom Lane wrote: > "Joshua D. Drake" <jd@commandprompt.com> writes: > > Tom can you check if this has been resolved? If not I am going to start > > paging people. > > The machines I have access to (cvs and svr1) are still unpatched > according to the "dig porttest" test. Personally, though, I'd be > more interested in getting someone to work on fixing the mailing > lists. I've received nothing since about midnight EDT, and I know > that there has been traffic. Maybe the mail servers have been DDoS'ed again? -- Alvaro Herrera http://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc.
Alvaro Herrera wrote: > Tom Lane wrote: >> "Joshua D. Drake" <jd@commandprompt.com> writes: >>> Tom can you check if this has been resolved? If not I am going to start >>> paging people. >> The machines I have access to (cvs and svr1) are still unpatched >> according to the "dig porttest" test. Personally, though, I'd be >> more interested in getting someone to work on fixing the mailing >> lists. I've received nothing since about midnight EDT, and I know >> that there has been traffic. > > Maybe the mail servers have been DDoS'ed again? I did have a bunch of alerts throw recovery around 2:00am but then where is all the traffic? >
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 - --On Friday, July 25, 2008 11:02:03 -0400 Tom Lane <tgl@sss.pgh.pa.us> wrote: > I just noted that cvs.postgresql.org and svr1.postgresql.org are not Was out camping, just saw this ... svr1.postgresql.org's name server is shutdown/disabled now, since its not needed ... and all of the primary name servers have had recursion disabled since Andrew pointed out to me awhile back how to do so ... - -- Marc G. Fournier Hub.Org Hosting Solutions S.A. (http://www.hub.org) Email . scrappy@hub.org MSN . scrappy@hub.org Yahoo . yscrappy Skype: hub.org ICQ . 7615664 -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (FreeBSD) iEYEARECAAYFAkiNSnwACgkQ4QvfyHIvDvOs2QCfTJPaWg2si8wbdmGSVy4p8Xdx GIsAoIl4TDOy4axlpAKDfmgwgOoxPb4s =Ro1U -----END PGP SIGNATURE-----