Thread: DNS lookup for git.postgresql.org
I have an automated job that pulls from our public git server, and
several times a day I get this error:
fatal: unable to look up git.postgresql.org (port 9418)
(Temporary failure in name resolution)
I don't see this error for any of my other automated scripts that pull
from URLs. Is this something other people see too? Is it a problem on
my end? My git config entry is:
[remote "pgsql-public"]
url = git://git.postgresql.org/git/postgresql.git
--
Bruce Momjian <bruce@momjian.us> https://momjian.us
EDB https://enterprisedb.com
If only the physical world exists, free will is an illusion.
Bruce Momjian <bruce@momjian.us> writes:
> I have an automated job that pulls from our public git server, and
> several times a day I get this error:
> fatal: unable to look up git.postgresql.org (port 9418)
> (Temporary failure in name resolution)
> I don't see this error for any of my other automated scripts that pull
> from URLs. Is this something other people see too? Is it a problem on
> my end? My git config entry is:
My buildfarm animals frequently complain of being unable to contact the
buildfarm server. Sometimes there's an identifiable problem at my end,
but usually not. A typical error looks like
error getting branches of interest: 500 Can't connect to buildfarm.postgresql.org:443 (Network is unreachable) at
./run_branches.plline 199.
I've not noticed instability of connections to the git servers, though.
regards, tom lane
On Tue, Jun 29, 2021 at 11:26:54PM -0400, Tom Lane wrote:
> Bruce Momjian <bruce@momjian.us> writes:
> > I have an automated job that pulls from our public git server, and
> > several times a day I get this error:
> > fatal: unable to look up git.postgresql.org (port 9418)
> > (Temporary failure in name resolution)
> > I don't see this error for any of my other automated scripts that pull
> > from URLs. Is this something other people see too? Is it a problem on
> > my end? My git config entry is:
>
> My buildfarm animals frequently complain of being unable to contact the
> buildfarm server. Sometimes there's an identifiable problem at my end,
> but usually not. A typical error looks like
>
> error getting branches of interest: 500 Can't connect to buildfarm.postgresql.org:443 (Network is unreachable) at
./run_branches.plline 199.
>
> I've not noticed instability of connections to the git servers, though.
OK, thanks. When no one said they saw similar behavior, I started
digging deeper. It seems on Debian 10, bind/named uses IPv6 by default,
and my ISP, Verison FIOS, doesn't support IPv6 yet; see:
https://www.reddit.com/r/Fios/comments/li4ri7/does_fios_support_ipv6/
I also failed this test:
http://test-ipv6.com/
Turns out I had a number of failures for other domains as well, so it
wasn't only git.postgresql.org. I disabled IPv6 by modifying
/etc/default/bind9:
https://serverfault.com/questions/833270/why-am-i-seeing-ipv6-resolve-errors-despite-having-disabled-ipv6-in-bind
Anyway, I am attaching my compressed bind/named debug logs, in case it
is helpful to anyone.
--
Bruce Momjian <bruce@momjian.us> https://momjian.us
EDB https://enterprisedb.com
If only the physical world exists, free will is an illusion.
On Wed, Jun 30, 2021 at 12:23:32PM -0400, Bruce Momjian wrote: > Anyway, I am attaching my compressed bind/named debug logs, in case it > is helpful to anyone. Sorry, attachment included now. -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com If only the physical world exists, free will is an illusion.
Attachment
Bruce Momjian <bruce@momjian.us> writes: > On Tue, Jun 29, 2021 at 11:26:54PM -0400, Tom Lane wrote: >> My buildfarm animals frequently complain of being unable to contact the >> buildfarm server. Sometimes there's an identifiable problem at my end, >> but usually not. A typical error looks like >> error getting branches of interest: 500 Can't connect to buildfarm.postgresql.org:443 (Network is unreachable) at ./run_branches.plline 199. > OK, thanks. When no one said they saw similar behavior, I started > digging deeper. It seems on Debian 10, bind/named uses IPv6 by default, > and my ISP, Verison FIOS, doesn't support IPv6 yet; see: > https://www.reddit.com/r/Fios/comments/li4ri7/does_fios_support_ipv6/ Oh, that's an interesting point. I have no IPv6 connectivity here either. I'd figured out some time ago that with recent bind versions I need to use named's "-4" option to prevent DNS lookup timeouts ... but now I see that that doesn't stop it from returning IPv6 addresses. I wonder if the failures occur when my animals try to use the buildfarm server's IPv6 address. I'll try the hack mentioned in the serverfault thread. regards, tom lane
On Wed, Jun 30, 2021 at 12:53:24PM -0400, Tom Lane wrote: > Bruce Momjian <bruce@momjian.us> writes: > > On Tue, Jun 29, 2021 at 11:26:54PM -0400, Tom Lane wrote: > >> My buildfarm animals frequently complain of being unable to contact the > >> buildfarm server. Sometimes there's an identifiable problem at my end, > >> but usually not. A typical error looks like > >> error getting branches of interest: 500 Can't connect to buildfarm.postgresql.org:443 (Network is unreachable) at ./run_branches.plline 199. > > > OK, thanks. When no one said they saw similar behavior, I started > > digging deeper. It seems on Debian 10, bind/named uses IPv6 by default, > > and my ISP, Verison FIOS, doesn't support IPv6 yet; see: > > https://www.reddit.com/r/Fios/comments/li4ri7/does_fios_support_ipv6/ > > Oh, that's an interesting point. I have no IPv6 connectivity here either. > I'd figured out some time ago that with recent bind versions I need to use > named's "-4" option to prevent DNS lookup timeouts ... but now I see that > that doesn't stop it from returning IPv6 addresses. I wonder if the > failures occur when my animals try to use the buildfarm server's IPv6 > address. I'll try the hack mentioned in the serverfault thread. Oh, I used the -4 option and my failures stopped. Glad this thread was helpful for you too. I never expected IPv6 to lead to failures, just possible delays, but I have now learned, at least with DNS, it can cause failures too. I will also add the bind options mentioned to disable dnssec and aaaa records. -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com If only the physical world exists, free will is an illusion.
Bruce Momjian <bruce@momjian.us> writes:
> On Wed, Jun 30, 2021 at 12:53:24PM -0400, Tom Lane wrote:
>> ... I'll try the hack mentioned in the serverfault thread.
> I will also add the bind options mentioned to disable
> dnssec and aaaa records.
I found better practice described at
https://kb.isc.org/docs/aa-00576
to wit you can set "filter-aaaa-on-v4" to "break-dnssec" to just
adjust what is returned to clients, rather than disabling DNSSEC
globally. Also, if you use views to configure your bind setup,
it works to make that an option in the view that serves your
local clients (the ones you don't want to see IPv6 addys).
I have that installed locally now, but it'll take awhile to
determine whether it improves matters.
regards, tom lane
On Wed, Jun 30, 2021 at 9:20 PM Bruce Momjian <bruce@momjian.us> wrote: > > On Wed, Jun 30, 2021 at 12:53:24PM -0400, Tom Lane wrote: > > Bruce Momjian <bruce@momjian.us> writes: > > > On Tue, Jun 29, 2021 at 11:26:54PM -0400, Tom Lane wrote: > > >> My buildfarm animals frequently complain of being unable to contact the > > >> buildfarm server. Sometimes there's an identifiable problem at my end, > > >> but usually not. A typical error looks like > > >> error getting branches of interest: 500 Can't connect to buildfarm.postgresql.org:443 (Network is unreachable) at./run_branches.pl line 199. > > > > > OK, thanks. When no one said they saw similar behavior, I started > > > digging deeper. It seems on Debian 10, bind/named uses IPv6 by default, > > > and my ISP, Verison FIOS, doesn't support IPv6 yet; see: > > > https://www.reddit.com/r/Fios/comments/li4ri7/does_fios_support_ipv6/ > > > > Oh, that's an interesting point. I have no IPv6 connectivity here either. > > I'd figured out some time ago that with recent bind versions I need to use > > named's "-4" option to prevent DNS lookup timeouts ... but now I see that > > that doesn't stop it from returning IPv6 addresses. I wonder if the > > failures occur when my animals try to use the buildfarm server's IPv6 > > address. I'll try the hack mentioned in the serverfault thread. > > Oh, I used the -4 option and my failures stopped. Glad this thread was > helpful for you too. I never expected IPv6 to lead to failures, just > possible delays, but I have now learned, at least with DNS, it can cause It shouldn't. I regularly work from networks with no native ipv6 and these things work perfectly fine. Do you have an actual public ipv6 address on your system, and it just doesn't work? Like maybe a tunnel you set up at some point that doesn't work? If not it seems very strange that it should even try to get out over ipv6. > failures too. I will also add the bind options mentioned to disable > dnssec and aaaa records. You should *not* disable dnssec. It's an important security feature. Filtering them in the DNS response sounds more like trying to apply a crude workaround. -- Magnus Hagander Me: https://www.hagander.net/ Work: https://www.redpill-linpro.com/
On Wed, Jun 30, 2021 at 04:20:28PM -0400, Tom Lane wrote: > Bruce Momjian <bruce@momjian.us> writes: > > On Wed, Jun 30, 2021 at 12:53:24PM -0400, Tom Lane wrote: > >> ... I'll try the hack mentioned in the serverfault thread. > > > I will also add the bind options mentioned to disable > > dnssec and aaaa records. > > I found better practice described at > https://kb.isc.org/docs/aa-00576 > to wit you can set "filter-aaaa-on-v4" to "break-dnssec" to just > adjust what is returned to clients, rather than disabling DNSSEC > globally. Also, if you use views to configure your bind setup, > it works to make that an option in the view that serves your > local clients (the ones you don't want to see IPv6 addys). Oh, I am now trying just the "filter-aaaa-on-v4 break-dnssec" option. Not sure why this is so complicated --- there must be many people without IPv6 who use bind. > I have that installed locally now, but it'll take awhile to > determine whether it improves matters. OK, I will keep an eye on my bind debug logs to see if I see failures. -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com If only the physical world exists, free will is an illusion.
On Wed, Jun 30, 2021 at 11:04:23PM +0200, Magnus Hagander wrote: > On Wed, Jun 30, 2021 at 9:20 PM Bruce Momjian <bruce@momjian.us> wrote: > > Oh, I used the -4 option and my failures stopped. Glad this thread was > > helpful for you too. I never expected IPv6 to lead to failures, just > > possible delays, but I have now learned, at least with DNS, it can cause > > It shouldn't. > > I regularly work from networks with no native ipv6 and these things > work perfectly fine. Yes, I am confused, but as you can see from the logs I posted, bind is occasionally failing. > Do you have an actual public ipv6 address on your system, and it just > doesn't work? Like maybe a tunnel you set up at some point that > doesn't work? If not it seems very strange that it should even try to > get out over ipv6. I have no IPv6 IP address and never use tunnels. I just did a grep for "ipv6" in /etc and found only default commented-out lines in sysctl.conf. Where else would I look? > > failures too. I will also add the bind options mentioned to disable > > dnssec and aaaa records. > > You should *not* disable dnssec. It's an important security feature. > Filtering them in the DNS response sounds more like trying to apply a > crude workaround. So just using "filter-aaaa-on-v4 break-dnssec" and not using "dnssec-enable no" is what you recommend? -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com If only the physical world exists, free will is an illusion.
Magnus Hagander <magnus@hagander.net> writes:
> On Wed, Jun 30, 2021 at 9:20 PM Bruce Momjian <bruce@momjian.us> wrote:
>> Oh, I used the -4 option and my failures stopped. Glad this thread was
>> helpful for you too. I never expected IPv6 to lead to failures, just
>> possible delays, but I have now learned, at least with DNS, it can cause
> It shouldn't.
> I regularly work from networks with no native ipv6 and these things
> work perfectly fine.
IME, things work just well enough that it takes awhile to realize that
they aren't entirely okay. It took me a few months after upgrading to
RHEL8 to make the connection between occasional (maybe one-in-a-thousand)
mail failures and not having bind set up with "-4". I don't know whether
the previous setup avoided that because older bind defaulted differently
or because there was some change in kernel configuration.
regards, tom lane
On Wed, Jun 30, 2021 at 06:00:28PM -0400, Tom Lane wrote: > IME, things work just well enough that it takes awhile to realize that > they aren't entirely okay. It took me a few months after upgrading to > RHEL8 to make the connection between occasional (maybe one-in-a-thousand) > mail failures and not having bind set up with "-4". I don't know whether > the previous setup avoided that because older bind defaulted differently > or because there was some change in kernel configuration. Agreed. The failures were very rare, but you know me, I had to dig into it to find the cause. ;-) I would see occasional Firefox page load failures, but once my cron job occasionally failed, I now had a way to find the root cause. -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com If only the physical world exists, free will is an illusion.