Re: Premature timeout on git.postgresql.org? - Mailing list pgsql-www

Bruce Momjian <bruce@momjian.us> writes:
> On Sun, Feb 20, 2022 at 08:31:23AM -0800, Adrian Klaver wrote:
>> For a point of reference I just cloned on a Linode instance in Fremont CA.
>> For the Postgres repo I got 2.25 MiB/s. Trying some other project repos I
>> got 25-27 MiB/s

> My transfer number fluctuated from 800kB to 2.2MB.

Yeah, I'd supposed that the el-cheapo ethernet dongle I'm using on
florican's host was the source of the crummy performance, but it is not,
or at least not all of it: there is something rotten between here and
Amsterdam.  Trying it now on my primary workstation, I can clone from
the github mirror at a more or less respectable speed:

$ time git clone https://github.com/postgres/postgres.git pgsql-github
Cloning into 'pgsql-github'...
remote: Enumerating objects: 879421, done.
remote: Counting objects: 100% (240/240), done.
remote: Compressing objects: 100% (185/185), done.
remote: Total 879421 (delta 111), reused 82 (delta 55), pack-reused 879181
Receiving objects: 100% (879421/879421), 616.24 MiB | 33.59 MiB/s, done.
Resolving deltas: 100% (721439/721439), done.

real    0m36.548s
user    0m59.798s
sys     0m2.830s

but not so much from our official repo:

$ time git clone https://git.postgresql.org/git/postgresql.git pgsql-pg
Cloning into 'pgsql-pg'...
remote: Enumerating objects: 14782, done.
remote: Counting objects: 100% (14782/14782), done.
remote: Compressing objects: 100% (9581/9581), done.
remote: Total 879421 (delta 9965), reused 6933 (delta 5125), pack-reused 864639
Receiving objects: 100% (879421/879421), 289.26 MiB | 1.40 MiB/s, done.
Resolving deltas: 100% (754313/754313), done.

real    3m48.432s
user    0m54.379s
sys     0m4.514s

The 1.4MB/s average seen here hides a very variable speed: like
Bruce, I saw a speed around 700/800 kB/s to start, and then it
gradually ramped up to something over 2MB/s.

Now in fairness, the ping time to github.com from here is circa 10ms
while the ping time to gothos.postgresql.org is circa 100ms, so
I'd not really expect equivalent performance ... but this seems
seriously bad.

Trying it again on the machine where I originally saw the problem,
cloning from github works:

$ time git clone https://github.com/postgres/postgres.git pgsql-github
Cloning into 'pgsql-github'...
remote: Enumerating objects: 879421, done.
remote: Counting objects: 100% (240/240), done.
remote: Compressing objects: 100% (129/129), done.
remote: Total 879421 (delta 111), reused 235 (delta 111), pack-reused 879181
Receiving objects: 100% (879421/879421), 616.37 MiB | 4.00 MiB/s, done.
Resolving deltas: 100% (721488/721488), done.
Updating files: 100% (6058/6058), done.
      318.52 real       332.96 user        26.12 sys

It's slow, but I expected that.  I then tried twice to clone from
git.postgresql.org.  The first try succeeded:

$ time git clone https://git.postgresql.org/git/postgresql.git pgsql-pg
Cloning into 'pgsql-pg'...
remote: Enumerating objects: 14782, done.
remote: Counting objects: 100% (14782/14782), done.
remote: Compressing objects: 100% (9581/9581), done.
remote: Total 879421 (delta 9966), reused 6933 (delta 5125), pack-reused 864639
Receiving objects: 100% (879421/879421), 289.20 MiB | 862.00 KiB/s, done.
Resolving deltas: 100% (754314/754314), done.
Updating files: 100% (6058/6058), done.
      502.96 real       307.15 user        15.66 sys

The second try got the same results as yesterday:

$ time git clone https://git.postgresql.org/git/postgresql.git pgsql-pg
Cloning into 'pgsql-pg'...
remote: Enumerating objects: 14782, done.
remote: Counting objects: 100% (14782/14782), done.
remote: Compressing objects: 100% (9581/9581), done.
[ I hit return a few extra times to capture state, last printout was this: ]
Receiving objects:  99% (876395/879421), 285.62 MiB | 1.19 MiB/s
error: RPC failed; curl 92 HTTP/2 stream 3 was not closed cleanly before end of the underlying stream
fetch-pack: unexpected disconnect while reading sideband packet
fatal: early EOF
fatal: fetch-pack: invalid index-pack output
      323.67 real         0.00 user         0.11 sys

(The instantaneous transfer speed varied from circa 1.3MB/s to
as low as a couple hundred kB/s.)

I doubt there's much we can do at the project level about the
poor transfer speed of the transatlantic link.  However, the
fact that it's failing outright looks to me to be due to a
timeout of 5 or possibly 6 minutes that's breaking the "sideband"
connection.  That probably is within our control.

            regards, tom lane