Thread: significant jump in sql statement timing for on server vs a remote connection

Hi all;


We are debugging a sql performance issue. We have a sql file with 50,000 
simple select statements in it. If I run the file locally it completes 
in less than 15sec.  If I force the local connection to be a tcp/ip 
connection via psql -h and I get approximately the same results, 15 - 16sec.


However if we move the file to another server in the same network and 
run with a psql -h then it runs for more than 10min. Are there any 
postgres specific issues / settings / connection overhead  we should 
look at? Or is this simply a network issue and fully outside the scope 
of the postgres database?


FYI:

postgresql 13

1.5TB of RAM

512GB of buffer_pool

10GB of work_mem



Thanks in advance

On Tue, Apr 19, 2022 at 03:00:09PM -0600, Sbob wrote:
> We are debugging a sql performance issue. We have a sql file with 50,000
> simple select statements in it. If I run the file locally it completes in
> less than 15sec.  If I force the local connection to be a tcp/ip connection
> via psql -h and I get approximately the same results, 15 - 16sec.
> 
> 
> However if we move the file to another server in the same network and run
> with a psql -h then it runs for more than 10min. Are there any postgres
> specific issues / settings / connection overhead  we should look at? Or is
> this simply a network issue and fully outside the scope of the postgres
> database?

What OS ?  What kind of authentication are you using ?
Is there a connection pooler involved ?  Did you try like that ?

Did you test how long it takes to run 10k empty statements locally vs remotely ?
time yes 'SELECT;' |head -9999 |psql ... >/dev/null

-- 
Justin

On Tue, Apr 19, 2022 at 5:00 PM Sbob <sbob@quadratum-braccas.com> wrote:

However if we move the file to another server in the same network and
run with a psql -h then it runs for more than 10min.

What is the ping time?  Packet loss? You can't take for granted that the network is good and fast just because they are on the same LAN.

Cheers,

Jeff

Here is the ping stats:

--- db-primary ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3000ms
rtt min/avg/max/mdev = 0.304/0.348/0.400/0.039 ms

This seems pretty good yes? Anything else I could look at?