Re: psycopg2 conn.poll() hangs when used with sshtunnel - Mailing list psycopg

From Akshay Joshi
Subject Re: psycopg2 conn.poll() hangs when used with sshtunnel
Date
Msg-id CANxoLDemxzjQitt_4K_3-Ear08898ug7bryHz0eYyWas5j_xxw@mail.gmail.com
Whole thread Raw
In response to Re: psycopg2 conn.poll() hangs when used with sshtunnel  (Daniele Varrazzo <daniele.varrazzo@gmail.com>)
Responses Re: psycopg2 conn.poll() hangs when used with sshtunnel  (Akshay Joshi <akshay.joshi@enterprisedb.com>)
List psycopg


On Tue, 30 Oct 2018, 21:06 Daniele Varrazzo <daniele.varrazzo@gmail.com wrote:
On Tue, Oct 30, 2018 at 1:55 PM Akshay Joshi <akshay.joshi@enterprisedb.com> wrote:
 
    What changes should I made to fix this? Or it should be fixed in psycopg2 itself.

Actually, I have another observation. I was surprised to see that the whole test suite passed running the connection through an ssh tunnel: there are definitely failing queries in the test suite. So I made other tests and it seems the bad condition only happens using the `sshtunnel` module: If I open a tunnel manually with something like:

    ssh -L 36421:localhost:5432 -N localhost

and point your script to port 36421 everything works ok.


Conversely, I don't seem able to run the test suite through the tunnel open by the sshtunnel module. It doesn't hang, but the tunnel breaks in a test with the following reported on the sshtunnel side:

    2018-10-30 15:24:25,550| ERROR   | Socket exception: Bad file descriptor (9)
    2018-10-30 15:24:25,551| ERROR   | Could not establish connection from ('127.0.0.1', 33743) to remote side of the tunnel

and the following tests fail to run as the connection is broken.


Con-conversely, the patch I had in mind to fix #801, which very brutally is:

```
@@ -1136,6 +1136,13 @@ pq_get_last_result(connectionObject *conn)
                 || status == PGRES_COPY_IN) {
             break;
         }
+        if (PQisBusy(conn->pgconn)) {
+            /* This happens connecting through ssl tunnel
+             * TODO: just kill this function. The loop should happen within
+             * the async/green machinery. */
+            Dprintf("pq_get_last_result: we are busy");
+            break;
+        }
     }
 
     return result;
```

doesn't work, and failing queries leave the connection in an inconsistent state.


So, wrapping up, I think there is something you can do on your side, checking why the sshtunnel module behaves differently from a normal ssh tunnel, and if you use the latter (or configure sshtunnel to behave so) you shouldn't hit the problem. On our side I don't think we can fix #801 with a quick band aid, and we should rather do #802, but as things stands now I don't trust sshtunnel to do the right thing, and have less of an urgency to do so.

Please let us know if you understand what is the difference between the module and the tunnel via `ssh -L`, thank you!

    I have send the sample application and created an issue to sshtunnel git hub. Not sure where is the problem, as poll() function hangs so i thought its issue in psycopg2.

-- Daniele

psycopg by date:

Previous
From: Daniele Varrazzo
Date:
Subject: Re: psycopg2 conn.poll() hangs when used with sshtunnel
Next
From: Akshay Joshi
Date:
Subject: Re: psycopg2 conn.poll() hangs when used with sshtunnel