Thread: How to debug incomplete message on postgres 12.3 docker
Hello,
I hope i reached the right mailing list, i have psql 12.3 container that is the db of apache guacamole container is connected to,
the apache guacamole stopped working and i started to get this message on the postgres:
LOG: incomplete message from client
How can I debug it?
i want to see when and who did the translation and what is the incomplete query ? because i did approached guacamole mailing list and they said my DB is closing connection in the middle of the transaction
and its psql issue and i am not able to debug it because i only have this incomlit msg.
Is there a way to catch this transaction?
full logs :
postgres_gqm_1 | 2022-01-13T19:32:24.187916615Z 2022-01-13 19:32:24.187 UTC [4394] LOG: incomplete message from client
guacamole_1 | 2022-01-13T19:32:24.214231454Z 19:32:24.214 [http-nio-8080-exec-6] ERROR o.a.g.rest.RESTExceptionMapper - Unexpected internal error:
guacamole_1 | 2022-01-13T19:32:24.214398980Z ### Error rolling back transaction. Cause: org.postgresql.util.PSQLException: This connection has been closed.
guacamole_1 | 2022-01-13T19:32:24.214445260Z ### Cause: org.postgresql.util.PSQLException: This connection has been closed.
docker file :
postgres_gqm:
image: postgres:12.3 environment: PGDATA: /var/lib/postgresql/data/guacamole POSTGRES_DB: db POSTGRES_PASSWORD: XXX POSTGRES_USER: user restart: always volumes: - ./init:/docker-entrypoint-initdb.d:ro - ./data:/var/lib/postgresql/data:rw logging: driver: "json-file" options: max-size: "750m"
Dave Cramer
www.postgres.rocks
On Sun, 16 Jan 2022 at 10:13, R Batchen <batchenr@gmail.com> wrote:
Hello,I hope i reached the right mailing list, i have psql 12.3 container that is the db of apache guacamole container is connected to,the apache guacamole stopped working and i started to get this message on the postgres:LOG: incomplete message from clientHow can I debug it?i want to see when and who did the translation and what is the incomplete query ? because i did approached guacamole mailing list and they said my DB is closing connection in the middle of the transactionand its psql issue and i am not able to debug it because i only have this incomlit msg.Is there a way to catch this transaction?full logs :postgres_gqm_1 | 2022-01-13T19:32:24.187916615Z 2022-01-13 19:32:24.187 UTC [4394] LOG: incomplete message from client guacamole_1 | 2022-01-13T19:32:24.214231454Z 19:32:24.214 [http-nio-8080-exec-6] ERROR o.a.g.rest.RESTExceptionMapper - Unexpected internal error: guacamole_1 | 2022-01-13T19:32:24.214398980Z ### Error rolling back transaction. Cause: org.postgresql.util.PSQLException: This connection has been closed. guacamole_1 | 2022-01-13T19:32:24.214445260Z ### Cause: org.postgresql.util.PSQLException: This connection has been closed.docker file :postgres_gqm:image: postgres:12.3 environment: PGDATA: /var/lib/postgresql/data/guacamole POSTGRES_DB: db POSTGRES_PASSWORD: XXX POSTGRES_USER: user restart: always volumes: - ./init:/docker-entrypoint-initdb.d:ro - ./data:/var/lib/postgresql/data:rw logging: driver: "json-file" options: max-size: "750m"
My guess is that the container dropped the connection somehow. Either network error or application error. I would look at the logs on the container.
There is no way to debug this on the postgres side (that I am aware of)
Dave
R Batchen <batchenr@gmail.com> writes:
> I hope i reached the right mailing list, i have psql 12.3 container that is
> the db of apache guacamole container is connected to,
> the apache guacamole stopped working and i started to get this message on
> the postgres:
> LOG: incomplete message from client
> How can I debug it?
This message indicates that the server collected a message header
(length word) from the communication channel, but then detected EOF
while trying to read that number of bytes. The two most plausible
explanations for that are (1) some external force closed the network
connection or (2) the client sent a invalid (far too large) length
word. But to believe (2) you'd also have to believe that the client
gave up and closed the connection before sending the number of bytes
the server thought it had promised. That could happen if the client
thought the command had timed out, but I'd expect it to log something
about a timeout if it had.
In short, I think something at the kernel or container level is
closing the connection under you. You need to be looking into those
system-level logs, as neither the postgres nor client logs would
have any evidence of the primary cause.
regards, tom lane
On 1/16/22 07:13, R Batchen wrote: > > Hello, > > I hope i reached the right mailing list, i have psql 12.3 container that > is the db of apache guacamole container is connected to, > the apache guacamole stopped working and i started to get this message > on the postgres: > > |LOG: incomplete message from client | > > How can I debug it? > > i want to see when and who did the translation and what is the incomplete query ? because i did approached guacamole mailinglist and they said my DB is closing connection in the middle of the transaction > > and its psql issue and i am not able to debug it because i only have this incomlit msg. The message is pretty clear, the error is coming from the client which is not Postgres. > > *Is there a way to catch this transaction?* > > full logs : > > |postgres_gqm_1 | 2022-01-13T19:32:24.187916615Z 2022-01-13 19:32:24.187 > UTC [4394] LOG: incomplete message from client guacamole_1 | > 2022-01-13T19:32:24.214231454Z 19:32:24.214 [http-nio-8080-exec-6] ERROR > o.a.g.rest.RESTExceptionMapper - Unexpected internal error: guacamole_1 The above would seem to be the place to look. > | 2022-01-13T19:32:24.214398980Z ### Error rolling back transaction. > Cause: org.postgresql.util.PSQLException: This connection has been > closed. guacamole_1 | 2022-01-13T19:32:24.214445260Z ### Cause: > org.postgresql.util.PSQLException: This connection has been closed. | > > > docker file : > > postgres_gqm: > > |image: postgres:12.3 environment: PGDATA: > /var/lib/postgresql/data/guacamole POSTGRES_DB: db POSTGRES_PASSWORD: > XXX POSTGRES_USER: user restart: always volumes: - > ./init:/docker-entrypoint-initdb.d:ro - > ./data:/var/lib/postgresql/data:rw logging: driver: "json-file" options: > max-size: "750m" | > > > > -- Adrian Klaver adrian.klaver@aklaver.com
Thanks all for the fast replay, the thing is i cannot get the transection query to know what really happened, ill try backing up db and rm containers and re build them as i dont have any other idea also server logs looks fine also cpu..
thanks!
בתאריך יום א׳, 16 בינו׳ 2022 ב-19:05 מאת Adrian Klaver <adrian.klaver@aklaver.com>:
On 1/16/22 07:13, R Batchen wrote:
>
> Hello,
>
> I hope i reached the right mailing list, i have psql 12.3 container that
> is the db of apache guacamole container is connected to,
> the apache guacamole stopped working and i started to get this message
> on the postgres:
>
> |LOG: incomplete message from client |
>
> How can I debug it?
>
> i want to see when and who did the translation and what is the incomplete query ? because i did approached guacamole mailing list and they said my DB is closing connection in the middle of the transaction
>
> and its psql issue and i am not able to debug it because i only have this incomlit msg.
The message is pretty clear, the error is coming from the client which
is not Postgres.
>
> *Is there a way to catch this transaction?*
>
> full logs :
>
> |postgres_gqm_1 | 2022-01-13T19:32:24.187916615Z 2022-01-13 19:32:24.187
> UTC [4394] LOG: incomplete message from client guacamole_1 |
> 2022-01-13T19:32:24.214231454Z 19:32:24.214 [http-nio-8080-exec-6] ERROR
> o.a.g.rest.RESTExceptionMapper - Unexpected internal error: guacamole_1
The above would seem to be the place to look.
> | 2022-01-13T19:32:24.214398980Z ### Error rolling back transaction.
> Cause: org.postgresql.util.PSQLException: This connection has been
> closed. guacamole_1 | 2022-01-13T19:32:24.214445260Z ### Cause:
> org.postgresql.util.PSQLException: This connection has been closed. |
>
>
> docker file :
>
> postgres_gqm:
>
> |image: postgres:12.3 environment: PGDATA:
> /var/lib/postgresql/data/guacamole POSTGRES_DB: db POSTGRES_PASSWORD:
> XXX POSTGRES_USER: user restart: always volumes: -
> ./init:/docker-entrypoint-initdb.d:ro -
> ./data:/var/lib/postgresql/data:rw logging: driver: "json-file" options:
> max-size: "750m" |
>
>
>
>
--
Adrian Klaver
adrian.klaver@aklaver.com