Thread: Socket command type I unknown

Good evening,

I hope that I am posting this to the right place. If not, please
direct me to the appropriate mailing list and I will send to that one
instead.

Background Info:
    Debian Linux (Sarge)
    Server A -- Apache 2.0.54 + mod_perl + DBD::Pg
    Server B -- PostgreSQL 7.3.4, Compiled, not debian package
    (yes, I know we should upgrade)

Although I haven't been able to pay much attention to it until
recently, we occasionally get the following message in our apache log
file. I'm looking for information on what it means:

   DBD::Pg::db selectrow_array failed: FATAL:  Socket command type I
unknown
   server closed the connection unexpectedly
   This probably means the server terminated abnormally
   before or while processing the request.
   DBD::Pg::db selectrow_array failed: no connection to the server

After this, we get a series of errors like this:

   DBD::Pg::db selectrow_array failed: no connection to the server
   DBD::Pg::db selectrow_array failed: no connection to the server
   DBD::Pg::db selectrow_array failed: no connection to the server

And I -know- that's causing trouble on my web server. :) A restart of
the web server 'corrects' the problem by reestablishing the
connections to the database. I suspect this might be caused by a
mismatch between the client libraries on Server A (7.4.X) whereas
Server B is a 7.3.X install. (I'm working on correcting this ASAP,
which will also get us on 7.4.)

Basically my question is this: What does this error indicate? I can't
seem to find much about it on the net and to be honest, I have become
rather illiterate in C over the past several years, so reading the
source is not really a viable option.

Any info would be appreciated. Thank you for your time.

--Joel

Joel Richard <postgresql@joelrichard.com> writes:
> ... we occasionally get the following message in our apache log
> file. I'm looking for information on what it means:

>    DBD::Pg::db selectrow_array failed: FATAL:  Socket command type I
> unknown

This looks to me like a protocol-level incompatibility: probably the
client code is sending data in a slightly different format than the
server is expecting, or one side or the other is off-by-one about
message lengths, or something like that.  One way or another the
server is receiving an 'I' when it wasn't expecting that.

I'm not aware of any such bugs on the server side in 7.3.4.  What I
suspect is a problem on the DBD::Pg side, where you did not specify
what version you are using ... but if it's recent, it probably thinks
that talking to 7.3.4 is a legacy problem ...

            regards, tom lane

Sorry to spam you guys with my noise, but for the sake of posterity,
I thought I'd reply with the solution to this problem.

I learned a lot about the PGSQL protocol during this time. So, I
ended up getting a Ethereal installed on a machine so I could follow
the protocol and see what exactly was being sent to it. As it turns
out, a separate piece of software on this server, which uses a single
database handle to do its work was somehow sending an error message
from another perl module as a message to the database.

It's a guess that somehow it got its file handles mixed up or
something along those lines. So the "73" (PgSQL 7.4) or the
"I" (PgSQL 7.3) was truly an I since the message going to the server
was something like:

    "Invalid blah blah blah"

instead of:

    "Q....SELECT * FROM TABLE"

Mystery solved. I've disabled the offensive code until I can more
closely investigate in a controlled environment. Give that we didn't
really need it, turning it off is an adequate solution for a
production server.

--Joel

> On Feb 4, 2006, at 1:16 AM, Tom Lane wrote:
>
>> Joel Richard <postgresql@joelrichard.com> writes:
>>> ... we occasionally get the following message in our apache log
>>> file. I'm looking for information on what it means:
>>
>>>    DBD::Pg::db selectrow_array failed: FATAL:  Socket command type I
>>> unknown
>>
>> This looks to me like a protocol-level incompatibility: probably the
>> client code is sending data in a slightly different format than the
>> server is expecting, or one side or the other is off-by-one about
>> message lengths, or something like that.  One way or another the
>> server is receiving an 'I' when it wasn't expecting that.
>>
>> I'm not aware of any such bugs on the server side in 7.3.4.  What I
>> suspect is a problem on the DBD::Pg side, where you did not specify
>> what version you are using ... but if it's recent, it probably thinks
>> that talking to 7.3.4 is a legacy problem ...
>>
>>             regards, tom lane
>

Joel Richard
The Richard Group
joel@richard-group.com
703-584-5802

Tom (and others who may benefit from this),

I'm sorry for taking so long to reply to this. PostgreSQL
administration is not my primary task. :) After a client noticed this
error twice this morning, I spent several hours investigating our
machine and researching on the web. This email led me in this direction

   http://archives.postgresql.org/pgsql-interfaces/2004-05/msg00011.php

and made me look further into what exactly was going on.

It turns out that we had two versions of libpq installed on the
server. One was from Debian's package manager which was for version
7.4.7 and the other was my compiled version for 7.3.X. This weekend I
upgraded us from 7.3 to 7.4 on both the server and the client. I've
removed any instances of the 7.3 libraries.

So, now we have a different error happening about as often.

   DBD::Pg::db selectrow_array failed: FATAL:  invalid frontend
message type 73

I read that 73 is an ASCII value. Surprise 73 is a capital letter I.
Same error, different method of reporting. DBD::Pg is at version 1.41
and DBI is 1.46. I'm going to try to bring DBD::Pg up to 1.48 just to
get on the latest version of everything I can find. At this point, I
want to say that it's my code that's causing the problem. It's as if
there's some perl code/query that's creating the error, but a
subsequent query to the server is what's manifesting the problem and
causing the entry in the error_log. Am I on the right track here?

I'm pretty sure the error can be traced back to my code. Therefore,
I've started a global DBI->trace() on the postgres client server. I
know that's going to be an inordinate amount of data (busy website),
but sometimes the brute force method is what will work.

I can't have my clients losing faith in my ability to solve their
problems, so I get to track this down. :)

Thanks,
--Joel


On Feb 4, 2006, at 1:16 AM, Tom Lane wrote:

> Joel Richard <postgresql@joelrichard.com> writes:
>> ... we occasionally get the following message in our apache log
>> file. I'm looking for information on what it means:
>
>>    DBD::Pg::db selectrow_array failed: FATAL:  Socket command type I
>> unknown
>
> This looks to me like a protocol-level incompatibility: probably the
> client code is sending data in a slightly different format than the
> server is expecting, or one side or the other is off-by-one about
> message lengths, or something like that.  One way or another the
> server is receiving an 'I' when it wasn't expecting that.
>
> I'm not aware of any such bugs on the server side in 7.3.4.  What I
> suspect is a problem on the DBD::Pg side, where you did not specify
> what version you are using ... but if it's recent, it probably thinks
> that talking to 7.3.4 is a legacy problem ...
>
>             regards, tom lane

Joel Richard
The Richard Group
joel@richard-group.com
703-584-5802