Thread: FATAL: invalid frontend message type 8
I've started seeing the following in my logs: FATAL: invalid frontend message type 8 I searched back over a month and there are 5 instances of this error of which 4 are in the last 24 hours. I could not find this error defined. Any ideas of what it means, it's severity, how to track the cause and cure? Cheers, Steve
Steve Crawford <scrawford@pinpointresearch.com> writes:
> I've started seeing the following in my logs:
> FATAL: invalid frontend message type 8
> I searched back over a month and there are 5 instances of this error
> of which 4 are in the last 24 hours.
> I could not find this error defined. Any ideas of what it means, it's
> severity, how to track the cause and cure?
It looks to me like you've got an erroneous client that is sending bad
data. The error is from tcop/postgres.c, when it doesn't recognize the
first byte of what should be a message:
default:
/*
* Otherwise we got garbage from the frontend. We treat this
* as fatal because we have probably lost message boundary
* sync, and there's no good way to recover.
*/
ereport(FATAL,
(errcode(ERRCODE_PROTOCOL_VIOLATION),
errmsg("invalid frontend message type %d", qtype)));
Can you determine exactly which client-side code is provoking the error?
regards, tom lane
On Wednesday 18 August 2004 10:34 am, Tom Lane wrote:
> Steve Crawford <scrawford@pinpointresearch.com> writes:
> > I've started seeing the following in my logs:
> > FATAL: invalid frontend message type 8
> >
> > I searched back over a month and there are 5 instances of this
> > error of which 4 are in the last 24 hours.
> >
> > I could not find this error defined. Any ideas of what it means,
> > it's severity, how to track the cause and cure?
>
> It looks to me like you've got an erroneous client that is sending
> bad data. The error is from tcop/postgres.c, when it doesn't
> recognize the first byte of what should be a message:
>
> default:
> /*
> * Otherwise we got garbage from the frontend. We
> treat this * as fatal because we have probably lost message
> boundary * sync, and there's no good way to recover.
> */
> ereport(FATAL,
> (errcode(ERRCODE_PROTOCOL_VIOLATION),
> errmsg("invalid frontend message type %d",
> qtype)));
>
> Can you determine exactly which client-side code is provoking the
> error?
I started recording all the raw network data and waited for the error
in the log. From this I have determined the machine and the likely
process. It is a telephony app written in C and using the pg
libraries. The piece in question opens a connection, verifies the
existence of the appropriate table (select from pg_ tables) and
inserts a record to that table.
All of the above works but following the terminating 0x00 of the final
insert there is about 12k of junk consisting of a mix of 0x08 and
0x80 characters ending with an 'X' (a normal connection is exactly
the same but there is no junk between the final 0x00 and 'X'). I've
only trapped one full instance so I don't know if the junk is always
the same.
Fortunately the data is actually inserted and the error occurs when
the app exits so the user-experience is not affected.
The client libraries are slightly older than the server but this
doesn't seem to be an issue as it's only happened 5-6 times in
tens-of-thousands of cases.
I'll look for causes in the code. If this symptom looks familar to
anyone, let me know.
Cheers,
Steve
Steve Crawford <scrawford@pinpointresearch.com> writes:
> I started recording all the raw network data and waited for the error
> in the log. From this I have determined the machine and the likely
> process. It is a telephony app written in C and using the pg
> libraries.
What version of libpq is it using, exactly?
regards, tom lane
On Wednesday 18 August 2004 2:25 pm, Tom Lane wrote: > Steve Crawford <scrawford@pinpointresearch.com> writes: > > I started recording all the raw network data and waited for the > > error in the log. From this I have determined the machine and the > > likely process. It is a telephony app written in C and using the > > pg libraries. > > What version of libpq is it using, exactly? > > regards, tom lane Client libs are 7.3.1. Client and PG libs are compiled and installed on SCO_SV 3.2v5.0.4. IIRC (this app's developer is in Europe for a month), the PG libs are statically compiled into the app. Server version is 7.4.1. We will be migrating the front-end to Linux and will update both client and server to match. Cheers, Steve
Steve Crawford <scrawford@pinpointresearch.com> writes:
> On Wednesday 18 August 2004 2:25 pm, Tom Lane wrote:
>> What version of libpq is it using, exactly?
> Client libs are 7.3.1.
> Server version is 7.4.1.
Hmm. Okay, so old (v2) protocol ... there's sure no reason for a bunch
of garbage in that ...
You aren't by any chance running the connection SSL-encrypted are you?
(I'd suppose not, or sniffing the network traffic wouldn't have helped,
but I see one heck of a lot of post-7.3.1 SSL-related bug fixes in the
CVS logs...)
regards, tom lane
On Wednesday 18 August 2004 3:52 pm, Tom Lane wrote: > Steve Crawford <scrawford@pinpointresearch.com> writes: > > On Wednesday 18 August 2004 2:25 pm, Tom Lane wrote: > >> What version of libpq is it using, exactly? > > > > Client libs are 7.3.1. > > Server version is 7.4.1. > > Hmm. Okay, so old (v2) protocol ... there's sure no reason for a > bunch of garbage in that ... > > You aren't by any chance running the connection SSL-encrypted are > you? (I'd suppose not, or sniffing the network traffic wouldn't > have helped, but I see one heck of a lot of post-7.3.1 SSL-related > bug fixes in the CVS logs...) I _wish_ I could read/understand raw SSL but no, it's unencrypted. I didn't see anything unusual in the submitted data (very long text data, etc.) so I'm going to poke around in the code and wait till the current projects on this machine are finished so I can do a good memtest on it. Cheers, Steve