Thread: Peculiar behaviour in libpq
I've noticed some strange behaviour when using libpq with PostgreSQL 6.4.2 on RedHat Linux 5.1. I'm using libpq asynchronously, with a select() loop, calling PQconsumeInput etc. What is happening is that occasionally, after I have received a callback via select() and processed the result, I get an extra callback. These extra callbacks have no result attached to them, PQisBusy() returns false, and, by the records I'm keeping in my own program, there is no database operation pending (ie I already have a result for every operation I've submitted). I guess it's some sort of random noise on the socket or something. Or maybe the backend just sends a nothing-in-particular packet down the line every now and then. But I'm wondering if this is a symptom of something else going wrong. It tends to happen after several other operations, ie while the database is reasonably busy, not when it's idle. Has anyone else ever seen this behaviour? Is it anything to worry about? Will it go away in 6.5? On the whole it seems reasonably benign, I just worry there may be something bad going on that will bite me eventually. Tim ----------------------------------------------- Tim Allen Proximity Pty Ltd http://www.proximity.com.au -----------------------------------------------
Tim Allen <tim@proximity.com.au> writes:
> I've noticed some strange behaviour when using libpq with
> PostgreSQL 6.4.2 on RedHat Linux 5.1. I'm using libpq asynchronously, with
> a select() loop, calling PQconsumeInput etc. What is happening is that
> occasionally, after I have received a callback via select() and processed
> the result, I get an extra callback. These extra callbacks have no result
> attached to them, PQisBusy() returns false, and, by the records I'm
> keeping in my own program, there is no database operation pending (ie I
> already have a result for every operation I've submitted).
Hmm. Could be that the terminating ReadyForQuery message ('Z') is
arriving in a separate packet from the last message of the query
response proper --- but that doesn't seem real probable. Other
possibilities are that backend NOTIFY or NOTICE messages are arriving;
but if you were using NOTIFY you'd know it, and NOTICEs ought to show
up on stderr (unless you've suppressed that).
If you really care, you could instrument the low-level guts of libpq
to see what is arriving when (the routines in fe-misc.c would be the
place to put debug logging).
> Will it go away in 6.5? On the whole it seems reasonably benign, I just
> worry there may be something bad going on that will bite me eventually.
Yeah, it seems worth understanding at least. I'm pretty sure 6.5 will
behave the same, since nothing much has changed in the BE/FE protocol.
I'd be interested to know what you find out, if you pursue it.
regards, tom lane
On Thu, 24 Jun 1999, Tom Lane wrote:
> Tim Allen <tim@proximity.com.au> writes:
> > I've noticed some strange behaviour when using libpq with
> > PostgreSQL 6.4.2 on RedHat Linux 5.1. I'm using libpq asynchronously, with
> > a select() loop, calling PQconsumeInput etc. What is happening is that
> > occasionally, after I have received a callback via select() and processed
> > the result, I get an extra callback. These extra callbacks have no result
> > attached to them, PQisBusy() returns false, and, by the records I'm
> > keeping in my own program, there is no database operation pending (ie I
> > already have a result for every operation I've submitted).
>
> Hmm. Could be that the terminating ReadyForQuery message ('Z') is
> arriving in a separate packet from the last message of the query
> response proper --- but that doesn't seem real probable. Other
> possibilities are that backend NOTIFY or NOTICE messages are arriving;
> but if you were using NOTIFY you'd know it, and NOTICEs ought to show
> up on stderr (unless you've suppressed that).
I've performed some more experiments, after instrumenting bits of libpq.
I'm still bemused...
No, it's not the 'Z' message. What appears to be happening is that a
select condition is occurring without there being any data to read from
the socket. Inside PQconsumeInput(), which is calling pqReadData(), the
call to recv() returns -1 with errno set to EAGAIN (I gather this is
implementation dependent, and could be EWOULDBLOCK on some platforms).
This happens after _every_ result, ie the result from a query gets
returned, with the data being successfully read when select() says we can,
and then one extra select condition occurs, with no data to read.
So the mystery is, why does select() think there is something to read from
the socket when recv() thinks not?
I was starting to suspect it could be merely an oddity of Linux's socket
implementation, but discovered that the same behaviour occurs on SGI/IRIX.
This is a local (Unix domain) socket, btw, so it's not a TCP/IP issue.
> If you really care, you could instrument the low-level guts of libpq
> to see what is arriving when (the routines in fe-misc.c would be the
> place to put debug logging).
>
> > Will it go away in 6.5? On the whole it seems reasonably benign, I just
> > worry there may be something bad going on that will bite me eventually.
>
> Yeah, it seems worth understanding at least. I'm pretty sure 6.5 will
> behave the same, since nothing much has changed in the BE/FE protocol.
> I'd be interested to know what you find out, if you pursue it.
It still seems benign enough, but I would also like to understand it...
> regards, tom lane
Tim
-----------------------------------------------
Tim Allen
Proximity Pty Ltd http://www.proximity.com.au
-----------------------------------------------
Tim Allen <tim@proximity.com.au> writes:
> No, it's not the 'Z' message. What appears to be happening is that a
> select condition is occurring without there being any data to read from
> the socket. Inside PQconsumeInput(), which is calling pqReadData(), the
> call to recv() returns -1 with errno set to EAGAIN (I gather this is
> implementation dependent, and could be EWOULDBLOCK on some platforms).
> This happens after _every_ result, ie the result from a query gets
> returned, with the data being successfully read when select() says we can,
> and then one extra select condition occurs, with no data to read.
> So the mystery is, why does select() think there is something to read from
> the socket when recv() thinks not?
How odd. That sure sounds like a kernel bug to me.
> I was starting to suspect it could be merely an oddity of Linux's socket
> implementation, but discovered that the same behaviour occurs on SGI/IRIX.
Do you know whether Linux wrote their own Unix-socket code, or just
adopted the Berkeley socket code? This could be a common failure in
a lot of Unixes, if it's actually in the Berkeley code...
> This is a local (Unix domain) socket, btw, so it's not a TCP/IP issue.
Do you see the same behavior if you use a TCP connection?
regards, tom lane