Thread: Re: Re: [HACKERS] statement logging / extended query protocol

Re: Re: [HACKERS] statement logging / extended query protocol

From
Date:
David Fetter <david@fetter.org> wrote on 26.09.2005, 18:31:16:
> On Sun, Sep 25, 2005 at 11:41:11PM -0400, Bruce Momjian wrote:
> > Tom Lane wrote:
> > > Bruce Momjian  writes:
> > > > OK.  At this stage it should all just wait for 8.2.
> > >
> > > Haven't we already committed some feature changes in this area?  I
> > > dislike the idea of shipping a transient state just because "we
> > > ran out of time".  Once we ship 8.1, that will be another behavior
> > > that we have to consider backwards compatibility with.  We should
> > > either fix it right, or revert to the 8.0 behavior for this
> > > release.
> >
> > I documented in TODO what doesn't log properly in 8.1.  I don't see
> > how logging output has to be backward compatible.
>
> Dunno about this particular case, but not having backward-compatible
> logging output has already broken pqa.  Proposal on logging to
> -hackers follows...

Guys,

The changes I initiated, were caused solely by the complete inability of
PostgreSQL 8.0 log files to allow performance tuning for v3 protocol
clients (i.e. JDBC as the main one). That's been a major obstacle for
everybody I have talked to all year.

I had no wish at all to alter logging, but neither did anyone else, so
something had to be done. Regression to 8.0 way of doing things (i.e.
no logging) is *not* an option unless we want to lose friends, rather
than gain them. So, the log format needs changing... and we have no
need for backwards compatibility.

The current state of logging is that there is sufficient info in the
logs to fully support performance tuning whatever is thrown against the
server, v2 or v3, client or server. Bind parameters are useful but the
use case for that is more to do with debugging support than tuning. So
the current state is that the logging is not in a transient state, its
in a useful end state for an important use case. Though further testing
by all concerned is still required...

Recent changes are only the result of Oliver pointing out a flaw/hole in
the logging for complex v3 query streams. Credit and thanks. Nobody was
planning to do this so late in the day. Least of all myself because I
haven't worked much on the client protocol internals.

(pqa 1.5 is broken. It doesn't even run its sample 7.4 log file
properly. Thats a separate issue to what the log file format is. I do
like 1.4 though.)

A very public thank you to Bruce for running with me on this at every
stage. I think my own name should be secondary to his on the credit,
should those patches stay applied.

Best Regards, Simon Riggs

Re: [HACKERS] statement logging / extended query protocol

From
David Fetter
Date:
On Mon, Sep 26, 2005 at 11:50:01PM +0200, simon@2ndquadrant.com wrote:
>
> David Fetter <david@fetter.org> wrote on 26.09.2005, 18:31:16:
> > On Sun, Sep 25, 2005 at 11:41:11PM -0400, Bruce Momjian wrote:
> > > Tom Lane wrote:
> > > > Bruce Momjian  writes:
> > > > > OK.  At this stage it should all just wait for 8.2.
> > > >
> > > > Haven't we already committed some feature changes in this
> > > > area?  I dislike the idea of shipping a transient state just
> > > > because "we ran out of time".  Once we ship 8.1, that will be
> > > > another behavior that we have to consider backwards
> > > > compatibility with.  We should either fix it right, or revert
> > > > to the 8.0 behavior for this release.
> > >
> > > I documented in TODO what doesn't log properly in 8.1.  I don't
> > > see how logging output has to be backward compatible.
> >
> > Dunno about this particular case, but not having
> > backward-compatible logging output has already broken pqa.
> > Proposal on logging to -hackers follows...
>
> Guys,
>
> The changes I initiated, were caused solely by the complete
> inability of PostgreSQL 8.0 log files to allow performance tuning
> for v3 protocol clients (i.e. JDBC as the main one). That's been a
> major obstacle for everybody I have talked to all year.

I agree that you fixed a bug, and I apologize for haring off on that
thread with a not-quite-related suggestion.

> I had no wish at all to alter logging, but neither did anyone else, so
> something had to be done. Regression to 8.0 way of doing things (i.e.
> no logging) is *not* an option unless we want to lose friends, rather
> than gain them. So, the log format needs changing... and we have no
> need for backwards compatibility.

Right.

> (pqa 1.5 is broken. It doesn't even run its sample 7.4 log file
> properly. Thats a separate issue to what the log file format is. I
> do like 1.4 though.)

pqa 1.5 works on log files from 8.0 if they're only prefixed with a
timestamp.

Cheers,
D
--
David Fetter david@fetter.org http://fetter.org/
phone: +1 510 893 6100   mobile: +1 415 235 3778

Remember to vote!