Re: syslog performance when logging big statements - Mailing list pgsql-performance

From achill@matrix
Subject Re: syslog performance when logging big statements
Date
Msg-id 200807091531.05372.achill@matrix.gatewaynet.com
Whole thread Raw
In response to Re: syslog performance when logging big statements  (Tom Lane <tgl@sss.pgh.pa.us>)
List pgsql-performance
Στις Tuesday 08 July 2008 21:34:01 ο/η Tom Lane έγραψε:
> Achilleas Mantzios <achill@matrix.gatewaynet.com> writes:
> > Στις Tuesday 08 July 2008 17:35:16 ο/η Tom Lane έγραψε:
> >> Hmm.  There's a function in elog.c that breaks log messages into chunks
> >> for syslog.  I don't think anyone's ever looked hard at its performance
> >> --- maybe there's an O(N^2) behavior?
> >
> > Thanx,
> > i changed PG_SYSLOG_LIMIT in elog.c:1269 from 128 to 1048576
> > and i got super fast stderr performance. :)
>
> Doesn't seem like a very good solution given its impact on the stack
> depth right there.
>
> Looking at the code, the only bit that looks like repeated work are the
> repeated calls to strchr(), which would not be an issue in the "typical"
> case where the very long message contains reasonably frequent newlines.
> Am I right in guessing that your problematic statement contained
> megabytes worth of text with nary a newline?

Yes it was the input source of a bytea field containing tiff or pdf data,
of size of some 1-3 megabytes, and it did not contain (many) newlines.

>
> If so, we can certainly fix it by arranging to remember the last
> strchr() result across loop iterations, but I'd like to confirm the
> theory before doing that work.
>
>             regards, tom lane



pgsql-performance by date:

Previous
From: "achill@matrix"
Date:
Subject: Re: syslog performance when logging big statements
Next
From: Vivek Khera
Date:
Subject: Re: max fsm pages question