Thread: log_line_prefix additions

log_line_prefix additions

From
"Ed L."
Date:
This patch against 8.0.0beta1 source adds log_line_prefix options for
millisecond timestamps (%m), remote host (%h), and remote port (%P).  The
milliseconds are useful for QPS measurements, and the remote port is
worthless to us as part of %r.



Attachment

Re: log_line_prefix additions

From
"Ed L."
Date:
Attached also is a patch to comments in sample postgresql.conf file.

Subject: [PATCHES]  log_line_prefix additions
Date: Wednesday August 25 2004 3:26
From: "Ed L." <pgsql@bluepolka.net>
To: pgsql-patches@postgresql.org

This patch against 8.0.0beta1 source adds log_line_prefix options for
millisecond timestamps (%m), remote host (%h), and remote port (%P).  The
milliseconds are useful for QPS measurements, and the remote port is
worthless to us as part of %r.

-------------------------------------------------------



Attachment

Re: log_line_prefix additions

From
Andrew Dunstan
Date:

Ed L. wrote:

>Attached also is a patch to comments in sample postgresql.conf file.
>
>Subject: [PATCHES]  log_line_prefix additions
>Date: Wednesday August 25 2004 3:26
>From: "Ed L." <pgsql@bluepolka.net>
>To: pgsql-patches@postgresql.org
>
>This patch against 8.0.0beta1 source adds log_line_prefix options for
>millisecond timestamps (%m), remote host (%h), and remote port (%P).  The
>milliseconds are useful for QPS measurements, and the remote port is
>worthless to us as part of %r.
>
>
>
[snip]

>+                     struct timezone tz = {0, 0};
>+
>+                     gettimeofday(&tv, &tz);
>
>
>



The timezone is pointless. Just pass NULL as the second argument to
gettimeofday().

Also, I don't understand what you mean by the remote port being
"worthless to us as part of %r." Please explain.

cheers

andrew



Re: log_line_prefix additions

From
"Ed L."
Date:
On Wednesday August 25 2004 4:25, Andrew Dunstan wrote:
> From: "Ed L." <pgsql@bluepolka.net>
>
> >To: pgsql-patches@postgresql.org
> >
> >This patch against 8.0.0beta1 source adds log_line_prefix options for
> >millisecond timestamps (%m), remote host (%h), and remote port (%P).
> > The milliseconds are useful for QPS measurements, and the remote port
> > is worthless to us as part of %r.
>
> [snip]
>
> I don't understand what you mean by the remote port being
> "worthless to us as part of %r." Please explain.

"Worthless to us as part of %r" means we don't use the remote port, and
would rather not have it in our logs.  With %h, we can log what we want
(hostname/IP), and others can still get both ip and port if they like with
"%h(%P)" or "%r".

Ed


Re: log_line_prefix additions

From
Bruce Momjian
Date:
This has been saved for the 8.1 release:

    http:/momjian.postgresql.org/cgi-bin/pgpatches2

---------------------------------------------------------------------------

Ed L. wrote:
>
> This patch against 8.0.0beta1 source adds log_line_prefix options for
> millisecond timestamps (%m), remote host (%h), and remote port (%P).  The
> milliseconds are useful for QPS measurements, and the remote port is
> worthless to us as part of %r.
>
>

[ Attachment, skipping... ]

[ Attachment, skipping... ]

>
> ---------------------------(end of broadcast)---------------------------
> TIP 5: Have you checked our extensive FAQ?
>
>                http://www.postgresql.org/docs/faqs/FAQ.html

--
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Re: log_line_prefix additions

From
Bruce Momjian
Date:
Patch additions added to the patch queue.

---------------------------------------------------------------------------


Andrew Dunstan wrote:
>
>
> Ed L. wrote:
>
> >Attached also is a patch to comments in sample postgresql.conf file.
> >
> >Subject: [PATCHES]  log_line_prefix additions
> >Date: Wednesday August 25 2004 3:26
> >From: "Ed L." <pgsql@bluepolka.net>
> >To: pgsql-patches@postgresql.org
> >
> >This patch against 8.0.0beta1 source adds log_line_prefix options for
> >millisecond timestamps (%m), remote host (%h), and remote port (%P).  The
> >milliseconds are useful for QPS measurements, and the remote port is
> >worthless to us as part of %r.
> >
> >
> >
> [snip]
>
> >+                     struct timezone tz = {0, 0};
> >+
> >+                     gettimeofday(&tv, &tz);
> >
> >
> >
>
>
>
> The timezone is pointless. Just pass NULL as the second argument to
> gettimeofday().
>
> Also, I don't understand what you mean by the remote port being
> "worthless to us as part of %r." Please explain.
>
> cheers
>
> andrew
>
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 8: explain analyze is your friend
>

--
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Re: log_line_prefix additions

From
Bruce Momjian
Date:
qEd L. wrote:
> Attached also is a patch to comments in sample postgresql.conf file.
>
> Subject: [PATCHES]  log_line_prefix additions
> Date: Wednesday August 25 2004 3:26
> From: "Ed L." <pgsql@bluepolka.net>
> To: pgsql-patches@postgresql.org
>
> This patch against 8.0.0beta1 source adds log_line_prefix options for
> millisecond timestamps (%m), remote host (%h), and remote port (%P).  The
> milliseconds are useful for QPS measurements, and the remote port is
> worthless to us as part of %r.

OK, I have modified your patch and applied it.  I changed gettimeofday()
to use NULL for timezone, as suggested by Andrew Dunstan.  I also
modified the code to not print the very long timezone name on Win32,
just as we do with the current non-millisecond timezone example.

Because we already have a remote host and port specification, it seems
unnecessary to add a remote port only specification.  I have removed the
%P/port part of your patch, so we have %r host/port and $h host-only.

One other approach would be to remove %r host/port, but %r has some
smarts about not printing the remote port number of it isn't available
(like unix sockets), and the remote port alone seems meaningless, so
this direction seems best.

(Some of these ideas are mine, some are Tom's.)

--
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073
Index: doc/src/sgml/runtime.sgml
===================================================================
RCS file: /cvsroot/pgsql/doc/src/sgml/runtime.sgml,v
retrieving revision 1.322
diff -c -c -r1.322 runtime.sgml
*** doc/src/sgml/runtime.sgml    4 Jun 2005 20:42:41 -0000    1.322
--- doc/src/sgml/runtime.sgml    9 Jun 2005 22:24:15 -0000
***************
*** 2830,2835 ****
--- 2830,2840 ----
               <entry>yes</entry>
              </row>
              <row>
+              <entry><literal>%h</literal></entry>
+              <entry>Remote Hostname or IP address</entry>
+              <entry>yes</entry>
+             </row>
+             <row>
               <entry><literal>%p</literal></entry>
               <entry>Process ID</entry>
               <entry>no</entry>
***************
*** 2840,2845 ****
--- 2845,2855 ----
               <entry>no</entry>
              </row>
              <row>
+              <entry><literal>%m</literal></entry>
+              <entry>Timestamp with milliseconds</entry>
+              <entry>no</entry>
+             </row>
+             <row>
               <entry><literal>%i</literal></entry>
               <entry>Command tag: This is the command that generated the log line.</entry>
               <entry>yes</entry>
Index: src/backend/utils/error/elog.c
===================================================================
RCS file: /cvsroot/pgsql/src/backend/utils/error/elog.c,v
retrieving revision 1.158
diff -c -c -r1.158 elog.c
*** src/backend/utils/error/elog.c    12 Mar 2005 01:54:44 -0000    1.158
--- src/backend/utils/error/elog.c    9 Jun 2005 22:24:17 -0000
***************
*** 1375,1380 ****
--- 1375,1407 ----
              case 'l':
                  appendStringInfo(buf, "%ld", log_line_number);
                  break;
+             case 'm':
+                 {
+                     time_t stamp_time;
+                     char strfbuf[128], msbuf[5];
+                     struct timeval tv;
+
+                     gettimeofday(&tv, NULL);
+                      stamp_time = tv.tv_sec;
+
+                     strftime(strfbuf, sizeof(strfbuf),
+                     /* leave room for milliseconds... */
+                     /* Win32 timezone names are too long so don't print them. */
+ #ifndef WIN32
+                         "%Y-%m-%d %H:%M:%S     %Z",
+ #else
+                         "%Y-%m-%d %H:%M:%S     ",
+ #endif
+                         localtime(&stamp_time));
+
+                     /* 'paste' milliseconds into place... */
+                      sprintf(msbuf, ".%03d",
+                         (int)(tv.tv_usec/1000));
+                     strncpy(strfbuf+19, msbuf, 4);
+
+                     appendStringInfoString(buf, strfbuf);
+                 }
+                 break;
              case 't':
                  {
                      /*
***************
*** 1426,1431 ****
--- 1453,1462 ----
                                           MyProcPort->remote_port);
                  }
                  break;
+             case 'h':
+                 if (MyProcPort)
+                     appendStringInfo(buf, "%s", MyProcPort->remote_host);
+                 break;
              case 'q':
                  /* in postmaster and friends, stop if %q is seen */
                  /* in a backend, just ignore */
Index: src/backend/utils/misc/postgresql.conf.sample
===================================================================
RCS file: /cvsroot/pgsql/src/backend/utils/misc/postgresql.conf.sample,v
retrieving revision 1.143
diff -c -c -r1.143 postgresql.conf.sample
*** src/backend/utils/misc/postgresql.conf.sample    4 Jun 2005 20:42:42 -0000    1.143
--- src/backend/utils/misc/postgresql.conf.sample    9 Jun 2005 22:24:17 -0000
***************
*** 241,248 ****
  #log_duration = false
  #log_line_prefix = ''        # e.g. '<%u%%%d> '
                  # %u=user name %d=database name
!                 # %r=remote host and port
                  # %p=PID %t=timestamp %i=command tag
                  # %c=session id %l=session line number
                  # %s=session start timestamp %x=transaction id
                  # %q=stop here in non-session processes
--- 241,249 ----
  #log_duration = false
  #log_line_prefix = ''        # e.g. '<%u%%%d> '
                  # %u=user name %d=database name
!                 # %r=remote host and port %h=remote host
                  # %p=PID %t=timestamp %i=command tag
+                 # %m=timestamp with milliseconds
                  # %c=session id %l=session line number
                  # %s=session start timestamp %x=transaction id
                  # %q=stop here in non-session processes