Thread: logfile rotation

logfile rotation

From
Andreas Pflug
Date:
Tom doesn't like returning the server's logfile using a pgsql function

unless logfile rotation is implemented, so here it is.

This proposal doesn't include a daemon that triggers pg_logfile_rotate
when appropriate (timestamp and/or length dependent), pg_autovacuum
might be a good place to do that.

I'd also like to see a table pg_logfile which contains all logfile
names, to be able to access older logfiles.


Comments?

Regards,
Andreas



Index: doc/src/sgml/func.sgml
===================================================================
RCS file: /projects/cvsroot/pgsql-server/doc/src/sgml/func.sgml,v
retrieving revision 1.206
diff -u -r1.206 func.sgml
--- doc/src/sgml/func.sgml    2 Jun 2004 21:34:49 -0000    1.206
+++ doc/src/sgml/func.sgml    13 Jun 2004 15:57:06 -0000
@@ -7308,6 +7308,80 @@
     columns do not have OIDs of their own.
    </para>

+   <indexterm zone="functions-misc">
+   <primary>pg_logfile_get</primary>
+   </indexterm>
+   <indexterm zone="functions-misc">
+   <primary>pg_logfile_length</primary>
+   </indexterm>
+   <indexterm zone="functions-misc">
+   <primary>pg_logfile_name</primary>
+   </indexterm>
+   <indexterm zone="functions-misc">
+   <primary>pg_logfile_rotate</primary>
+   </indexterm>
+   <para>
+    The functions shown in <xref linkend="functions-misc-logfile">
+    deal with the server log file if configured with log_destination
+    <quote>file</quote>.
+   </para>
+
+   <table id="functions-misc-logfile">
+    <title>Server Logfile Functions</title>
+    <tgroup cols="3">
+     <thead>
+      <row><entry>Name</entry> <entry>Return Type</entry> <entry>Description</entry></row>
+     </thead>
+
+     <tbody>
+      <row>
+       <entry><literal><function>pg_logfile_get</function>(<parameter>size_int4</parameter>,
+       <parameter>offset_int4</parameter>,<parameter>filename_text</parameter>)</literal></entry>
+       <entry><type>cstring</type></entry>
+       <entry>get a part of the current server log file</entry>
+      </row>
+      <row>
+       <entry><literal><function>pg_logfile_length</function>(<paramater>filename_text</parameter>)</literal></entry>
+       <entry><type>int4</type></entry>
+       <entry>return the current length of the server log file</entry>
+      </row>
+      <row>
+       <entry><literal><function>pg_logfile_rotate</function>()</literal></entry>
+       <entry><type>cstring</type></entry>
+       <entry>rotates the server log file and returns the new log file
+       name</entry>
+      </row>
+      <row>
+       <entry><literal><function>pg_logfile_name</function>()</literal></entry>
+       <entry><type>cstring</type></entry>
+       <entry>returns the current server log file name</entry>
+      </row>
+      <row>
+       <entry><literal><function>pg_logfile_rotate</function>()</literal></entry>
+       <entry><type>cstring</type></entry>
+       <entry>rotates the server log file and returns the previous log file
+       name</entry>
+      </row>
+      </tbody>
+</tgroup>
+</table>
+<para>
+The <function>pg_logfile_get</function> function will return the
+       contents of the current server log file, limited by the size
+       parameter. If size is NULL, a server internal limit (currently
+       50000) is applied. The position parameter specifies the
+       starting position of the server log chunk to be returned. A
+       positive number or 0 will be counted from the start of the file,
+       a negative number from the end; if NULL, -size is assumed
+       (i.e. the tail of the log file).
+</para>
+<para>
+Both <function>pg_logfile_get</function> and
+       <function>pg_logfile_length</function> have a filename
+       parameter which may specify the logfile to examine or the
+       current logfile if NULL.
+</para>
+
   </sect1>

  <sect1 id="functions-array">
Index: doc/src/sgml/runtime.sgml
===================================================================
RCS file: /projects/cvsroot/pgsql-server/doc/src/sgml/runtime.sgml,v
retrieving revision 1.266
diff -u -r1.266 runtime.sgml
--- doc/src/sgml/runtime.sgml    10 Jun 2004 22:26:17 -0000    1.266
+++ doc/src/sgml/runtime.sgml    13 Jun 2004 15:57:17 -0000
@@ -1721,14 +1721,25 @@
       <listitem>
        <para>
     <productname>PostgreSQL</productname> supports several methods
-     for loggning, including <systemitem>stderr</systemitem> and
-     <systemitem>syslog</systemitem>. On Windows,
-     <systemitem>eventlog</systemitem> is also supported. Set this
+     for logging, including <systemitem>stderr</systemitem>,
+     <systemitem>file></systemitem> and <systemitem>syslog</systemitem>.
+      On Windows, <systemitem>eventlog</systemitem> is also supported. Set this
      option to a list of desired log destinations separated by a
      comma. The default is to log to <systemitem>stderr</systemitem>
      only. This option must be set at server start.
        </para>
       </listitem>
+     </varlistentry>
+
+     <varlistentry id="guc-syslog-facility" xreflabel="log_filename">
+      <term><varname>log_filename</varname> (<type>string</type>)</term>
+       <listitem>
+        <para>
+          This option sets the target filename for the log destination
+          <quote>file</quote> option. It may be specified as absolute
+          path or relative to the <application>cluster directory</application>.
+        </para>
+       </listitem>
      </varlistentry>

      <varlistentry id="guc-syslog-facility" xreflabel="syslog_facility">
Index: src/backend/postmaster/postmaster.c
===================================================================
RCS file: /projects/cvsroot/pgsql-server/src/backend/postmaster/postmaster.c,v
retrieving revision 1.403
diff -u -r1.403 postmaster.c
--- src/backend/postmaster/postmaster.c    11 Jun 2004 03:54:43 -0000    1.403
+++ src/backend/postmaster/postmaster.c    13 Jun 2004 15:57:24 -0000
@@ -727,6 +727,11 @@
     reset_shared(PostPortNumber);

     /*
+     * Opens alternate log file
+     */
+    LogFileInit();
+
+    /*
      * Estimate number of openable files.  This must happen after setting
      * up semaphores, because on some platforms semaphores count as open
      * files.
Index: src/backend/storage/ipc/ipc.c
===================================================================
RCS file: /projects/cvsroot/pgsql-server/src/backend/storage/ipc/ipc.c,v
retrieving revision 1.87
diff -u -r1.87 ipc.c
--- src/backend/storage/ipc/ipc.c    12 Dec 2003 18:45:09 -0000    1.87
+++ src/backend/storage/ipc/ipc.c    13 Jun 2004 15:57:24 -0000
@@ -111,6 +111,8 @@
                               on_proc_exit_list[on_proc_exit_index].arg);

     elog(DEBUG3, "exit(%d)", code);
+
+//    LogFileClose();
     exit(code);
 }

Index: src/backend/utils/adt/misc.c
===================================================================
RCS file: /projects/cvsroot/pgsql-server/src/backend/utils/adt/misc.c,v
retrieving revision 1.34
diff -u -r1.34 misc.c
--- src/backend/utils/adt/misc.c    2 Jun 2004 21:29:29 -0000    1.34
+++ src/backend/utils/adt/misc.c    13 Jun 2004 15:57:30 -0000
@@ -103,3 +103,130 @@
 {
     PG_RETURN_INT32(pg_signal_backend(PG_GETARG_INT32(0),SIGINT));
 }
+
+
+
+
+extern FILE *logfile; // in elog.c
+#define MAXLOGFILECHUNK 50000
+
+static char *absClusterPath(text *arg)
+{
+    char *filename;
+
+    if (is_absolute_path(VARDATA(arg)))
+        filename=VARDATA(arg);
+    else
+    {
+        filename = palloc(strlen(DataDir)+VARSIZE(arg)+2);
+        sprintf(filename, "%s/%s", DataDir, VARDATA(arg));
+    }
+    return filename;
+}
+
+
+Datum pg_logfile_get(PG_FUNCTION_ARGS)
+{
+    size_t size=MAXLOGFILECHUNK;
+    char *buf=0;
+    size_t nbytes;
+    FILE *f;
+
+    if (!PG_ARGISNULL(0))
+        size = PG_GETARG_INT32(0);
+    if (size > MAXLOGFILECHUNK)
+    {
+        size = MAXLOGFILECHUNK;
+        ereport(WARNING,
+                (errcode(ERRCODE_OUT_OF_MEMORY),
+                 errmsg("Maximum size is %d.", size)));
+    }
+
+    if (PG_ARGISNULL(2))
+        f = logfile;
+    else
+    {
+        /* explicitely named logfile */
+        char *filename = absClusterPath(PG_GETARG_TEXT_P(2));
+        f = fopen(filename, "r");
+        if (!f)
+        {
+            ereport(WARNING,
+                    (errcode_for_file_access(),
+                     errmsg("file not found %s", filename)));
+            PG_RETURN_NULL();
+        }
+    }
+
+    if (f)
+    {
+
+        if (PG_ARGISNULL(1))
+            fseek(f, -size, SEEK_END);
+        else
+        {
+            long pos = PG_GETARG_INT32(1);
+            if (pos >= 0)
+                fseek(f, pos, SEEK_SET);
+            else
+                fseek(f, pos, SEEK_END);
+        }
+        buf = palloc(size+1);
+        nbytes = fread(buf, 1, size, f);
+        buf[nbytes] = 0;
+
+        fseek(f, 0, SEEK_END);
+
+        if (!PG_ARGISNULL(2))
+            fclose(f);
+    }
+
+    if (buf)
+        PG_RETURN_CSTRING(buf);
+    else
+        PG_RETURN_NULL();
+}
+
+
+Datum pg_logfile_length(PG_FUNCTION_ARGS)
+{
+    if (PG_ARGISNULL(0))
+    {
+        if (logfile)
+            PG_RETURN_INT32(ftell(logfile));
+    }
+    else
+    {
+        struct stat fst;
+        fst.st_size=0;
+        stat(absClusterPath(PG_GETARG_TEXT_P(0)), &fst);
+
+        PG_RETURN_INT32(fst.st_size);
+    }
+    PG_RETURN_INT32(0);
+}
+
+
+Datum pg_logfile_name(PG_FUNCTION_ARGS)
+{
+    char *filename=LogFileName();
+    if (filename)
+    {
+        if (strncmp(filename, DataDir, strlen(DataDir)))
+            PG_RETURN_CSTRING(filename);
+        else
+            PG_RETURN_CSTRING(filename+strlen(DataDir)+1);
+    }
+    PG_RETURN_NULL();
+}
+
+
+Datum pg_logfile_rotate(PG_FUNCTION_ARGS)
+{
+    char *renamedFile = LogFileRotate();
+
+    if (renamedFile)
+        PG_RETURN_CSTRING(renamedFile);
+    else
+        PG_RETURN_NULL();
+}
Index: src/backend/utils/error/elog.c
===================================================================
RCS file: /projects/cvsroot/pgsql-server/src/backend/utils/error/elog.c,v
retrieving revision 1.140
diff -u -r1.140 elog.c
--- src/backend/utils/error/elog.c    3 Jun 2004 02:08:04 -0000    1.140
+++ src/backend/utils/error/elog.c    13 Jun 2004 15:57:33 -0000
@@ -63,7 +63,6 @@
 #include "utils/memutils.h"
 #include "utils/guc.h"

-
 /* Global variables */
 ErrorContextCallback *error_context_stack = NULL;

@@ -71,9 +70,22 @@
 PGErrorVerbosity Log_error_verbosity = PGERROR_VERBOSE;
 char       *Log_line_prefix = NULL; /* format for extra log line info */
 unsigned int Log_destination = LOG_DESTINATION_STDERR;
+char *Log_filename = NULL;

 bool in_fatal_exit = false;

+typedef struct
+{
+    long fileversion;
+    char filename[MAXPGPATH];
+}
+logfileShmemStruct;
+
+FILE *logfile = NULL;
+static long logfileVersion=0;
+static logfileShmemStruct *logfileShmem  = NULL;
+
+
 #ifdef HAVE_SYSLOG
 char       *Syslog_facility;    /* openlog() parameters */
 char       *Syslog_ident;
@@ -140,6 +152,9 @@
 static const char *error_severity(int elevel);
 static void append_with_tabs(StringInfo buf, const char *str);

+static char *logfile_newname(void);
+static void logfile_reopen(void);
+

 /*
  * errstart --- begin an error-reporting cycle
@@ -931,11 +946,145 @@
     /*
      * And let errfinish() finish up.
      */
+
     errfinish(0);
 }


 /*
+ * Initialize shared mem for logfile rotation
+ */
+
+void
+LogFileInit(void)
+{
+    if (Log_filename && (Log_destination & LOG_DESTINATION_FILE))
+    {
+        logfileShmem = ShmemAlloc(sizeof(logfileShmemStruct));
+        if (!logfileShmem)
+        {
+            ereport(FATAL,
+                    (errcode(ERRCODE_OUT_OF_MEMORY),
+                     errmsg("Out of shared memory")));
+            return;
+        }
+
+        memset(logfileShmem, 0, sizeof(logfileShmemStruct));
+
+        char *fn = logfile_newname();
+        if (fn)
+        {
+            strcpy(logfileShmem->filename, fn);
+            logfile_reopen();
+            pfree(fn);
+        }
+    }
+}
+
+
+/*
+ * Rotate log file
+ */
+char *
+LogFileRotate(void)
+{
+    char *oldFilename;
+    char *filename;
+
+    if (!logfileShmem || !(Log_destination & LOG_DESTINATION_FILE))
+        return NULL;
+
+    filename = logfile_newname();
+    if (!filename)
+        return NULL;
+
+
+    if (!strcmp(filename, logfileShmem->filename))
+    {
+        ereport(ERROR,
+                (errcode(ERRCODE_CONFIG_FILE_ERROR),
+                 errmsg("Log_filename not suitable for rotation.")));
+        return NULL;
+    }
+
+    oldFilename = pstrdup(logfileShmem->filename);
+    strcpy(logfileShmem->filename, filename);
+
+    logfileShmem->fileversion++;
+
+    ereport(NOTICE,
+            (errcode(ERRCODE_WARNING),
+             errmsg("Opened new log file %s; previous logfile %s", logfileShmem->filename, oldFilename)));
+
+    return oldFilename;
+}
+
+
+/*
+ * return current log file name
+ */
+char*
+LogFileName(void)
+{
+    if (logfileShmem)
+        return logfileShmem->filename;
+    return NULL;
+}
+
+
+/*
+ * creates a new logfile name using current timestamp information
+ */
+static char*
+logfile_newname()
+{
+    char *filetemplate;
+    char *filename;
+    pg_time_t now = time(NULL);
+
+    if (is_absolute_path(Log_filename))
+        filetemplate = pstrdup(Log_filename);
+    else
+    {
+        filetemplate = palloc(strlen(DataDir) + strlen(Log_filename) + 2);
+        sprintf(filetemplate, "%s/%s", DataDir, Log_filename);
+    }
+    filename = palloc(MAXPGPATH);
+    pg_strftime(filename, MAXPGPATH, filetemplate, pg_localtime(&now));
+
+    pfree(filetemplate);
+
+    return filename;
+}
+
+
+/*
+ * reopen log file.
+ */
+static void
+logfile_reopen(void)
+{
+    if (logfile)
+    {
+        fclose(logfile);
+        logfile = NULL;
+    }
+
+    if ((Log_destination & LOG_DESTINATION_FILE) && logfileShmem)
+    {
+        logfileVersion = logfileShmem->fileversion;
+
+        logfile = fopen(logfileShmem->filename, "a+");
+
+        if (!logfile)
+            ereport(ERROR,
+                    (errcode_for_file_access(),
+                     errmsg("failed to open log file %s", logfileShmem->filename)));
+    }
+}
+
+
+/*
  * Initialization of error output file
  */
 void
@@ -1445,6 +1594,24 @@
     if ((Log_destination & LOG_DESTINATION_STDERR) || whereToSendOutput == Debug)
     {
         fprintf(stderr, "%s", buf.data);
+    }
+
+    /* Write to file, if enabled */
+    if (logfile && (Log_destination & LOG_DESTINATION_FILE))
+    {
+        /* check if logfile changed */
+        if (logfileShmem && logfileShmem->fileversion != logfileVersion)
+        {
+            logfileVersion = logfileShmem->fileversion;
+            logfile_reopen();
+        }
+
+        if (logfile)
+        {
+            fseek(logfile, 0, SEEK_END);
+            fprintf(logfile, "%s", buf.data);
+            fflush(logfile);
+        }
     }

     pfree(buf.data);
Index: src/backend/utils/misc/guc.c
===================================================================
RCS file: /projects/cvsroot/pgsql-server/src/backend/utils/misc/guc.c,v
retrieving revision 1.211
diff -u -r1.211 guc.c
--- src/backend/utils/misc/guc.c    11 Jun 2004 03:54:54 -0000    1.211
+++ src/backend/utils/misc/guc.c    13 Jun 2004 15:57:42 -0000
@@ -76,6 +76,8 @@
 static const char *assign_log_destination(const char *value,
                 bool doit, GucSource source);

+extern char *Log_filename;
+
 #ifdef HAVE_SYSLOG
 extern char *Syslog_facility;
 extern char *Syslog_ident;
@@ -1644,13 +1646,23 @@
     {
         {"log_destination", PGC_POSTMASTER, LOGGING_WHERE,
          gettext_noop("Sets the target for log output."),
-         gettext_noop("Valid values are combinations of stderr, syslog "
+         gettext_noop("Valid values are combinations of stderr, file, syslog "
                       "and eventlog, depending on platform."),
          GUC_LIST_INPUT | GUC_REPORT
         },
         &log_destination_string,
         "stderr", assign_log_destination, NULL
     },
+    {
+        {"log_filename", PGC_POSTMASTER, LOGGING_WHERE,
+         gettext_noop("Sets the target filename for log output."),
+         gettext_noop("May be specified as relative to the cluster directory "
+                      "or as absolute path."),
+         GUC_LIST_INPUT | GUC_REPORT
+        },
+        &Log_filename,
+        "postgresql.log.%Y-%m-%d_%H%M%S", NULL, NULL
+    },

 #ifdef HAVE_SYSLOG
     {
@@ -4779,6 +4791,8 @@

         if (pg_strcasecmp(tok,"stderr") == 0)
             newlogdest |= LOG_DESTINATION_STDERR;
+        else if (pg_strcasecmp(tok,"file") == 0)
+            newlogdest |= LOG_DESTINATION_FILE;
 #ifdef HAVE_SYSLOG
         else if (pg_strcasecmp(tok,"syslog") == 0)
             newlogdest |= LOG_DESTINATION_SYSLOG;
Index: src/backend/utils/misc/postgresql.conf.sample
===================================================================
RCS file: /projects/cvsroot/pgsql-server/src/backend/utils/misc/postgresql.conf.sample,v
retrieving revision 1.113
diff -u -r1.113 postgresql.conf.sample
--- src/backend/utils/misc/postgresql.conf.sample    7 Apr 2004 05:05:50 -0000    1.113
+++ src/backend/utils/misc/postgresql.conf.sample    13 Jun 2004 15:57:42 -0000
@@ -147,9 +147,12 @@

 # - Where to Log -

-#log_destination = 'stderr'    # Valid values are combinations of stderr,
+#log_destination = 'stderr'    # Valid values are combinations of stderr, file,
                                 # syslog and eventlog, depending on
                                 # platform.
+#log_filename = 'postgresql.log.%Y-%m-%d_%H%M%S' # filename if
+                                # 'file' log_destination is used.
+
 #syslog_facility = 'LOCAL0'
 #syslog_ident = 'postgres'

Index: src/include/catalog/pg_proc.h
===================================================================
RCS file: /projects/cvsroot/pgsql-server/src/include/catalog/pg_proc.h,v
retrieving revision 1.335
diff -u -r1.335 pg_proc.h
--- src/include/catalog/pg_proc.h    6 Jun 2004 19:07:00 -0000    1.335
+++ src/include/catalog/pg_proc.h    13 Jun 2004 15:57:57 -0000
@@ -3588,6 +3588,16 @@
 DATA(insert OID = 2243 ( bit_or                           PGNSP PGUID 12 t f f f i 1 1560 "1560" _null_
aggregate_dummy- _null_)); 
 DESCR("bitwise-or bit aggregate");

+DATA(insert OID = 2550(  pg_logfile_get                PGNSP PGUID 12 f f f f v 3 2275 "23 23 25" _null_
pg_logfile_get- _null_ )); 
+DESCR("return log file contents");
+DATA(insert OID = 2551(  pg_logfile_length               PGNSP PGUID 12 f f f f v 1 23 "25" _null_ pg_logfile_length -
_null_)); 
+DESCR("name of log file");
+DATA(insert OID = 2552(  pg_logfile_name               PGNSP PGUID 12 f f f f v 0 2275 "" _null_ pg_logfile_name -
_null_)); 
+DESCR("length of log file");
+DATA(insert OID = 2553(  pg_logfile_rotate               PGNSP PGUID 12 f f f f v 0 2275 "" _null_ pg_logfile_rotate -
_null_)); 
+DESCR("rotate log file");
+
+
 /*
  * Symbolic values for provolatile column: these indicate whether the result
  * of a function is dependent *only* on the values of its explicit arguments,
Index: src/include/utils/builtins.h
===================================================================
RCS file: /projects/cvsroot/pgsql-server/src/include/utils/builtins.h,v
retrieving revision 1.241
diff -u -r1.241 builtins.h
--- src/include/utils/builtins.h    2 Jun 2004 21:29:29 -0000    1.241
+++ src/include/utils/builtins.h    13 Jun 2004 15:58:00 -0000
@@ -356,6 +356,10 @@
 extern Datum current_database(PG_FUNCTION_ARGS);
 extern Datum pg_terminate_backend(PG_FUNCTION_ARGS);
 extern Datum pg_cancel_backend(PG_FUNCTION_ARGS);
+extern Datum pg_logfile_get(PG_FUNCTION_ARGS);
+extern Datum pg_logfile_length(PG_FUNCTION_ARGS);
+extern Datum pg_logfile_name(PG_FUNCTION_ARGS);
+extern Datum pg_logfile_rotate(PG_FUNCTION_ARGS);

 /* not_in.c */
 extern Datum int4notin(PG_FUNCTION_ARGS);
Index: src/include/utils/elog.h
===================================================================
RCS file: /projects/cvsroot/pgsql-server/src/include/utils/elog.h,v
retrieving revision 1.68
diff -u -r1.68 elog.h
--- src/include/utils/elog.h    5 Apr 2004 03:02:10 -0000    1.68
+++ src/include/utils/elog.h    13 Jun 2004 15:58:01 -0000
@@ -182,8 +182,11 @@
 #define LOG_DESTINATION_STDERR   1
 #define LOG_DESTINATION_SYSLOG   2
 #define LOG_DESTINATION_EVENTLOG 4
+#define LOG_DESTINATION_FILE     8

 /* Other exported functions */
 extern void DebugFileOpen(void);
-
+extern void LogFileInit(void);
+extern char *LogFileRotate(void);
+extern char *LogFileName(void);
 #endif   /* ELOG_H */


Re: logfile rotation

From
Tom Lane
Date:
Andreas Pflug <pgadmin@pse-consulting.de> writes:
> Tom doesn't like returning the server's logfile using a pgsql function
> unless logfile rotation is implemented, so here it is.

I'll repeat what I said in response to your other posting:

This uses a shared memory area with no lock, which seems a bad design;
but the alternative of having a lock is even worse, since the postmaster
would also have to take the lock.  We agreed long ago that the
postmaster should never depend on the correctness of any shared memory
data structure; but this patch would make it do so.

I really don't think this is an acceptable solution.
        regards, tom lane


Re: logfile rotation

From
Andreas Pflug
Date:
Tom Lane wrote:

>I'll repeat what I said in response to your other posting:
>  
>
Hm? I never posted something with shared mem usage before, what do you mean?

>This uses a shared memory area with no lock, which seems a bad design;
>  
>

AFAICS there should be no lock necessary.


>We agreed long ago that the
>postmaster should never depend on the correctness of any shared memory
>data structure; but this patch would make it do so.
>  
>
I understand that, so what's the suggested way to store data common for 
all backends?

Regards,
Andreas



Re: logfile rotation

From
Andreas Pflug
Date:
Andreas Pflug wrote:

>
>> We agreed long ago that the
>> postmaster should never depend on the correctness of any shared memory
>> data structure; but this patch would make it do so.
>>  
>>
> I understand that, so what's the suggested way to store data common 
> for all backends?


Answering my own question, the distribution of the current logfile name 
could be done trough a file handle. So the only thing remaining in 
shared mem would be the "reopen logfile" flag, which seems 
nonproblematic. In case of garbled sharedmem, a backend would reopen the 
logfile, which does no harm at all, or continue writing the outdated 
logfile which would be annoying but not harmful either.

Acceptable?

Regards,
Andreas



Re: logfile rotation

From
Andreas Pflug
Date:
Andreas Pflug wrote:

> Andreas Pflug wrote:
>
>>
>>> We agreed long ago that the
>>> postmaster should never depend on the correctness of any shared memory
>>> data structure; but this patch would make it do so.
>>>  
>>>
>> I understand that, so what's the suggested way to store data common 
>> for all backends?
>
>
>
> Answering my own question, the distribution of the current logfile 
> name could be done trough a file handle. So the only thing remaining 
> in shared mem would be the "reopen logfile" flag, which seems 
> nonproblematic. In case of garbled sharedmem, a backend would reopen 
> the logfile, which does no harm at all, or continue writing the 
> outdated logfile which would be annoying but not harmful either.
>
> Acceptable?


Tom,
would you mind commenting on my suggestion so I can continue on that topic?

Regards,
Andreas



Re: logfile rotation

From
Tom Lane
Date:
Andreas Pflug <pgadmin@pse-consulting.de> writes:
>> Answering my own question, the distribution of the current logfile 
>> name could be done trough a file handle.

> would you mind commenting on my suggestion so I can continue on that topic?

There is no portable way to redistribute a file handle.
        regards, tom lane


Re: logfile rotation

From
Andreas Pflug
Date:
Tom Lane wrote:

>Andreas Pflug <pgadmin@pse-consulting.de> writes:
>  
>
>>>Answering my own question, the distribution of the current logfile 
>>>name could be done trough a file handle.
>>>      
>>>
>
>  
>
>>would you mind commenting on my suggestion so I can continue on that topic?
>>    
>>
>
>There is no portable way to redistribute a file handle.
>  
>

Seems I didn't make clear enough what I mean.

I'd fopen a file handle  in the postmaster, and all subsequent processes 
will inherit that handle just as they do for stderr; no redistribution 
required.
The log filename is written to that file when pg_logfile_rotate is called:
fseek(fh, 0, SEEK_SET);
fprintf(fh, "%s", newlogfilname);
fflush();

and all subprocesses may retrieve the filename when required by

char buf[MAXPGPATH];
fseek(fh, 0, SEEK_SET);
fread(buf, 1, MAXPGPATH, fh);
buf[MAXPGPATH-1]=0; // prevent buffer overflow
logfile=fopen(buf, "a+");

Regards,
Andreas




Re: logfile rotation

From
Bruce Momjian
Date:
Actually, this is the current state of this issue.

---------------------------------------------------------------------------

Andreas Pflug wrote:
> Tom Lane wrote:
> 
> >Andreas Pflug <pgadmin@pse-consulting.de> writes:
> >  
> >
> >>>Answering my own question, the distribution of the current logfile 
> >>>name could be done trough a file handle.
> >>>      
> >>>
> >
> >  
> >
> >>would you mind commenting on my suggestion so I can continue on that topic?
> >>    
> >>
> >
> >There is no portable way to redistribute a file handle.
> >  
> >
> 
> Seems I didn't make clear enough what I mean.
> 
> I'd fopen a file handle  in the postmaster, and all subsequent processes 
> will inherit that handle just as they do for stderr; no redistribution 
> required.
> The log filename is written to that file when pg_logfile_rotate is called:
> fseek(fh, 0, SEEK_SET);
> fprintf(fh, "%s", newlogfilname);
> fflush();
> 
> and all subprocesses may retrieve the filename when required by
> 
> char buf[MAXPGPATH];
> fseek(fh, 0, SEEK_SET);
> fread(buf, 1, MAXPGPATH, fh);
> buf[MAXPGPATH-1]=0; // prevent buffer overflow
> logfile=fopen(buf, "a+");
> 
> Regards,
> Andreas
> 
> 

--  Bruce Momjian                        |  http://candle.pha.pa.us pgman@candle.pha.pa.us               |  (610)
359-1001+  If your life is a hard drive,     |  13 Roberts Road +  Christ can be your backup.        |  Newtown Square,
Pennsylvania19073
 


Re: logfile rotation

From
Andreas Pflug
Date:
Bruce Momjian wrote:

>Actually, this is the current state of this issue.
>  
>

Right, please comment on this. To recall, it uses shared memory for a 
"switch to next logfile name" flag, which can't cause harm in case of 
shmem corruption, and a postmaster opened filehandle (kept open) to a 
dummy file containing the actual file name (pseudo code below).

Regards,
Andreas

>---------------------------------------------------------------------------
>
>Andreas Pflug wrote:
>  
>
>>Tom Lane wrote:
>>
>>    
>>
>>>Andreas Pflug <pgadmin@pse-consulting.de> writes:
>>> 
>>>
>>>      
>>>
>>>>>Answering my own question, the distribution of the current logfile 
>>>>>name could be done trough a file handle.
>>>>>     
>>>>>
>>>>>          
>>>>>
>>> 
>>>
>>>      
>>>
>>>>would you mind commenting on my suggestion so I can continue on that topic?
>>>>   
>>>>
>>>>        
>>>>
>>>There is no portable way to redistribute a file handle.
>>> 
>>>
>>>      
>>>
>>Seems I didn't make clear enough what I mean.
>>
>>I'd fopen a file handle  in the postmaster, and all subsequent processes 
>>will inherit that handle just as they do for stderr; no redistribution 
>>required.
>>The log filename is written to that file when pg_logfile_rotate is called:
>>fseek(fh, 0, SEEK_SET);
>>fprintf(fh, "%s", newlogfilname);
>>fflush();
>>
>>and all subprocesses may retrieve the filename when required by
>>
>>char buf[MAXPGPATH];
>>fseek(fh, 0, SEEK_SET);
>>fread(buf, 1, MAXPGPATH, fh);
>>buf[MAXPGPATH-1]=0; // prevent buffer overflow
>>logfile=fopen(buf, "a+");
>>
>>Regards,
>>Andreas
>>
>>
>>    
>>
>
>  
>