Thread: BUG #7814: Rotation of the log is not carried out.

The following bug has been logged on the website:

Bug reference:      7814
Logged by:          Tsunezumi
Email address:      tsunezumi@efficlabo.com
PostgreSQL version: 9.2.2
Operating system:   WindowsServer2008Standard 64bit
Description:        =


Rotation of the log is not carried out. =


"log_rotation_size" is not effective. =

tsunezumi@efficlabo.com wrote:

> Rotation of the log is not carried out.
>
> "log_rotation_size" is not effective.

We need more details.

http://wiki.postgresql.org/wiki/Guide_to_reporting_problems

-Kevin

I installed ordinarily.
I did not correct postgresql.conf.

Installation Directory
 C:\PostgreSQL\9.2

Data Directory
 C:\PostgreSQL\9.2\data

locale
 C


I do not have this problem on PostgreSQL 9.1.7.





On Thu, 17 Jan 2013 16:57:53 -0500
"Kevin Grittner" <kgrittn@mail.com> wrote:

> tsunezumi@efficlabo.com wrote:
>
> > Rotation of the log is not carried out.
> >
> > "log_rotation_size" is not effective.
>
> We need more details.
>
> http://wiki.postgresql.org/wiki/Guide_to_reporting_problems
>
> -Kevin

On Fri, Jan 18, 2013 at 3:26 AM, Tsunezumi <tsunezumi@efficlabo.com> wrote:
>
> I installed ordinarily.
> I did not correct postgresql.conf.
>
> Installation Directory
>  C:\PostgreSQL\9.2
>
> Data Directory
>  C:\PostgreSQL\9.2\data
>
> locale
>  C

Please post the results of the following query:

SELECT  'version'::text AS "name",
        version() AS "current_setting"
UNION ALL
SELECT  name,
        current_setting(name)
FROM pg_settings
WHERE NOT source=3D'default'AND NOT name IN
('config_file','data_directory','hba_file','ident_file','log_timezone','Dat=
eStyle','lc_messages','lc_monetary','lc_numeric','lc_time','timezone_abbrev=
iations','default_text_search_config','application_name','transaction_defer=
rable','transaction_isolation','transaction_read_only');

I corrected "postgresql.conf", in order to improve the phenomenon.
A result is the same.
It was same phenomenon on Windows8 professional 64bit.

"name","current_setting"
"version","PostgreSQL 9.2.2, compiled by Visual C++ build 1600, 64-bit"
"bytea_output","hex"
"client_encoding","SJIS"
"lc_collate","C"
"lc_ctype","C"
"listen_addresses","*"
"log_destination","stderr"
"log_line_prefix","%t %h %a"
"log_rotation_size","1MB"
"log_statement","all"
"logging_collector","on"
"max_connections","100"
"max_stack_depth","2MB"
"port","5433"
"server_encoding","UTF8"
"shared_buffers","32MB"
"TimeZone","Asia/Tokyo"
"wal_buffers","1MB"


On Fri, 18 Jan 2013 07:35:43 -0800
bricklen <bricklen@gmail.com> wrote:

> On Fri, Jan 18, 2013 at 3:26 AM, Tsunezumi <tsunezumi@efficlabo.com> wrote:
> >
> > I installed ordinarily.
> > I did not correct postgresql.conf.
> >
> > Installation Directory
> >  C:\PostgreSQL\9.2
> >
> > Data Directory
> >  C:\PostgreSQL\9.2\data
> >
> > locale
> >  C
>
> Please post the results of the following query:
>
> SELECT  'version'::text AS "name",
>         version() AS "current_setting"
> UNION ALL
> SELECT  name,
>         current_setting(name)
> FROM pg_settings
> WHERE NOT source='default'AND NOT name IN
>
('config_file','data_directory','hba_file','ident_file','log_timezone','DateStyle','lc_messages','lc_monetary','lc_numeric','lc_time','timezone_abbreviations','default_text_search_config','application_name','transaction_deferrable','transaction_isolation','transaction_read_only');

On Friday, January 18, 2013, Tsunezumi wrote:

>
> I installed ordinarily.
>

Could you be more specific?  I do not know what is ordinary for you.

I ordinarily install from source (although not on Windows).  Other people
ordinarily do it differently.


> I do not have this problem on PostgreSQL 9.1.7.
>

Could you show a directory listing of the pg_log directory under both 9.1.7
and 9.2.2, so that we can see what you are seeing that leads to the
conclusion that rotation is not occurring?

Cheers,

Jeff

On Tue, Jan 22, 2013 at 4:42 AM, Tsunezumi <tsunezumi@efficlabo.com> wrote:
> Thank you.
>
> I send the picture of a screen.
>

Thanks.

For what it is worth, I can reproduce this on Windows 7, using the
9.2.2 and 9.1.7 windows 64 installers from EDB (i.e.
http://www.enterprisedb.com/postgresql-922-installers-win64?ls=Crossover&type=Crossover),
with completely default installation (EDB changes the default to be
logging_collector=on), and using the below to blow up the log files
with error messages:

perl -le 'print "FOO$_;" foreach 1..1e7' | psql


9.1.7 rotates the log files in $DATA/pg_log, while 9.2.2 does not
rotate based on size.

PostgreSQL 9.1.7, compiled by Visual C++ build 1500, 64-bit
PostgreSQL 9.2.2, compiled by Visual C++ build 1600, 64-bit

I do not see the problem under 9.2.2 on Linux, it rotates as expected
(after turning logging_collector on)

PostgreSQL 9.2.2 on x86_64-unknown-linux-gnu, compiled by gcc (GCC)
4.4.6 20120305 (Red Hat 4.4.6-4), 64-bit

Cheers,

Jeff

Jeff Janes <jeff.janes@gmail.com> writes:
> For what it is worth, I can reproduce this on Windows 7, using the
> 9.2.2 and 9.1.7 windows 64 installers from EDB (i.e.
> http://www.enterprisedb.com/postgresql-922-installers-win64?ls=Crossover&type=Crossover),
> with completely default installation (EDB changes the default to be
> logging_collector=on), and using the below to blow up the log files
> with error messages:

> perl -le 'print "FOO$_;" foreach 1..1e7' | psql

> 9.1.7 rotates the log files in $DATA/pg_log, while 9.2.2 does not
> rotate based on size.

[ scratches head for awhile... ]  Hm, what you are testing is the
default rotation parameters, right?  log_rotation_age = 1 day,
log_rotation_size = 10MB?  So you didn't wait to see what would happen
at a rotation-age boundary.  If so, does "pg_ctl reload" kick it into
rotating once the file is too big?  (Or you could reduce
log_rotation_age to a small value, but that would make it hard to
tell which condition triggered rotation.)

Here's what I think is probably happening: on Windows, the syslogger
process actually contains two threads, one of which handles data
transfer while the other worries about things like log rotation.
In prior versions, the control thread woke up once a second, so it
would notice within one second if the data transfer thread had filled
the current file to more than log_rotation_size.  In 9.2, we got rid
of those "unnecessary" wakeups, which means log rotation won't occur
until there's some other event that kicks the control thread into
activity.

There's no bug on non-Windows because we don't use two threads on other
platforms; data transfer is done by the main thread so it's already
awake upon any write to the log file, and it won't miss the rotation
condition.  (Size-based rotations were therefore always more precise on
non-Windows machines...)

So what we need on Windows is for the data transfer thread to notice
when "Log_RotationSize > 0 && ftell(syslogFile) >= Log_RotationSize",
and then either signal the control thread to wake up, or do the
rotation itself.  Probably the former is less risk.

I'm not coding this fix though, since I'm not in a position to test it.

            regards, tom lane

On Tuesday, January 22, 2013, Tom Lane wrote:

> Jeff Janes <jeff.janes@gmail.com <javascript:;>> writes:
> > For what it is worth, I can reproduce this on Windows 7, using the
> > 9.2.2 and 9.1.7 windows 64 installers from EDB (i.e.
> >
> http://www.enterprisedb.com/postgresql-922-installers-win64?ls=Crossover&type=Crossover
> ),
> > with completely default installation (EDB changes the default to be
> > logging_collector=on), and using the below to blow up the log files
> > with error messages:
>
> > perl -le 'print "FOO$_;" foreach 1..1e7' | psql
>
> > 9.1.7 rotates the log files in $DATA/pg_log, while 9.2.2 does not
> > rotate based on size.
>
> [ scratches head for awhile... ]  Hm, what you are testing is the
> default rotation parameters, right?  log_rotation_age = 1 day,
> log_rotation_size = 10MB?  So you didn't wait to see what would happen
> at a rotation-age boundary.


Correct, I did not wait.  But from the original reporter's screenshot, it
seemed to be rotating correctly based on time, just not on size.


>  If so, does "pg_ctl reload" kick it into
> rotating once the file is too big?


Yes, triggering a reload (through pgAdmin) does cause it to kick over to a
new log file.

...

>
> So what we need on Windows is for the data transfer thread to notice
> when "Log_RotationSize > 0 && ftell(syslogFile) >= Log_RotationSize",
> and then either signal the control thread to wake up, or do the
> rotation itself.  Probably the former is less risk.
>
> I'm not coding this fix though, since I'm not in a position to test it.
>

If I can follow though on Andrew Dunstan's instructions to get mingw up and
running, I'll take it for a spin.

Cheers,

Jeff

Thank you.

I send the picture of a screen.



On Sat, 19 Jan 2013 14:21:29 -0800
Jeff Janes <jeff.janes@gmail.com> wrote:

> On Friday, January 18, 2013, Tsunezumi wrote:
>
> >
> > I installed ordinarily.
> >
>
> Could you be more specific?  I do not know what is ordinary for you.
>
> I ordinarily install from source (although not on Windows).  Other people
> ordinarily do it differently.
>
>
> > I do not have this problem on PostgreSQL 9.1.7.
> >
>
> Could you show a directory listing of the pg_log directory under both 9.1.7
> and 9.2.2, so that we can see what you are seeing that leads to the
> conclusion that rotation is not occurring?
>
> Cheers,
>
> Jeff

On Tue, Jan 22, 2013 at 9:29 PM, Jeff Janes <jeff.janes@gmail.com> wrote:
> On Tuesday, January 22, 2013, Tom Lane wrote:
>>
>>
>> So what we need on Windows is for the data transfer thread to notice
>> when "Log_RotationSize > 0 && ftell(syslogFile) >= Log_RotationSize",
>> and then either signal the control thread to wake up, or do the
>> rotation itself.  Probably the former is less risk.
>>
>> I'm not coding this fix though, since I'm not in a position to test it.
>
>
> If I can follow though on Andrew Dunstan's instructions to get mingw up and
> running, I'll take it for a spin.

Jeff Janes <jeff.janes@gmail.com> writes:
> On Tue, Jan 22, 2013 at 9:29 PM, Jeff Janes <jeff.janes@gmail.com> wrote:
>> On Tuesday, January 22, 2013, Tom Lane wrote:
>>> So what we need on Windows is for the data transfer thread to notice
>>> when "Log_RotationSize > 0 && ftell(syslogFile) >= Log_RotationSize",
>>> and then either signal the control thread to wake up, or do the
>>> rotation itself.  Probably the former is less risk.

> The attached patch fixes the problem, tested under mingw.
> This is my first effort in both Windows and in the Latch code, so committer
> beware.

Looks good, except I moved the test out of the if-block.  The ereport in
the other if branch could also fill the file, in principle, and in any
case it hardly seems like we need to optimize any of the non-success
cases.

> It should be applied to 9_2 and to head.

Done.  Thanks for doing this!

            regards, tom lane