Thread: Specifying the log file name of pgbench -l option

Specifying the log file name of pgbench -l option

From
Masahiko Sawada
Date:
Hi all,

The log file generated by pgbench -l option is fixed file name
'pgbench_log.<pid>.<thread id>'. And it's a little complicated for the
script that runs pgbench repeatedly to identify the log file name.
Attached patch make it possible to specify the log file name. I think
it's useful for the use who want to run pgbench repeatedly in script
and collects and analyze the result.

The one thing I concern is that this patch changes -l option so that
it requires argument.
But changing its behavior would be good rather than adding new option.

Please give me feedback.

Regards,

--
Masahiko Sawada
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center

Attachment

Re: Specifying the log file name of pgbench -l option

From
Fabien COELHO
Date:
> The log file generated by pgbench -l option is fixed file name
> 'pgbench_log.<pid>.<thread id>'. And it's a little complicated for the
> script that runs pgbench repeatedly to identify the log file name.
> Attached patch make it possible to specify the log file name. I think
> it's useful for the use who want to run pgbench repeatedly in script
> and collects and analyze the result.
>
> The one thing I concern is that this patch changes -l option so that
> it requires argument.
> But changing its behavior would be good rather than adding new option.
>
> Please give me feedback.

Patch applies but does not compile, because "logfilename" is not declared.
I guess "logfile" was meant instead.

I understand and agree that in some case having only a predefined file 
prefix in the current directory as the only option can be a hindrance for 
scripts which use pgbench and rely on the log.

I'm not at ease either with changing the behavior of such an option, as 
some people may be happy with it and some script may be using it. I would 
suggest not to do so.

Moreover, what is provided is not a file name, but a prefix used to build 
file names.

So I would suggest to: - fix the compilation issue - leave -l/--log as it is, i.e. use "pgbench_log" as a prefix - add
--log-prefix=...(long option only) for changing this prefix
 

-- 
Fabien.



Re: Specifying the log file name of pgbench -l option

From
Masahiko Sawada
Date:
On Wed, Nov 2, 2016 at 1:41 AM, Fabien COELHO <coelho@cri.ensmp.fr> wrote:
>
>> The log file generated by pgbench -l option is fixed file name
>> 'pgbench_log.<pid>.<thread id>'. And it's a little complicated for the
>> script that runs pgbench repeatedly to identify the log file name.
>> Attached patch make it possible to specify the log file name. I think
>> it's useful for the use who want to run pgbench repeatedly in script
>> and collects and analyze the result.
>>
>> The one thing I concern is that this patch changes -l option so that
>> it requires argument.
>> But changing its behavior would be good rather than adding new option.
>>
>> Please give me feedback.
>
>
> Patch applies but does not compile, because "logfilename" is not declared.
> I guess "logfile" was meant instead.
>
> I understand and agree that in some case having only a predefined file
> prefix in the current directory as the only option can be a hindrance for
> scripts which use pgbench and rely on the log.
>
> I'm not at ease either with changing the behavior of such an option, as some
> people may be happy with it and some script may be using it. I would suggest
> not to do so.
>
> Moreover, what is provided is not a file name, but a prefix used to build
> file names.
>
> So I would suggest to:
>  - fix the compilation issue
>  - leave -l/--log as it is, i.e. use "pgbench_log" as a prefix
>  - add --log-prefix=... (long option only) for changing this prefix
>

Thank you for reviewing this patch!

I agree. It's better to add the separated option to specify the prefix
of log file instead of changing the existing behaviour. Attached
latest patch incorporated review comments.
Please review it.

Regards,

--
Masahiko Sawada
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center

Attachment

Re: Specifying the log file name of pgbench -l option

From
Fabien COELHO
Date:
Hello Masahiko,

>> So I would suggest to:
>>  - fix the compilation issue
>>  - leave -l/--log as it is, i.e. use "pgbench_log" as a prefix
>>  - add --log-prefix=... (long option only) for changing this prefix
>
> I agree. It's better to add the separated option to specify the prefix
> of log file instead of changing the existing behaviour. Attached
> latest patch incorporated review comments.
> Please review it.

Patch applies, compiles and works for me.

This new option is for benchmarking, so "benchmarking_option_set" should 
be set to true.

To simplify the code, I would suggest to initialize explicitely 
"logfile_prefix" to the default value which is then overriden when the 
option is set, which allows to get rid of the "prefix" variable later.

There is no reason to change the documentation by breaking a line at 
another place if the text is not changed (where ... with 1).

I would suggest to simplify a little bit the documentation:  "prefix of log file ..." ->  "default log file prefix that
canbe changed with <option>...</>"
 

Otherwise the explanations seem clear enough to me. If a native English 
speaker could check them though, it would be nice.

-- 
Fabien.



Re: Specifying the log file name of pgbench -l option

From
Beena Emerson
Date:
Hello,

On Wed, Nov 2, 2016 at 4:16 AM, Fabien COELHO <coelho@cri.ensmp.fr> wrote:

Hello Masahiko,

So I would suggest to:
 - fix the compilation issue
 - leave -l/--log as it is, i.e. use "pgbench_log" as a prefix
 - add --log-prefix=... (long option only) for changing this prefix

I agree. It's better to add the separated option to specify the prefix
of log file instead of changing the existing behaviour. Attached
latest patch incorporated review comments.
Please review it.

Patch applies, compiles and works for me.

It works for me as well.
 

This new option is for benchmarking, so "benchmarking_option_set" should be set to true.

To simplify the code, I would suggest to initialize explicitely "logfile_prefix" to the default value which is then overriden when the option is set, which allows to get rid of the "prefix" variable later.

There is no reason to change the documentation by breaking a line at another place if the text is not changed (where ... with 1).

I would suggest to simplify a little bit the documentation:
  "prefix of log file ..." ->
  "default log file prefix that can be changed with <option>...</>"

Otherwise the explanations seem clear enough to me. If a native English speaker could check them though, it would be nice.

For the explanation of the option --log-prefix, I feel it would be better to say "Custom prefix for transaction log file. Default is pgbench_log" 


-   <filename>pgbench_log.<replaceable>nnn</></filename>, where
+   <filename><replaceable>pgbench_log</replaceable>.<replaceable>nnn</></filename>,
+   where <replaceable>pgbench_log</replaceable> is the prefix of log file that can
+   be changed by specifying <option>--log-prefix</option>, and where

It could just say "the default prefix pgbench_log can be changed with option --log-prefix, and " we need not use where again. 

Also the error message is made similar to that of aggregate-interval but I think the one in sampling-rate is better:

$ pgbench --log-prefix=chk -t 20
log file prefix (--log-prefix) is allowed only when actually logging transactions

pgbench  --sampling-rate=1 -t 20
log sampling (--sampling-rate) is allowed only when logging transactions (-l)


Thanks,

Beena Emerson

Have a Great Day!

Re: Specifying the log file name of pgbench -l option

From
Masahiko Sawada
Date:
On Wed, Nov 2, 2016 at 3:57 PM, Beena Emerson <memissemerson@gmail.com> wrote:
> Hello,
>
> On Wed, Nov 2, 2016 at 4:16 AM, Fabien COELHO <coelho@cri.ensmp.fr> wrote:
>>
>>
>> Hello Masahiko,
>>
>>>> So I would suggest to:
>>>>  - fix the compilation issue
>>>>  - leave -l/--log as it is, i.e. use "pgbench_log" as a prefix
>>>>  - add --log-prefix=... (long option only) for changing this prefix
>>>
>>>
>>> I agree. It's better to add the separated option to specify the prefix
>>> of log file instead of changing the existing behaviour. Attached
>>> latest patch incorporated review comments.
>>> Please review it.
>>
>>
>> Patch applies, compiles and works for me.
>
>
> It works for me as well.
>
>>
>>
>> This new option is for benchmarking, so "benchmarking_option_set" should
>> be set to true.
>>
>> To simplify the code, I would suggest to initialize explicitely
>> "logfile_prefix" to the default value which is then overriden when the
>> option is set, which allows to get rid of the "prefix" variable later.
>>
>> There is no reason to change the documentation by breaking a line at
>> another place if the text is not changed (where ... with 1).
>>
>> I would suggest to simplify a little bit the documentation:
>>   "prefix of log file ..." ->
>>   "default log file prefix that can be changed with <option>...</>"
>>
>> Otherwise the explanations seem clear enough to me. If a native English
>> speaker could check them though, it would be nice.
>
>
> For the explanation of the option --log-prefix, I feel it would be better to
> say "Custom prefix for transaction log file. Default is pgbench_log"
>
>
> -   <filename>pgbench_log.<replaceable>nnn</></filename>, where
> +
> <filename><replaceable>pgbench_log</replaceable>.<replaceable>nnn</></filename>,
> +   where <replaceable>pgbench_log</replaceable> is the prefix of log file
> that can
> +   be changed by specifying <option>--log-prefix</option>, and where
>
> It could just say "the default prefix pgbench_log can be changed with option
> --log-prefix, and " we need not use where again.
>
> Also the error message is made similar to that of aggregate-interval but I
> think the one in sampling-rate is better:
>
> $ pgbench --log-prefix=chk -t 20
> log file prefix (--log-prefix) is allowed only when actually logging
> transactions
>
> pgbench  --sampling-rate=1 -t 20
> log sampling (--sampling-rate) is allowed only when logging transactions
> (-l)
>

Thank you for reviewing this patch!

Attached latest patch incorporated comments.
Please review it.

Regards,

--
Masahiko Sawada
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center

Attachment

Re: Specifying the log file name of pgbench -l option

From
Beena Emerson
Date:
Hello Sawada-san,

On Mon, Nov 7, 2016 at 4:47 PM, Masahiko Sawada <sawada.mshk@gmail.com> wrote:
On Wed, Nov 2, 2016 at 3:57 PM, Beena Emerson <memissemerson@gmail.com> wrote:
> Hello,
>
> On Wed, Nov 2, 2016 at 4:16 AM, Fabien COELHO <coelho@cri.ensmp.fr> wrote:
>>
>>
>> Hello Masahiko,
>>
>>>> So I would suggest to:
>>>>  - fix the compilation issue
>>>>  - leave -l/--log as it is, i.e. use "pgbench_log" as a prefix
>>>>  - add --log-prefix=... (long option only) for changing this prefix
>>>
>>>
>>> I agree. It's better to add the separated option to specify the prefix
>>> of log file instead of changing the existing behaviour. Attached
>>> latest patch incorporated review comments.
>>> Please review it.
>>
>>
>> Patch applies, compiles and works for me.
>
>
> It works for me as well.
>
>>
>>
>> This new option is for benchmarking, so "benchmarking_option_set" should
>> be set to true.
>>
>> To simplify the code, I would suggest to initialize explicitely
>> "logfile_prefix" to the default value which is then overriden when the
>> option is set, which allows to get rid of the "prefix" variable later.
>>
>> There is no reason to change the documentation by breaking a line at
>> another place if the text is not changed (where ... with 1).
>>
>> I would suggest to simplify a little bit the documentation:
>>   "prefix of log file ..." ->
>>   "default log file prefix that can be changed with <option>...</>"
>>
>> Otherwise the explanations seem clear enough to me. If a native English
>> speaker could check them though, it would be nice.
>
>
> For the explanation of the option --log-prefix, I feel it would be better to
> say "Custom prefix for transaction log file. Default is pgbench_log"
>
>
> -   <filename>pgbench_log.<replaceable>nnn</></filename>, where
> +
> <filename><replaceable>pgbench_log</replaceable>.<replaceable>nnn</></filename>,
> +   where <replaceable>pgbench_log</replaceable> is the prefix of log file
> that can
> +   be changed by specifying <option>--log-prefix</option>, and where
>
> It could just say "the default prefix pgbench_log can be changed with option
> --log-prefix, and " we need not use where again.
>
> Also the error message is made similar to that of aggregate-interval but I
> think the one in sampling-rate is better:
>
> $ pgbench --log-prefix=chk -t 20
> log file prefix (--log-prefix) is allowed only when actually logging
> transactions
>
> pgbench  --sampling-rate=1 -t 20
> log sampling (--sampling-rate) is allowed only when logging transactions
> (-l)
>

Thank you for reviewing this patch!

Attached latest patch incorporated comments.
Please review it.


Thank you for updating the patch.

I note that the current changes, break the behaviour when we do not use -l option.  

Since the log_prefix variable is set to default value, the check " if (!use_log && logfile_prefix)"  always returns true. This throws error even when we have not used the -l and --log-prefix option as shown below.

$ pgbench -T 50
log file prefix (--log-prefix) is allowed only when logging transactions (-l)

Kindly fix this. 

Thank you,

  
--

Beena Emerson

Have a Great Day!

Re: Specifying the log file name of pgbench -l option

From
Masahiko Sawada
Date:
On Mon, Nov 7, 2016 at 10:52 PM, Beena Emerson <memissemerson@gmail.com> wrote:
> Hello Sawada-san,
>
> On Mon, Nov 7, 2016 at 4:47 PM, Masahiko Sawada <sawada.mshk@gmail.com>
> wrote:
>>
>> On Wed, Nov 2, 2016 at 3:57 PM, Beena Emerson <memissemerson@gmail.com>
>> wrote:
>> > Hello,
>> >
>> > On Wed, Nov 2, 2016 at 4:16 AM, Fabien COELHO <coelho@cri.ensmp.fr>
>> > wrote:
>> >>
>> >>
>> >> Hello Masahiko,
>> >>
>> >>>> So I would suggest to:
>> >>>>  - fix the compilation issue
>> >>>>  - leave -l/--log as it is, i.e. use "pgbench_log" as a prefix
>> >>>>  - add --log-prefix=... (long option only) for changing this prefix
>> >>>
>> >>>
>> >>> I agree. It's better to add the separated option to specify the prefix
>> >>> of log file instead of changing the existing behaviour. Attached
>> >>> latest patch incorporated review comments.
>> >>> Please review it.
>> >>
>> >>
>> >> Patch applies, compiles and works for me.
>> >
>> >
>> > It works for me as well.
>> >
>> >>
>> >>
>> >> This new option is for benchmarking, so "benchmarking_option_set"
>> >> should
>> >> be set to true.
>> >>
>> >> To simplify the code, I would suggest to initialize explicitely
>> >> "logfile_prefix" to the default value which is then overriden when the
>> >> option is set, which allows to get rid of the "prefix" variable later.
>> >>
>> >> There is no reason to change the documentation by breaking a line at
>> >> another place if the text is not changed (where ... with 1).
>> >>
>> >> I would suggest to simplify a little bit the documentation:
>> >>   "prefix of log file ..." ->
>> >>   "default log file prefix that can be changed with <option>...</>"
>> >>
>> >> Otherwise the explanations seem clear enough to me. If a native English
>> >> speaker could check them though, it would be nice.
>> >
>> >
>> > For the explanation of the option --log-prefix, I feel it would be
>> > better to
>> > say "Custom prefix for transaction log file. Default is pgbench_log"
>> >
>> >
>> > -   <filename>pgbench_log.<replaceable>nnn</></filename>, where
>> > +
>> >
>> > <filename><replaceable>pgbench_log</replaceable>.<replaceable>nnn</></filename>,
>> > +   where <replaceable>pgbench_log</replaceable> is the prefix of log
>> > file
>> > that can
>> > +   be changed by specifying <option>--log-prefix</option>, and where
>> >
>> > It could just say "the default prefix pgbench_log can be changed with
>> > option
>> > --log-prefix, and " we need not use where again.
>> >
>> > Also the error message is made similar to that of aggregate-interval but
>> > I
>> > think the one in sampling-rate is better:
>> >
>> > $ pgbench --log-prefix=chk -t 20
>> > log file prefix (--log-prefix) is allowed only when actually logging
>> > transactions
>> >
>> > pgbench  --sampling-rate=1 -t 20
>> > log sampling (--sampling-rate) is allowed only when logging transactions
>> > (-l)
>> >
>>
>> Thank you for reviewing this patch!
>>
>> Attached latest patch incorporated comments.
>> Please review it.
>>
>
> Thank you for updating the patch.
>
> I note that the current changes, break the behaviour when we do not use -l
> option.
>
> Since the log_prefix variable is set to default value, the check " if
> (!use_log && logfile_prefix)"  always returns true. This throws error even
> when we have not used the -l and --log-prefix option as shown below.
>
> $ pgbench -T 50
> log file prefix (--log-prefix) is allowed only when logging transactions
> (-l)
>

Thanks.
Attached latest patch.
Please review it.

Regards,

--
Masahiko Sawada
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center

Attachment

Re: Specifying the log file name of pgbench -l option

From
Beena Emerson
Date:
Hello,

On Mon, Nov 7, 2016 at 8:42 PM, Masahiko Sawada <sawada.mshk@gmail.com> wrote:
On Mon, Nov 7, 2016 at 10:52 PM, Beena Emerson <memissemerson@gmail.com> wrote:
> Hello Sawada-san,
>
> On Mon, Nov 7, 2016 at 4:47 PM, Masahiko Sawada <sawada.mshk@gmail.com>
> wrote:
>>
>> On Wed, Nov 2, 2016 at 3:57 PM, Beena Emerson <memissemerson@gmail.com>
>> wrote:
>> > Hello,
>> >
>> > On Wed, Nov 2, 2016 at 4:16 AM, Fabien COELHO <coelho@cri.ensmp.fr>
>> > wrote:
>> >>
>> >>
>> >> Hello Masahiko,
>> >>
>> >>>> So I would suggest to:
>> >>>>  - fix the compilation issue
>> >>>>  - leave -l/--log as it is, i.e. use "pgbench_log" as a prefix
>> >>>>  - add --log-prefix=... (long option only) for changing this prefix
>> >>>
>> >>>
>> >>> I agree. It's better to add the separated option to specify the prefix
>> >>> of log file instead of changing the existing behaviour. Attached
>> >>> latest patch incorporated review comments.
>> >>> Please review it.
>> >>
>> >>
>> >> Patch applies, compiles and works for me.
>> >
>> >
>> > It works for me as well.
>> >
>> >>
>> >>
>> >> This new option is for benchmarking, so "benchmarking_option_set"
>> >> should
>> >> be set to true.
>> >>
>> >> To simplify the code, I would suggest to initialize explicitely
>> >> "logfile_prefix" to the default value which is then overriden when the
>> >> option is set, which allows to get rid of the "prefix" variable later.
>> >>
>> >> There is no reason to change the documentation by breaking a line at
>> >> another place if the text is not changed (where ... with 1).
>> >>
>> >> I would suggest to simplify a little bit the documentation:
>> >>   "prefix of log file ..." ->
>> >>   "default log file prefix that can be changed with <option>...</>"
>> >>
>> >> Otherwise the explanations seem clear enough to me. If a native English
>> >> speaker could check them though, it would be nice.
>> >
>> >
>> > For the explanation of the option --log-prefix, I feel it would be
>> > better to
>> > say "Custom prefix for transaction log file. Default is pgbench_log"
>> >
>> >
>> > -   <filename>pgbench_log.<replaceable>nnn</></filename>, where
>> > +
>> >
>> > <filename><replaceable>pgbench_log</replaceable>.<replaceable>nnn</></filename>,
>> > +   where <replaceable>pgbench_log</replaceable> is the prefix of log
>> > file
>> > that can
>> > +   be changed by specifying <option>--log-prefix</option>, and where
>> >
>> > It could just say "the default prefix pgbench_log can be changed with
>> > option
>> > --log-prefix, and " we need not use where again.
>> >
>> > Also the error message is made similar to that of aggregate-interval but
>> > I
>> > think the one in sampling-rate is better:
>> >
>> > $ pgbench --log-prefix=chk -t 20
>> > log file prefix (--log-prefix) is allowed only when actually logging
>> > transactions
>> >
>> > pgbench  --sampling-rate=1 -t 20
>> > log sampling (--sampling-rate) is allowed only when logging transactions
>> > (-l)
>> >
>>
>> Thank you for reviewing this patch!
>>
>> Attached latest patch incorporated comments.
>> Please review it.
>>
>
> Thank you for updating the patch.
>
> I note that the current changes, break the behaviour when we do not use -l
> option.
>
> Since the log_prefix variable is set to default value, the check " if
> (!use_log && logfile_prefix)"  always returns true. This throws error even
> when we have not used the -l and --log-prefix option as shown below.
>
> $ pgbench -T 50
> log file prefix (--log-prefix) is allowed only when logging transactions
> (-l)
>

Thanks.
Attached latest patch.
Please review it.

Thank you for the update.

I checked. It works as expected. I have no more comments.
If its okay with Fabien, we can mark it ready for committer. 

Thanks,
 
Beena Emerson

Have a Great Day!

Re: Specifying the log file name of pgbench -l option

From
Fabien COELHO
Date:
[ ... v4 ]

> I checked. It works as expected. I have no more comments.
> If its okay with Fabien, we can mark it ready for committer.

Patch applies, compiles (including the documentation), make check ok and 
features works for me. Code could be a little simpler, but it is okay. 
I've switched the status to "ready for committers".

-- 
Fabien.



Re: Specifying the log file name of pgbench -l option

From
Robert Haas
Date:
On Tue, Nov 8, 2016 at 4:04 AM, Fabien COELHO <coelho@cri.ensmp.fr> wrote:
>
> [ ... v4 ]
>
>> I checked. It works as expected. I have no more comments.
>> If its okay with Fabien, we can mark it ready for committer.
>
> Patch applies, compiles (including the documentation), make check ok and
> features works for me. Code could be a little simpler, but it is okay. I've
> switched the status to "ready for committers".

Committed with some wordsmithing and cosmetic tweaks.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company