Thread: pgbench - add minimal stats on initialization

pgbench - add minimal stats on initialization

From
Fabien COELHO
Date:
Hello devs,

The attached patch adds minimal stats during the initialization phase. 
Such a feature was already submitted by Doug Rady two years ago,

     https://commitfest.postgresql.org/15/1308/

but it needed to be adapted to the -I custom initialization approach 
developed in the same CF, and it ended 'returned with feedback'.

   sh> ./pgbench -i -s 3
   dropping old tables...
   creating tables...
   generating data...
   100000 of 300000 tuples (33%) done (elapsed 0.09 s, remaining 0.18 s)
   200000 of 300000 tuples (66%) done (elapsed 0.20 s, remaining 0.10 s)
   300000 of 300000 tuples (100%) done (elapsed 0.32 s, remaining 0.00 s)
   vacuuming...
   creating primary keys...
   done in 0.68 s (drop 0.06 s, create table 0.02 s, generate 0.34 s, vacuum 0.13 s, primary keys 0.13 s).

See the durations on the last line.

The intent is to test the initialization phase more precisely, and 
possibly accelerate it. For instance, is it better to do vacuum before or 
after primary keys?

-- 
Fabien.
Attachment

Re: pgbench - add minimal stats on initialization

From
Fabien COELHO
Date:
>  done in 0.68 s (drop 0.06 s, create table 0.02 s, generate 0.34 s, vacuum 
> 0.13 s, primary keys 0.13 s).
>
> See the durations on the last line.

It's even better with working TAP tests.

-- 
Fabien.
Attachment

Re: pgbench - add minimal stats on initialization

From
Ibrar Ahmed
Date:
The following review has been posted through the commitfest application:
make installcheck-world:  tested, failed
Implements feature:       tested, failed
Spec compliant:           tested, failed
Documentation:            not tested

Patch works perfectly and the code is well-written. I have one minor observation that in case of initDropTables you log
"drop"and in case of initCreateTables you log "create table". I think you need to be consistent. And why not "drop
tables"and "create tables" 

The new status of this patch is: Waiting on Author

Re: pgbench - add minimal stats on initialization

From
Ibrar Ahmed
Date:
The following review has been posted through the commitfest application:
make installcheck-world:  tested, passed
Implements feature:       tested, passed
Spec compliant:           tested, passed
Documentation:            not tested

Please ignore the last email.

Patch works perfectly and the code is well-written. I have one minor observation that in case of initDropTables you log
"drop"and in case of initCreateTables you log "create table". I think you need to be consistent. And why not "drop
tables"and "create tables"
 

The new status of this patch is: Waiting on Author

Re: pgbench - add minimal stats on initialization

From
Ibrar Ahmed
Date:
Hi Fabien,

I have one minor observation that in case of initDropTables you log
'drop' and in case of initCreateTables you log 'create table'. We need
to be consistent. The "drop tables" and "create tables" are the best
fit here. Otherwise, the patch is good.

On Wed, Apr 10, 2019 at 2:18 PM Ibrar Ahmed <ibrar.ahmad@gmail.com> wrote:
>
> The following review has been posted through the commitfest application:
> make installcheck-world:  tested, passed
> Implements feature:       tested, passed
> Spec compliant:           tested, passed
> Documentation:            not tested
>
> Please ignore the last email.
>
> Patch works perfectly and the code is well-written. I have one minor observation that in case of initDropTables you
log"drop" and in case of initCreateTables you log "create table". I think you need to be consistent. And why not "drop
tables"and "create tables" 
>
> The new status of this patch is: Waiting on Author



--
Ibrar Ahmed



Re: pgbench - add minimal stats on initialization

From
Fabien COELHO
Date:
Hello,

Thanks for the feedback.

> I have one minor observation that in case of initDropTables you log
> 'drop' and in case of initCreateTables you log 'create table'. We need
> to be consistent. The "drop tables" and "create tables" are the best
> fit here.

Ok.

Attached version does that, plus avoids re-assigning "first" on each loop, 
plus checks that --no-vacuum indeed removes all vacuums in the TAP test.

-- 
Fabien.
Attachment

Re: pgbench - add minimal stats on initialization

From
Ibrar Ahmed
Date:
The following review has been posted through the commitfest application:
make installcheck-world:  tested, passed
Implements feature:       tested, passed
Spec compliant:           tested, passed
Documentation:            not tested

Patch works fine on my machine.

The new status of this patch is: Ready for Committer

Re: pgbench - add minimal stats on initialization

From
Thomas Munro
Date:
On Fri, Apr 12, 2019 at 12:44 AM Ibrar Ahmed <ibrar.ahmad@gmail.com> wrote:
> The following review has been posted through the commitfest application:
> make installcheck-world:  tested, passed
> Implements feature:       tested, passed
> Spec compliant:           tested, passed
> Documentation:            not tested
>
> Patch works fine on my machine.
>
> The new status of this patch is: Ready for Committer

I spotted one typo, a comma where a semi-colon was wanted:

+                               op = "generate",
                                initGenerateData(con);
                                break;

I fixed that, ran it through pgindent and committed.  Thanks for the
patch and review!

-- 
Thomas Munro
https://enterprisedb.com