Thread: pgbench - add minimal stats on initialization

Hello devs,

The attached patch adds minimal stats during the initialization phase. 
Such a feature was already submitted by Doug Rady two years ago,

     https://commitfest.postgresql.org/15/1308/

but it needed to be adapted to the -I custom initialization approach 
developed in the same CF, and it ended 'returned with feedback'.

   sh> ./pgbench -i -s 3
   dropping old tables...
   creating tables...
   generating data...
   100000 of 300000 tuples (33%) done (elapsed 0.09 s, remaining 0.18 s)
   200000 of 300000 tuples (66%) done (elapsed 0.20 s, remaining 0.10 s)
   300000 of 300000 tuples (100%) done (elapsed 0.32 s, remaining 0.00 s)
   vacuuming...
   creating primary keys...
   done in 0.68 s (drop 0.06 s, create table 0.02 s, generate 0.34 s, vacuum 0.13 s, primary keys 0.13 s).

See the durations on the last line.

The intent is to test the initialization phase more precisely, and 
possibly accelerate it. For instance, is it better to do vacuum before or 
after primary keys?

-- 
Fabien.

>  done in 0.68 s (drop 0.06 s, create table 0.02 s, generate 0.34 s, vacuum 
> 0.13 s, primary keys 0.13 s).
>
> See the durations on the last line.

It's even better with working TAP tests.

-- 
Fabien.

The following review has been posted through the commitfest application:
make installcheck-world:  tested, failed
Implements feature:       tested, failed
Spec compliant:           tested, failed
Documentation:            not tested

Patch works perfectly and the code is well-written. I have one minor observation that in case of initDropTables you log
"drop"and in case of initCreateTables you log "create table". I think you need to be consistent. And why not "drop
tables"and "create tables" 

The new status of this patch is: Waiting on Author

The following review has been posted through the commitfest application:
make installcheck-world:  tested, passed
Implements feature:       tested, passed
Spec compliant:           tested, passed
Documentation:            not tested

Please ignore the last email.

Patch works perfectly and the code is well-written. I have one minor observation that in case of initDropTables you log
"drop"and in case of initCreateTables you log "create table". I think you need to be consistent. And why not "drop
tables"and "create tables"
 

The new status of this patch is: Waiting on Author

Hi Fabien,

I have one minor observation that in case of initDropTables you log
'drop' and in case of initCreateTables you log 'create table'. We need
to be consistent. The "drop tables" and "create tables" are the best
fit here. Otherwise, the patch is good.

On Wed, Apr 10, 2019 at 2:18 PM Ibrar Ahmed <ibrar.ahmad@gmail.com> wrote:
>
> The following review has been posted through the commitfest application:
> make installcheck-world:  tested, passed
> Implements feature:       tested, passed
> Spec compliant:           tested, passed
> Documentation:            not tested
>
> Please ignore the last email.
>
> Patch works perfectly and the code is well-written. I have one minor observation that in case of initDropTables you
log"drop" and in case of initCreateTables you log "create table". I think you need to be consistent. And why not "drop
tables"and "create tables" 
>
> The new status of this patch is: Waiting on Author



--
Ibrar Ahmed

Hello,

Thanks for the feedback.

> I have one minor observation that in case of initDropTables you log
> 'drop' and in case of initCreateTables you log 'create table'. We need
> to be consistent. The "drop tables" and "create tables" are the best
> fit here.

Ok.

Attached version does that, plus avoids re-assigning "first" on each loop, 
plus checks that --no-vacuum indeed removes all vacuums in the TAP test.

-- 
Fabien.

The following review has been posted through the commitfest application:
make installcheck-world:  tested, passed
Implements feature:       tested, passed
Spec compliant:           tested, passed
Documentation:            not tested

Patch works fine on my machine.

The new status of this patch is: Ready for Committer

On Fri, Apr 12, 2019 at 12:44 AM Ibrar Ahmed <ibrar.ahmad@gmail.com> wrote:
> The following review has been posted through the commitfest application:
> make installcheck-world:  tested, passed
> Implements feature:       tested, passed
> Spec compliant:           tested, passed
> Documentation:            not tested
>
> Patch works fine on my machine.
>
> The new status of this patch is: Ready for Committer

I spotted one typo, a comma where a semi-colon was wanted:

+                               op = "generate",
                                initGenerateData(con);
                                break;

I fixed that, ran it through pgindent and committed.  Thanks for the
patch and review!

-- 
Thomas Munro
https://enterprisedb.com