Thread: Add pgbench option: CHECKPOINT before starting benchmark
Hi, I add checkpoint option to pgbench. pgbench is simple and useful benchmark for every user. However, result of benchmark greatly changes by some situations which are in executing checkpoint, number of dirty buffers in share_buffers, and so on. For such a problem, it is custom to carry out a checkpoint before starting benchmark. But it is a fact that the making of the script takes time, like under following script. psql -U postgres -d pgbench -p5432 -c "CHECKPOINT" pgbench -T 600 -c 12 -j4 -U postgres -d pgbench -p 5432 However, this script have a problem. This script execute "CHECKPOINT" -> "VACUUM" -> "starting benchmark". If relpages have lot of dirty pages, VACUUM generate dirty buffers on shared_buffers, and it will cause bad heavily checkpoint. I think pgbench would be more easy and accuracy benchmark tools for everyone. So I set checkpoint before starting benchmark. This patch's output is here. ----------------------------------------------------- [mitsu-ko@localhost pgbench]$ ./pgbench starting vacuum...end. starting checkpoint...end. transaction type: TPC-B (sort of) scaling factor: 1 query mode: simple number of clients: 1 number of threads: 1 number of transactions per client: 10 number of transactions actually processed: 10/10 tps = 312.851958 (including connections establishing) tps = 364.524478 (excluding connections establishing) ----------------------------------------------------- It execute "VACUUM" -> "CHECKPOINT" -> "starting benchmark". I think it is ideal setting for more accuracy benchmark. My patches option difinition is here. [mitsu-ko@localhost pgbench]$ ./pgbench --help ~ -N, --no-checkpoint do not run CHECKPOINT after initialization ~ In latest commited pgbench, -N is "--skip-some-updates skip updates of pgbench_tellers and pgbench_branches". But I cannot understand why -N is this option, so I set this option -u, and -N is "do not run CHECKPOINT option". What do you think? -- Mitsumasa KONDO NTT Open Source Software Center
Attachment
On 30/08/13 19:54, KONDO Mitsumasa wrote: > Hi, > > I add checkpoint option to pgbench. > > pgbench is simple and useful benchmark for every user. However, result of > benchmark greatly changes by some situations which are in executing checkpoint, > number of dirty buffers in share_buffers, and so on. For such a problem, it is > custom to carry out a checkpoint before starting benchmark. But it is a fact that > the making of the script takes time, like under following script. > > psql -U postgres -d pgbench -p5432 -c "CHECKPOINT" > pgbench -T 600 -c 12 -j4 -U postgres -d pgbench -p 5432 > > However, this script have a problem. > This script execute "CHECKPOINT" -> "VACUUM" -> "starting benchmark". > If relpages have lot of dirty pages, VACUUM generate dirty buffers on > shared_buffers, and it will cause bad heavily checkpoint. > > I think pgbench would be more easy and accuracy benchmark tools for everyone. So > I set checkpoint before starting benchmark. > > This patch's output is here. > ----------------------------------------------------- > [mitsu-ko@localhost pgbench]$ ./pgbench > starting vacuum...end. > starting checkpoint...end. > transaction type: TPC-B (sort of) > scaling factor: 1 > query mode: simple > number of clients: 1 > number of threads: 1 > number of transactions per client: 10 > number of transactions actually processed: 10/10 > tps = 312.851958 (including connections establishing) > tps = 364.524478 (excluding connections establishing) > ----------------------------------------------------- > It execute "VACUUM" -> "CHECKPOINT" -> "starting benchmark". > I think it is ideal setting for more accuracy benchmark. > > My patches option difinition is here. > [mitsu-ko@localhost pgbench]$ ./pgbench --help > ~ > -N, --no-checkpoint do not run CHECKPOINT after initialization > ~ > In latest commited pgbench, -N is "--skip-some-updates skip updates of > pgbench_tellers and pgbench_branches". But I cannot understand why -N is this > option, so I set this option -u, and -N is "do not run CHECKPOINT option". > > What do you think? > +1 I have been using a script to add CHECKPOINT before pgbench runs for ages...adding the option to pgbench is a great idea (wish I had thought of it)! Regards Mark
> My patches option difinition is here. > [mitsu-ko@localhost pgbench]$ ./pgbench --help > ~ > -N, --no-checkpoint do not run CHECKPOINT after initialization > ~ > In latest commited pgbench, -N is "--skip-some-updates skip updates of > pgbench_tellers and pgbench_branches". But I cannot understand why -N is this > option, so I set this option -u, and -N is "do not run CHECKPOINT option". > > What do you think? Although I agree that the -N is not really meaningful, ISTM that changing option names is to be avoided if people are used to it. In this instance, I'm not sure that many people use this option, so maybe this is not an issue. The patch should also update the sgml documentation. -- Fabien.
KONDO Mitsumasa <kondo.mitsumasa@lab.ntt.co.jp> writes:
> My patches option difinition is here.
> [mitsu-ko@localhost pgbench]$ ./pgbench --help
> ~
> -N, --no-checkpoint do not run CHECKPOINT after initialization
> ~
> In latest commited pgbench, -N is "--skip-some-updates skip updates of
> pgbench_tellers and pgbench_branches". But I cannot understand why -N is this
> option, so I set this option -u, and -N is "do not run CHECKPOINT option".
> What do you think?
I think it's a bad idea to change the meaning of a pre-existing option
letter, and a worse idea to change pgbench's default behavior. Instead,
have the option be "--checkpoint do a CHECKPOINT after initialization".
It doesn't look like there's any free single-letter option that goes
nicely with "checkpoint" either, but that's life.
regards, tom lane