PATCH: pgbench - merging transaction logs - Mailing list pgsql-hackers

From Tomas Vondra
Subject PATCH: pgbench - merging transaction logs
Date
Msg-id 54FCB11B.5050907@2ndquadrant.com
Whole thread Raw
Responses Re: PATCH: pgbench - merging transaction logs  (Fabien COELHO <coelho@cri.ensmp.fr>)
List pgsql-hackers
Hi there,

attached is a patch implementing merging of pgbench logs. These logs are
written by each thread, so with N threads you get N files with names

    pgbench_log.PID
    pgbench_log.PID.1
    ...
    pgbench_log.PID.N

Before analyzing these logs, these files need to be combined. I usually
ended up wrinting ad-hoc scripts doing that, lost them, written them
again and so on over and over again.

The other disadvantage of the external scripts is that you have to pass
all the info about the logs (whether the logs are aggregated, whther
there's throttling, etc.).

So integrating this into pgbench directly seems like a better approach,
and the attached patch implements that.

With '-m' or '--merge-logs' on the command-line, the logs are merged at
the end, using a simple 2-way merge to keep the log sorted by the time
field. It should work with all the other options that influence the log
format (--rate, --aggregate-interval and --latency-limit).

I'll add this to CF 2016-06.

--
Tomas Vondra                http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Attachment

pgsql-hackers by date:

Previous
From: Josh Berkus
Date:
Subject: Re: MD5 authentication needs help -SCRAM
Next
From: Tomas Vondra
Date:
Subject: PATCH: pgbench - logging aggregated info and transactions at the same time