Re: few ideas for pgbench - Mailing list pgsql-hackers

From Pavel Stehule
Subject Re: few ideas for pgbench
Date
Msg-id CAFj8pRBCjRTsxHabme1R5A0ttTkgriXJf5hR_wj3qLYQ3cD=Ug@mail.gmail.com
Whole thread Raw
In response to Re: few ideas for pgbench  (Fabien COELHO <coelho@cri.ensmp.fr>)
List pgsql-hackers


pá 7. 5. 2021 v 11:28 odesílatel Fabien COELHO <coelho@cri.ensmp.fr> napsal:

>> Finally it is unclear how to add such a feature with minimal impact on the
>> source code.
>
>
> It is a question if this is possible without more changes or without
> compatibility break :( Probably not. All output should be centralized.

Yes and no.

For some things we could have "void report_sometype(file, name, data)"
which append "data," under csv but "name = data\n" under text, but this
does not work for nested data (eg -r -b/-f), which would rather require
some json/yaml/whatever format which can embed a hierarchy.

It can work with nested data too, but the result should be denormalized.



>> What I usually do is to put each pgbench run output in a separate file and
>> write a small shell/perl/python script to process these, possibly
>> generating CSV on the way.
>
> The goal of my proposal was a reduction of necessity to write auxiliary
> scripts. The produced document should not be "nice", but should be very
> easy to import it to some analytical tools.

Yes, I understood that. I tend to use CSV for that, import results in pg
or sqlite and analyse with SQL.

> There is an analogy with Postgres's CSV logs. It is the same. We can see
> the result of pgbench like some log.

Sure, but this works for simple flat data, not changing structures.

Denormalized tables are common.  Although it can be ugly, it should work.



--
Fabien.

pgsql-hackers by date:

Previous
From: "houzj.fnst@fujitsu.com"
Date:
Subject: RE: Allow batched insert during cross-partition updates
Next
From: Dilip Kumar
Date:
Subject: Re: Race condition in recovery?