On Mon, Aug 08, 2022 at 02:50:17PM +0200, Yedil Serzhan wrote: > Dear hackers, > > I'm Yedil. I'm working on the project "Postgres Performance Farm" during > Gsoc. Pgperffarm is a project like Postgres build farm but focuses on the > performance of the database. Now it has 2 types of benchmarks, pgbench and > tpc-h. The website is online here <http://140.211.168.145/>, and the repo > is here <https://github.com/PGPerfFarm/pgperffarm_server>. > > I would like you to take a look at our website and, if possible, give some > feedback on, for example, what other data should be collected or what other > metrics could be used to compare performance.
Nice work!
We need to be careful with how results based on the TPC-H specification are presented. It needs to be changed, but maybe not dramatically. Something like "Fair use derivation of TPC-H". It needs to be clear that it's not an official TPC-H result.
I think I've hinted at it in the #perffarm slack channel, that I think it would be better if you leveraged one of the already existing TPC-H derived kits. While I'm partial to dbt-3, because I'm trying to maintain it and because it sounded like you were starting to do something similar to that, I think you can save a good amount of effort from reimplementing another kit from scratch.
Regards, Mark
It makes sense to put it as a "fair use derivation of TPC-H". I also used the term "composite score" because of your previous feedback on it.
I'll also check out the dbt-3 tool and if the effort is worth it, and if it's necessary, I'll try to switch to it.
These are very valuable feedback, thank you again.
