Thread: Asking for feedback on Pgperffarm
Dear hackers,
I'm Yedil. I'm working on the project "Postgres Performance Farm" during Gsoc. Pgperffarm is a project like Postgres build farm but focuses on the performance of the database. Now it has 2 types of benchmarks, pgbench and tpc-h. The website is online here, and the repo is here.
I would like you to take a look at our website and, if possible, give some feedback on, for example, what other data should be collected or what other metrics could be used to compare performance.
Thanks for your time in advance!
Best regards
Yedil
Hi Yedil, On Mon, Aug 08, 2022 at 02:50:17PM +0200, Yedil Serzhan wrote: > Dear hackers, > > I'm Yedil. I'm working on the project "Postgres Performance Farm" during > Gsoc. Pgperffarm is a project like Postgres build farm but focuses on the > performance of the database. Now it has 2 types of benchmarks, pgbench and > tpc-h. The website is online here <http://140.211.168.145/>, and the repo > is here <https://github.com/PGPerfFarm/pgperffarm_server>. > > I would like you to take a look at our website and, if possible, give some > feedback on, for example, what other data should be collected or what other > metrics could be used to compare performance. Nice work! We need to be careful with how results based on the TPC-H specification are presented. It needs to be changed, but maybe not dramatically. Something like "Fair use derivation of TPC-H". It needs to be clear that it's not an official TPC-H result. I think I've hinted at it in the #perffarm slack channel, that I think it would be better if you leveraged one of the already existing TPC-H derived kits. While I'm partial to dbt-3, because I'm trying to maintain it and because it sounded like you were starting to do something similar to that, I think you can save a good amount of effort from reimplementing another kit from scratch. Regards, Mark -- Mark Wong EDB https://enterprisedb.com
Hi, Mark, really thank you for your feedback.
On Mon, Aug 8, 2022 at 7:06 PM Mark Wong <markwkm@gmail.com> wrote:
Hi Yedil,
On Mon, Aug 08, 2022 at 02:50:17PM +0200, Yedil Serzhan wrote:
> Dear hackers,
>
> I'm Yedil. I'm working on the project "Postgres Performance Farm" during
> Gsoc. Pgperffarm is a project like Postgres build farm but focuses on the
> performance of the database. Now it has 2 types of benchmarks, pgbench and
> tpc-h. The website is online here <http://140.211.168.145/>, and the repo
> is here <https://github.com/PGPerfFarm/pgperffarm_server>.
>
> I would like you to take a look at our website and, if possible, give some
> feedback on, for example, what other data should be collected or what other
> metrics could be used to compare performance.
Nice work!
We need to be careful with how results based on the TPC-H specification
are presented. It needs to be changed, but maybe not dramatically.
Something like "Fair use derivation of TPC-H". It needs to be clear
that it's not an official TPC-H result.
I think I've hinted at it in the #perffarm slack channel, that I think
it would be better if you leveraged one of the already existing TPC-H
derived kits. While I'm partial to dbt-3, because I'm trying to
maintain it and because it sounded like you were starting to do
something similar to that, I think you can save a good amount of effort
from reimplementing another kit from scratch.
Regards,
Mark
It makes sense to put it as a "fair use derivation of TPC-H". I also used the term "composite score" because of your previous feedback on it.
I'll also check out the dbt-3 tool and if the effort is worth it, and if it's necessary, I'll try to switch to it.
These are very valuable feedback, thank you again.
These are very valuable feedback, thank you again.
Best,
Yedil