Re: pgbench logging broken by time logic changes - Mailing list pgsql-hackers
| From | Gregory Smith |
|---|---|
| Subject | Re: pgbench logging broken by time logic changes |
| Date | |
| Msg-id | CAHLJuCWWL5dYO+eK+ii9LyUn4BfctsRM35Agxfve-JVGprh+Qg@mail.gmail.com Whole thread Raw |
| In response to | Re: pgbench logging broken by time logic changes (Fabien COELHO <coelho@cri.ensmp.fr>) |
| Responses |
Re: pgbench logging broken by time logic changes
|
| List | pgsql-hackers |
On Wed, Jun 16, 2021 at 2:59 PM Fabien COELHO <coelho@cri.ensmp.fr> wrote:
I'm unhappy because I already added tap tests for time-sensitive features
(-T and others, maybe logging aggregates, cannot remember), which have
been removed because they could fail under some circonstances (eg very
very very very slow hosts), or required some special handling (a few lines
of code) in pgbench, and the net result of this is there is not a single
test in place for some features:-(
I understand your struggle and I hope I was clear about two things:
-Doing better in this messy area takes a team that goes from development to release management, and I had no right to complain unless I brought resources to improve in the specific areas of the process that I want to be better.
I think the only thing you and I disagree on is that you see a "first issue in a corner case" where I see a process failure that is absolutely vital for me to improve. Since the reality is that I might be the best positioned person to actually move said process forward in a meaningful long-term way, I have every intention of applying pressure to the area you're frustrated at. Crunchy has a whole parallel review team to the community one now focused on what our corporate and government customers need for software process control and procedure compliance. The primary business problem I'm working on now is how to include performance review in that mix.
I already know I need to re-engage with you over how I need min/max numbers in the aggregate logging output to accomplish some valuable goals. When I get around to that this summer, I'd really enjoy talking with you a bit, video call or something, about really any community topic you're frustrated with. I have a lot riding now on the productivity of the PostgreSQL hacker community and I want everyone to succeed at the best goals.
There is no problem with proposing tests, the problem is that they are
accepted, or if they are accepted then that they are not removed at the
first small issue but rather fixed, or their limitations accepted, because
testing time-sensitive features is not as simple as testing functional
features.
For 2020 Crunchy gave me a sort of sabbatical year to research community oriented benchmarking topics. Having a self contained project in my home turned out to be the perfect way to spend *that* wreck of a year.
I made significant progress toward the idea of having a performance farm for PostgreSQL. On my laptop today is a 14GB database with 1s resolution latency traces for 663 days of pgbench time running 4 workloads across a small bare metal farm of various operating systems and hardware classes. I can answer questions like "how long does a typical SSD take to execute an INSERT commit?" across my farm with SQL. It's at the "works for me!" stage of development, and I thought this was the right time in the development cycle to start sharing improvement ideas from my work; thus the other submissions in progress I alluded to.
The logging feature is in an intermediate spot where validating it requires light custom tooling that compares its output against known variables like the system time. It doesn't quite have a performance component to it. Since this time logic detail is a well known portability minefield, I thought demanding that particular test was a pretty easy sell.
That you in particular are frustrated here makes perfect sense to me. I am fresh and ready to carry this forward some distance, and I hope the outcome makes you happy
pgsql-hackers by date: