Re: benchmark results comparing versions 15.2 and 16 - Mailing list pgsql-hackers

From Alexander Lakhin
Subject Re: benchmark results comparing versions 15.2 and 16
Date
Msg-id b32bed1b-0746-9b20-1472-4bdc9ca66d52@gmail.com
Whole thread Raw
In response to benchmark results comparing versions 15.2 and 16  (MARK CALLAGHAN <mdcallag@gmail.com>)
Responses Re: benchmark results comparing versions 15.2 and 16
Re: benchmark results comparing versions 15.2 and 16
List pgsql-hackers
Hello Mark,

05.05.2023 20:45, MARK CALLAGHAN wrote:
> This is mostly a hobby project for me - my other hobby is removing invasive weeds. I am happy to answer questions and

> run more tests, but turnaround for answers won't be instant. Getting results from Linux perf for these tests is on my

> TODO list. For now I am just re-running a subset of these to get more certainty that the regressions are real and not

> noise.
>

It's a very interesting topic to me, too. I had developed some scripts to
measure and compare postgres`s performance using miscellaneous public
benchmarks (ycsb, tpcds, benchmarksql_tpcc, htapbench, benchbase, gdprbench,
s64da-benchmark, ...). Having compared 15.3 (56e869a09) with master
(58f5edf84) I haven't seen significant regressions except a few minor ones.
First regression observed with a simple pgbench test:
pgbench -i benchdb
pgbench -c 10 -T 300 benchdb
(with default compilation options and fsync = off)

On master I get:
tps = 10349.826645 (without initial connection time)
On 15.3:
tps = 11296.064992 (without initial connection time)

This difference is confirmed by multiple test runs. `git bisect` for this
regression pointed at f193883fc.

Best regards,
Alexander



pgsql-hackers by date:

Previous
From: Robert Haas
Date:
Subject: Re: base backup vs. concurrent truncation
Next
From: Alvaro Herrera
Date:
Subject: Re: [PATCH] Allow Postgres to pick an unused port to listen