Thread: PostgresSQL 13.0 Beta 1 on Phoronix
Hi,
There was news in Phoronix about the Beta 1 Release of Postgres (1).
Unfortunately for Postgres advocacy it does not bring good news,
Unfortunately for Postgres advocacy it does not bring good news,
it is showing regressions in the benchmarks compared to version 12.
Without going into the technical merits of how the test was done,
Without going into the technical merits of how the test was done,
they have no way of knowing whether such regressions actually exist or if it is a failure of how the tests were done.
But it would be nice to have arguments to counter, for the sake of Postgres' promotion.
I'm using the development version (latest), and for now, it seems to be faster than version 12.
But it would be nice to have arguments to counter, for the sake of Postgres' promotion.
I'm using the development version (latest), and for now, it seems to be faster than version 12.
regards,
Ranier Vilela
On Sun, May 24, 2020 at 2:52 AM Ranier Vilela <ranier.vf@gmail.com> wrote: > There was news in Phoronix about the Beta 1 Release of Postgres (1). > Unfortunately for Postgres advocacy it does not bring good news, > it is showing regressions in the benchmarks compared to version 12. > Without going into the technical merits of how the test was done, > they have no way of knowing whether such regressions actually exist or if it is a failure of how the tests were done. This shellscript appears to be used by Phoronix to run pgbench: https://github.com/phoronix-test-suite/phoronix-test-suite/blob/f0f8c726f2700faea363f176a4b28dab026d45d0/ob-cache/test-profiles/pts/pgbench-1.8.4/install.sh It looks like they're only running pgbench for 60 second runs in all configurations -- notice that "-T 60" is passed to pgbench. I'm not entirely sure that that's all that there is to it. Still, there isn't any real attempt to make it clear what's going on here. I have my doubts about how representative these numbers are for that reason. -- Peter Geoghegan
Em dom., 24 de mai. de 2020 às 14:34, Peter Geoghegan <pg@bowt.ie> escreveu:
On Sun, May 24, 2020 at 2:52 AM Ranier Vilela <ranier.vf@gmail.com> wrote:
> There was news in Phoronix about the Beta 1 Release of Postgres (1).
> Unfortunately for Postgres advocacy it does not bring good news,
> it is showing regressions in the benchmarks compared to version 12.
> Without going into the technical merits of how the test was done,
> they have no way of knowing whether such regressions actually exist or if it is a failure of how the tests were done.
This shellscript appears to be used by Phoronix to run pgbench:
https://github.com/phoronix-test-suite/phoronix-test-suite/blob/f0f8c726f2700faea363f176a4b28dab026d45d0/ob-cache/test-profiles/pts/pgbench-1.8.4/install.sh
It looks like they're only running pgbench for 60 second runs in all
configurations -- notice that "-T 60" is passed to pgbench. I'm not
entirely sure that that's all that there is to it. Still, there isn't
any real attempt to make it clear what's going on here. I have my
doubts about how representative these numbers are for that reason.
I also find it very suspicious.V12 seems to be better at read-only workloads (at least it shows the graphics).
I'm using V13, in normal mode (read / write), medium load.
I'm using V13, in normal mode (read / write), medium load.
regards,
Ranier VIlela
On Sun, May 24, 2020 at 02:50:08PM -0300, Ranier Vilela wrote: > Em dom., 24 de mai. de 2020 às 14:34, Peter Geoghegan <pg@bowt.ie> escreveu: >> It looks like they're only running pgbench for 60 second runs in all >> configurations -- notice that "-T 60" is passed to pgbench. I'm not >> entirely sure that that's all that there is to it. Still, there isn't >> any real attempt to make it clear what's going on here. I have my >> doubts about how representative these numbers are for that reason. > > I also find it very suspicious. I don't know, but what seems pretty clear to me is this benchmark does zero customization of postgresql.conf (it disables autovacuum!?), and that the number of connections is calculated based on the number of cores while the scaling factor is visibly calculated from the amount of memory available in the environment. Perhaps the first part is wanted, but we are very conservative to allow PG to work on small-ish machines with the default configuration, and a 56-core machine with 378GB of memory is not something I would define as small-ish. -- Michael
Attachment
Em seg., 25 de mai. de 2020 às 03:57, Michael Paquier <michael@paquier.xyz> escreveu:
On Sun, May 24, 2020 at 02:50:08PM -0300, Ranier Vilela wrote:
> Em dom., 24 de mai. de 2020 às 14:34, Peter Geoghegan <pg@bowt.ie> escreveu:
>> It looks like they're only running pgbench for 60 second runs in all
>> configurations -- notice that "-T 60" is passed to pgbench. I'm not
>> entirely sure that that's all that there is to it. Still, there isn't
>> any real attempt to make it clear what's going on here. I have my
>> doubts about how representative these numbers are for that reason.
>
> I also find it very suspicious.
I don't know, but what seems pretty clear to me is this benchmark does
zero customization of postgresql.conf (it disables autovacuum!?), and
that the number of connections is calculated based on the number of
cores while the scaling factor is visibly calculated from the amount
of memory available in the environment. Perhaps the first part is
wanted, but we are very conservative to allow PG to work on small-ish
machines with the default configuration, and a 56-core machine with
378GB of memory is not something I would define as small-ish.
Does this mean that V13 would need additional settings in postgresql.conf,
to perform better than V12, out of the box?
If there is any new feature in V13 that needs some configuration in postgresql.conf,
If there is any new feature in V13 that needs some configuration in postgresql.conf,
Would be bettert it should be documented or configured in the installation itself,
to avoid this type of misunderstanding, which harms the perception of Postgres.
regards,
Ranier Vilela