Re: Improving connection scalability: GetSnapshotData() - Mailing list pgsql-hackers

From Peter Geoghegan
Subject Re: Improving connection scalability: GetSnapshotData()
Date
Msg-id CAH2-WzmNpBm0vqOaxzJhd-31SZVU904xrZvWPp1RyGOEFZQTug@mail.gmail.com
Whole thread Raw
In response to Improving connection scalability: GetSnapshotData()  (Andres Freund <andres@anarazel.de>)
Responses Re: Improving connection scalability: GetSnapshotData()  (Bruce Momjian <bruce@momjian.us>)
Re: Improving connection scalability: GetSnapshotData()  (Andres Freund <andres@anarazel.de>)
Re: Improving connection scalability: GetSnapshotData()  (Alexander Korotkov <a.korotkov@postgrespro.ru>)
List pgsql-hackers
On Sun, Mar 1, 2020 at 12:36 AM Andres Freund <andres@anarazel.de> wrote:
> The workload is a pgbench readonly, with pgbench -M prepared -c $conns
> -j $conns -S -n for each client count.  This is on a machine with 2
> Intel(R) Xeon(R) Platinum 8168, but virtualized.
>
> conns   tps master              tps pgxact-split
>
> 1       26842.492845            26524.194821
> 10      246923.158682           249224.782661
> 50      695956.539704           709833.746374
> 100     1054727.043139          1903616.306028
> 200     964795.282957           1949200.338012
> 300     906029.377539           1927881.231478
> 400     845696.690912           1911065.369776
> 500     812295.222497           1926237.255856
> 600     888030.104213           1903047.236273
> 700     866896.532490           1886537.202142
> 800     863407.341506           1883768.592610
> 900     871386.608563           1874638.012128
> 1000    887668.277133           1876402.391502
> 1500    860051.361395           1815103.564241
> 2000    890900.098657           1775435.271018
> 3000    874184.980039           1653953.817997
> 4000    845023.080703           1582582.316043
> 5000    817100.195728           1512260.802371
>
> I think these are pretty nice results.

This scalability improvement is clearly very significant. There is
little question that this is a strategically important enhancement for
the Postgres project in general. I hope that you will ultimately be
able to commit the patchset before feature freeze.

I have heard quite a few complaints about the scalability of snapshot
acquisition in Postgres. Generally from very large users that are not
well represented on the mailing lists, for a variety of reasons. The
GetSnapshotData() bottleneck is a *huge* problem for us. (As problems
for Postgres users go, I would probably rank it second behind issues
with VACUUM.)

-- 
Peter Geoghegan



pgsql-hackers by date:

Previous
From: Tom Lane
Date:
Subject: Can we get rid of GetLocaleInfoEx() yet?
Next
From: Bruce Momjian
Date:
Subject: Re: Improving connection scalability: GetSnapshotData()