Re: Improving connection scalability: GetSnapshotData() - Mailing list pgsql-hackers

From Andres Freund
Subject Re: Improving connection scalability: GetSnapshotData()
Date
Msg-id 20200906185619.nde4ykyukqgrnrow@alap3.anarazel.de
Whole thread Raw
In response to Re: Improving connection scalability: GetSnapshotData()  (Konstantin Knizhnik <k.knizhnik@postgrespro.ru>)
Responses Re: Improving connection scalability: GetSnapshotData()  (Konstantin Knizhnik <k.knizhnik@postgrespro.ru>)
List pgsql-hackers
Hi,

On 2020-09-06 14:05:35 +0300, Konstantin Knizhnik wrote:
> On 04.09.2020 21:53, Andres Freund wrote:
> > 
> > > May be it is because of more complex architecture of my server?
> > Think we'll need profiles to know...
> 
> This is "perf top" of pgebch -c 100 -j 100 -M prepared -S
> 
>   12.16%  postgres                           [.] PinBuffer
>   11.92%  postgres                           [.] LWLockAttemptLock
>    6.46%  postgres                           [.] UnpinBuffer.constprop.11
>    6.03%  postgres                           [.] LWLockRelease
>    3.14%  postgres                           [.] BufferGetBlockNumber
>    3.04%  postgres                           [.] ReadBuffer_common
>    2.13%  [kernel]                           [k] _raw_spin_lock_irqsave
>    2.11%  [kernel]                           [k] switch_mm_irqs_off
>    1.95%  postgres                           [.] _bt_compare
> 
> 
> Looks like most of the time is pent in buffers locks.

Hm, that is interesting / odd. If you record a profile with call graphs
(e.g. --call-graph dwarf), where are all the LWLockAttemptLock calls
comming from?


I assume the machine you're talking about is an 8 socket machine?

What if you:
a) start postgres and pgbench with numactl --interleave=all
b) start postgres with numactl --interleave=0,1 --cpunodebind=0,1 --membind=0,1
   in case you have 4 sockets, or 0,1,2,3 in case you have 8 sockets?


> And which pgbench database scale factor you have used?

200

Another thing you could try is to run 2-4 pgench instances in different
databases.

Greetings,

Andres Freund



pgsql-hackers by date:

Previous
From: Andres Freund
Date:
Subject: Re: Improving connection scalability: GetSnapshotData()
Next
From: Peter Geoghegan
Date:
Subject: Re: v13: show extended stats target in \d