Re: Improving connection scalability: GetSnapshotData() - Mailing list pgsql-hackers

From Andres Freund
Subject Re: Improving connection scalability: GetSnapshotData()
Date
Msg-id 20200906185214.fyujaew3mc3enqb5@alap3.anarazel.de
Whole thread Raw
In response to Re: Improving connection scalability: GetSnapshotData()  (Konstantin Knizhnik <k.knizhnik@postgrespro.ru>)
Responses Re: Improving connection scalability: GetSnapshotData()  (Konstantin Knizhnik <k.knizhnik@postgrespro.ru>)
List pgsql-hackers
Hi,

On 2020-09-05 16:58:31 +0300, Konstantin Knizhnik wrote:
> On 04.09.2020 21:53, Andres Freund wrote:
> > 
> > I also used huge_pages=on / configured them on the OS level. Otherwise
> > TLB misses will be a significant factor.
> 
> As far as I understand there should not be no any TLB misses because size of
> the shared buffers (8Mb) as several order of magnitude smaler that available
> physical memory.

I assume you didn't mean 8MB but 8GB? If so, that's way large enough to
be bigger than the TLB, particularly across processes (IIRC there's no
optimization to keep shared mappings de-duplicated between processes
from the view of the TLB).


> Yes, there is also noticeable difference in my case
> 
> | Idle Connections | Active Connections | TPS pre | TPS post |
> |-----------------:|-------------------:|--------:|---------:|
> |             5000 |                 48 |  758914 |  1184085 |

Sounds like you're somehow hitting another bottleneck around 1.2M TPS

Greetings,

Andres Freund



pgsql-hackers by date:

Previous
From: "Andrey M. Borodin"
Date:
Subject: Re: Yet another fast GiST build (typo)
Next
From: Andres Freund
Date:
Subject: Re: Improving connection scalability: GetSnapshotData()