Re: Built-in connection pooler - Mailing list pgsql-hackers

From Ryan Lambert
Subject Re: Built-in connection pooler
Date
Msg-id CAN-V+g9gTUhR_cxBr2yjc3MMbpk+kopwbbiHB6kRe2whfjFYnQ@mail.gmail.com
Whole thread Raw
In response to Re: Built-in connection pooler  (Konstantin Knizhnik <k.knizhnik@postgrespro.ru>)
Responses Re: Built-in connection pooler  (Konstantin Knizhnik <k.knizhnik@postgrespro.ru>)
Re: Built-in connection pooler  (Konstantin Knizhnik <k.knizhnik@postgrespro.ru>)
List pgsql-hackers
> I attached new version of the patch with fixed indentation problems and 
> Win32 specific fixes.  
 
Great, this latest patch applies cleanly to master.  installcheck world still passes.

> "connections_proxies" is used mostly to toggle connection pooling.
> Using more than 1 proxy is be needed only for huge workloads (hundreds 
> connections). 

My testing showed using only one proxy performing very poorly vs not using the pooler, even at 300 connections, with -3% TPS.  At lower numbers of connections it was much worse than other configurations I tried.  I just shared my full pgbench results [1], the "No Pool" and "# Proxies 2" data is what I used to generate the charts I previously shared.  The 1 proxy and 10 proxy data I had referred to but hadn't shared the results, sorry about that.

> And "session_pool_size" is core parameter  which determine efficiency of 
> pooling.
> The main trouble with it now, is that it is per database/user 
>  combination. Each such combination will have its own connection pool.
>  Choosing optimal value of pooler backends is non-trivial task. It 
>  certainly depends on number of available CPU cores.
>  But if backends and mostly disk-bounded, then optimal number of pooler 
>  worker can be large than number of cores.

I will do more testing around this variable next.  It seems that increasing session_pool_size for connection_proxies = 1 might help and leaving it at its default was my problem.

> PgPRO EE version of connection pooler has "idle_pool_worker_timeout" 
> parameter which allows to terminate idle workers.

+1

>  It is possible to implement it also for vanilla version of pooler. But 
>  primary intention of this patch was to minimize changes in Postgres core

Understood.

I attached a patch to apply after your latest patch [2] with my suggested changes to the docs.  I tried to make things read smoother without altering your meaning.  I don't think the connection pooler chapter fits in The SQL Language section, it seems more like Server Admin functionality so I moved it to follow the chapter on HA, load balancing and replication.  That made more sense to me looking at the overall ToC of the docs.

Thanks,

 
Ryan


Attachment

pgsql-hackers by date:

Previous
From: Fabien COELHO
Date:
Subject: Re: psql FETCH_COUNT feature does not work with combined queries
Next
From: Jehan-Guillaume de Rorthais
Date:
Subject: Re: Fetching timeline during recovery