Re: Built-in connection pooling - Mailing list pgsql-hackers

On 01/19/2018 07:35 PM, Claudio Freire wrote:
> 
> 
> On Fri, Jan 19, 2018 at 2:22 PM, Tomas Vondra
> <tomas.vondra@2ndquadrant.com <mailto:tomas.vondra@2ndquadrant.com>> wrote:
> 
> 
> 
>     On 01/19/2018 06:13 PM, Claudio Freire wrote:
>     >
>     >
>     > On Fri, Jan 19, 2018 at 2:07 PM, Konstantin Knizhnik
>     > <k.knizhnik@postgrespro.ru <mailto:k.knizhnik@postgrespro.ru>
>     <mailto:k.knizhnik@postgrespro.ru
>     <mailto:k.knizhnik@postgrespro.ru>>> wrote:
>     >
>     >
>     >
>     >
>     >         Well, I haven't said it has to be single-threaded like
>     pgbouncer. I
>     >         don't see why the bgworker could not use multiple threads
>     >         internally (of
>     >         course, it'd need to be not to mess the stuff that is not
>     >         thread-safe).
>     >
>     >
>     >     Certainly architecture with N multiple scheduling bgworkers and M
>     >     executors (backends) may be more flexible
>     >     than solution when scheduling is done in executor itself. But we
>     >     will have to pay extra cost for redirection.
>     >     I am not sure that finally it will allow to reach better
>     performance.
>     >     More flexible solution in many cases doesn't mean more efficient
>     >     solution.
>     >
>     >
>     > I think you can take the best of both worlds.
>     >
>     > You can take your approach of passing around fds, and build a "load
>     > balancing protocol" in a bgworker.
>     >
>     > The postmaster sends the socket to the bgworker, the bgworker
>     waits for
>     > a command as pgbouncer does, but instead of proxying everything, when
>     > commands arrive, it passes the socket to a backend to handle.
>     >
>     > That way, the bgworker can do what pgbouncer does, handle different
>     > pooling modes, match backends to databases, etc, but it doesn't
>     have to
>     > proxy all data, it just delegates handling of a command to a backend,
>     > and forgets about that socket.
>     >
>     > Sounds like it could work.
>     >
> 
>     How could it do all that without actually processing all the data? For
>     example, how could it determine the statement/transaction boundaries?
> 
> 
> It only needs to determine statement/transaction start.
> 
> After that, it hands off the connection to a backend, and the
> backend determines when to give it back.
> 
> So instead of processing all the data, it only processes a tiny part of it.
> 

How exactly would the backend "give back" the connection? The only way
for the backend and pgbouncer to communicate is by embedding information
in the data stream. Which means pgbouncer still has to parse it.

Furthermore, those are not the only bits of information pgbouncer may
need. For example, if pgbouncer gets improved to handle prepared
statements (which is likely) it'd need to handle PARSE/BIND/EXECUTE. And
it already needs to handle SET parameters. And so on.

In any case, this discussion is somewhat off topic in this thread, so
let's not hijack it.


regards

-- 
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services