Thread: Connection pooler / LDAP auth / Load Balancing on read-only queries
Dear Members I am searching (like many others) of the holy grail of PostgreSQL pooling + Load Balancing . Ideally I would like : - A pgbouncer with better/native LDAP (not PAM) / and eventually Kerberos support - A pgbouncer enhanced with load balancing read-only queries (like pgpool) or - A pgpool-II with better resources utilization / more efficient pooling (like pgbouncer) There are other solutions available pgcat (no LDAP), odyssey (no load balancing) , pgagroal, supavisor, none of which seem to cover the above. Some ppl advice , keeping pgbouncer close to the app(s) and pgpool close to the DB. But kinda had mixed feelings putting the two work together. I'd like to ask if there any thoughts or even hopes that some of the above will be available in a single software, or otherwise how do people tackle this. Thank you -- Achilleas Mantzios IT DEV - HEAD IT DEPT Dynacom Tankers Mgmt (as agents only)
the read only detection is tricky--consider transaction level pooling, and the first query is read only -- Scott Ribe scott_ribe@elevated-dev.com https://www.linkedin.com/in/scottribe/ > On Jul 3, 2024, at 2:25 PM, Achilleas Mantzios <a.mantzios@cloud.gatewaynet.com> wrote: > > Dear Members > > I am searching (like many others) of the holy grail of PostgreSQL pooling + Load Balancing . Ideally I would like : > > - A pgbouncer with better/native LDAP (not PAM) / and eventually Kerberos support > > - A pgbouncer enhanced with load balancing read-only queries (like pgpool) > > or > > - A pgpool-II with better resources utilization / more efficient pooling (like pgbouncer) > > > There are other solutions available pgcat (no LDAP), odyssey (no load balancing) , pgagroal, supavisor, none of which seemto cover the above. > > Some ppl advice , keeping pgbouncer close to the app(s) and pgpool close to the DB. But kinda had mixed feelings puttingthe two work together. > > I'd like to ask if there any thoughts or even hopes that some of the above will be available in a single software, or otherwisehow do people tackle this. > > Thank you > > -- > Achilleas Mantzios > IT DEV - HEAD > IT DEPT > Dynacom Tankers Mgmt (as agents only) > > >
Am 03.07.24 um 22:25 schrieb Achilleas Mantzios: > Dear Members > > I am searching (like many others) of the holy grail of PostgreSQL > pooling + Load Balancing . Ideally I would like : > > - A pgbouncer with better/native LDAP (not PAM) / and eventually > Kerberos support > > - A pgbouncer enhanced with load balancing read-only queries (like > pgpool) > > or > > - A pgpool-II with better resources utilization / more efficient > pooling (like pgbouncer) > > > There are other solutions available pgcat (no LDAP), odyssey (no load > balancing) , pgagroal, supavisor, none of which seem to cover the above. > > Some ppl advice , keeping pgbouncer close to the app(s) and pgpool > close to the DB. But kinda had mixed feelings putting the two work > together. > > I'd like to ask if there any thoughts or even hopes that some of the > above will be available in a single software, or otherwise how do > people tackle this. > > Thank you > Have a look at pgCat, a newcomer which might meet your needs. Cheers Holger -- Holger Jakobs, Bergisch Gladbach
Re: Connection pooler / LDAP auth / Load Balancing on read-only queries
From
Achilleas Mantzios - cloud
Date:
On 7/3/24 23:32, Scott Ribe wrote: > the read only detection is tricky--consider transaction level pooling, and the first query is read only Hello pgpool does a great job at that. pgpool does load balancing to the first statement that is considered a write statement, for that point on, the rest of the transaction is routed to the primary. But the question here is how to achieve all of the aforementioned features, of all those great tools combined, or ideally combined in one. > > -- > Scott Ribe > scott_ribe@elevated-dev.com > https://www.linkedin.com/in/scottribe/ > > > >> On Jul 3, 2024, at 2:25 PM, Achilleas Mantzios <a.mantzios@cloud.gatewaynet.com> wrote: >> >> Dear Members >> >> I am searching (like many others) of the holy grail of PostgreSQL pooling + Load Balancing . Ideally I would like : >> >> - A pgbouncer with better/native LDAP (not PAM) / and eventually Kerberos support >> >> - A pgbouncer enhanced with load balancing read-only queries (like pgpool) >> >> or >> >> - A pgpool-II with better resources utilization / more efficient pooling (like pgbouncer) >> >> >> There are other solutions available pgcat (no LDAP), odyssey (no load balancing) , pgagroal, supavisor, none of whichseem to cover the above. >> >> Some ppl advice , keeping pgbouncer close to the app(s) and pgpool close to the DB. But kinda had mixed feelings puttingthe two work together. >> >> I'd like to ask if there any thoughts or even hopes that some of the above will be available in a single software, orotherwise how do people tackle this. >> >> Thank you >> >> -- >> Achilleas Mantzios >> IT DEV - HEAD >> IT DEPT >> Dynacom Tankers Mgmt (as agents only) >> >> >>
Re: Connection pooler / LDAP auth / Load Balancing on read-only queries
From
Achilleas Mantzios - cloud
Date:
On 7/3/24 23:32, Holger Jakobs wrote: > Am 03.07.24 um 22:25 schrieb Achilleas Mantzios: >> Dear Members >> >> I am searching (like many others) of the holy grail of PostgreSQL >> pooling + Load Balancing . Ideally I would like : >> >> - A pgbouncer with better/native LDAP (not PAM) / and eventually >> Kerberos support >> >> - A pgbouncer enhanced with load balancing read-only queries (like >> pgpool) >> >> or >> >> - A pgpool-II with better resources utilization / more efficient >> pooling (like pgbouncer) >> >> >> There are other solutions available pgcat (no LDAP), odyssey (no load >> balancing) , pgagroal, supavisor, none of which seem to cover the above. >> >> Some ppl advice , keeping pgbouncer close to the app(s) and pgpool >> close to the DB. But kinda had mixed feelings putting the two work >> together. >> >> I'd like to ask if there any thoughts or even hopes that some of the >> above will be available in a single software, or otherwise how do >> people tackle this. >> >> Thank you >> > > Have a look at pgCat, a newcomer which might meet your needs. Hello, I already mentioned pgCat, and the reason it won't cut it. (no PAM, no LDAP, only MD5/SCRAM-SHA-256) > > Cheers > > Holger >
> Hello > > pgpool does a great job at that. pgpool does load balancing to the first statement that is considered a write statement,for that point on, the rest of the transaction is routed to the primary. > > But the question here is how to achieve all of the aforementioned features, of all those great tools combined, or ideallycombined in one. AFAIK, pgpool still doesn't do what I'd consider true pooling, that is MxN multiplexing of connections. (Every client connectionhas a connection from pgpool -> server, but the server connections become available for resuse when clients disconnect.) And to me, the mechanism for routing queries in a transaction is highly suspect, as the initial reads vs later writes couldpotentially be using different snapshots of the data. (There is a safeguard there, involving looking at replicationdelay, but that's not a transactional guarantee, just "here's how out of date the replica can be to get read queries".)
Re: Connection pooler / LDAP auth / Load Balancing on read-only queries
From
Achilleas Mantzios - cloud
Date:
On 7/4/24 15:47, Scott Ribe wrote: >> Hello >> >> pgpool does a great job at that. pgpool does load balancing to the first statement that is considered a write statement,for that point on, the rest of the transaction is routed to the primary. >> >> But the question here is how to achieve all of the aforementioned features, of all those great tools combined, or ideallycombined in one. > AFAIK, pgpool still doesn't do what I'd consider true pooling, that is MxN multiplexing of connections. (Every client connectionhas a connection from pgpool -> server, but the server connections become available for resuse when clients disconnect.) Yes, unfortunately, pgbouncer shines in this department. > > And to me, the mechanism for routing queries in a transaction is highly suspect, as the initial reads vs later writes couldpotentially be using different snapshots of the data. (There is a safeguard there, involving looking at replicationdelay, but that's not a transactional guarantee, just "here's how out of date the replica can be to get read queries".) It seems pgpool supports snapshot_isolation mode, which is similar to streaming_replication, + it adds visibility consistency, but at the expense of running with default_transaction_isolation = 'repeatable read' : https://www.pgpool.net/docs/latest/en/html/runtime-config-running-mode.html#GUC-SNAPSHOT-ISOLATION-MODE Read latency is an issue with asynchronous physical replication, but then again, the problem is there no matter the HA/pooling solution.