Thread: Middleware Messages for FE/BE

The current FE/BE protocol assumes that the client is talking to the
server directly. If middleware wants to do something like load
balancing, the only current option is to inspect the incoming commands
and take action from that. That approach performs poorly and has
proven difficult to maintain, limiting the functionality in Postgres
ecosystem middleware.

It would be useful to have a way to speak to middleware within a
session or prior to each command. There are ways to frig this and
obviously it can always be done out-of-core, but there is a clear
requirement for various client tools to agree a standard way for them
to send messages to middleware rather than the database server. If we
get PostgreSQL Project approval for this, then authors of client and
middleware tools will know how to interoperate and can begin adding
features to take advantage of this, allowing the Postgres ecosystem to
improve and extend its middleware.

Byte1('M')
Identifies the message as a middleware message. (Or perhaps use 'U'
for User Message?).

Int32
Length of message contents in bytes, including self.

Int64
Routing/class identifier, allowing middleware to quickly ignore whole
classes of message if not appropriate. We would run some kind of
allocation scheme to ensure unique meaning, probably via the Postgres
Wiki. The first 2 billion values would be reserved for allocation by
the PostgreSQL Project itself, values beyond that open for external
allocation.

Byten
The message itself, where n is the remaining bytes in the message.

The message is intended for middleware only. The server always ignores
these messages, with an optional debug facility that can be enabled to
allow printing NOTICEs to allow testing.

I will supply a patch to any agreed message format, together with a
new libpq call to utilise this.

In summary: the patch is easy, the point is we need agreement to allow
and encourage interoperation between clients and middleware.

Thoughts?

-- 
Simon Riggs                http://www.EnterpriseDB.com/

Need for this does come up quite often so I very much support this.

In addition to keeping a registry there likely need to be some other
"generally agreed" rules as well, like
* it being safe to ignore any and all of the middleware messages (at
least with no degradation from the state of not having them)
* and maybe even a standard way to turn them on and off.

  On/Off switch could be of course done using flags for each
individual use case, but it would be good to agree conventions.

Another thing to agree would be a set of standard messages, like "I am
overloaded, consider moving some load away" or "Planning to switch
over to replica x.x.x.x, please follow"

-----
Hannu Krosing
Google Cloud - We have a long list of planned contributions and we are hiring.
Contact me if interested.


On Thu, Aug 19, 2021 at 10:33 AM Simon Riggs
<simon.riggs@enterprisedb.com> wrote:
>
> The current FE/BE protocol assumes that the client is talking to the
> server directly. If middleware wants to do something like load
> balancing, the only current option is to inspect the incoming commands
> and take action from that. That approach performs poorly and has
> proven difficult to maintain, limiting the functionality in Postgres
> ecosystem middleware.
>
> It would be useful to have a way to speak to middleware within a
> session or prior to each command. There are ways to frig this and
> obviously it can always be done out-of-core, but there is a clear
> requirement for various client tools to agree a standard way for them
> to send messages to middleware rather than the database server. If we
> get PostgreSQL Project approval for this, then authors of client and
> middleware tools will know how to interoperate and can begin adding
> features to take advantage of this, allowing the Postgres ecosystem to
> improve and extend its middleware.
>
> Byte1('M')
> Identifies the message as a middleware message. (Or perhaps use 'U'
> for User Message?).
>
> Int32
> Length of message contents in bytes, including self.
>
> Int64
> Routing/class identifier, allowing middleware to quickly ignore whole
> classes of message if not appropriate. We would run some kind of
> allocation scheme to ensure unique meaning, probably via the Postgres
> Wiki. The first 2 billion values would be reserved for allocation by
> the PostgreSQL Project itself, values beyond that open for external
> allocation.
>
> Byten
> The message itself, where n is the remaining bytes in the message.
>
> The message is intended for middleware only. The server always ignores
> these messages, with an optional debug facility that can be enabled to
> allow printing NOTICEs to allow testing.
>
> I will supply a patch to any agreed message format, together with a
> new libpq call to utilise this.
>
> In summary: the patch is easy, the point is we need agreement to allow
> and encourage interoperation between clients and middleware.
>
> Thoughts?
>
> --
> Simon Riggs                http://www.EnterpriseDB.com/
>
>

One more set of "standard middleware messages" clients/middleware
could turn on could be reporting LSNs
 * various local LSNs for progress of WAL persisting
 * reporting replication state of some or all replicas

-----
Hannu Krosing
Google Cloud - We have a long list of planned contributions and we are hiring.
Contact me if interested.

On Thu, Aug 19, 2021 at 11:33 AM Hannu Krosing <hannuk@google.com> wrote:
>
> Need for this does come up quite often so I very much support this.
>
> In addition to keeping a registry there likely need to be some other
> "generally agreed" rules as well, like
> * it being safe to ignore any and all of the middleware messages (at
> least with no degradation from the state of not having them)
> * and maybe even a standard way to turn them on and off.
>
>   On/Off switch could be of course done using flags for each
> individual use case, but it would be good to agree conventions.
>
> Another thing to agree would be a set of standard messages, like "I am
> overloaded, consider moving some load away" or "Planning to switch
> over to replica x.x.x.x, please follow"
>
> -----
> Hannu Krosing
> Google Cloud - We have a long list of planned contributions and we are hiring.
> Contact me if interested.
>
>
> On Thu, Aug 19, 2021 at 10:33 AM Simon Riggs
> <simon.riggs@enterprisedb.com> wrote:
> >
> > The current FE/BE protocol assumes that the client is talking to the
> > server directly. If middleware wants to do something like load
> > balancing, the only current option is to inspect the incoming commands
> > and take action from that. That approach performs poorly and has
> > proven difficult to maintain, limiting the functionality in Postgres
> > ecosystem middleware.
> >
> > It would be useful to have a way to speak to middleware within a
> > session or prior to each command. There are ways to frig this and
> > obviously it can always be done out-of-core, but there is a clear
> > requirement for various client tools to agree a standard way for them
> > to send messages to middleware rather than the database server. If we
> > get PostgreSQL Project approval for this, then authors of client and
> > middleware tools will know how to interoperate and can begin adding
> > features to take advantage of this, allowing the Postgres ecosystem to
> > improve and extend its middleware.
> >
> > Byte1('M')
> > Identifies the message as a middleware message. (Or perhaps use 'U'
> > for User Message?).
> >
> > Int32
> > Length of message contents in bytes, including self.
> >
> > Int64
> > Routing/class identifier, allowing middleware to quickly ignore whole
> > classes of message if not appropriate. We would run some kind of
> > allocation scheme to ensure unique meaning, probably via the Postgres
> > Wiki. The first 2 billion values would be reserved for allocation by
> > the PostgreSQL Project itself, values beyond that open for external
> > allocation.
> >
> > Byten
> > The message itself, where n is the remaining bytes in the message.
> >
> > The message is intended for middleware only. The server always ignores
> > these messages, with an optional debug facility that can be enabled to
> > allow printing NOTICEs to allow testing.
> >
> > I will supply a patch to any agreed message format, together with a
> > new libpq call to utilise this.
> >
> > In summary: the patch is easy, the point is we need agreement to allow
> > and encourage interoperation between clients and middleware.
> >
> > Thoughts?
> >
> > --
> > Simon Riggs                http://www.EnterpriseDB.com/
> >
> >

Actually the way I have been thinking about this would be a message

 "Hey replica x.x.x.x, please take over the write head role"

Which would

1. cause the replica  x.x.x.x to self-promote at this point-in-WAL
1A - optionally the old write head becomes a replica (it should
mention it in message if it plans to stay around as a replica)
2. tells other replicas to switch over and start replicating from it
3. tells all connected clients to switch, if they need read-write access.

This could be even done without timeline switch, as it should
guarantee that there is no split brain writing
Of course there are (and should be) ways to still use the WALs
normally for cases where replica x.x.x.x does not exists, like PITR

And making this play nicely with Logical Decoding is another can of
worms needing to be solved, but it is a start to becoming "Cloud
Native" :)

-----
Hannu Krosing
Google Cloud - We have a long list of planned contributions and we are hiring.
Contact me if interested.


On Thu, Aug 19, 2021 at 11:36 AM Hannu Krosing <hannuk@google.com> wrote:
>
> One more set of "standard middleware messages" clients/middleware
> could turn on could be reporting LSNs
>  * various local LSNs for progress of WAL persisting
>  * reporting replication state of some or all replicas
>
> -----
> Hannu Krosing
> Google Cloud - We have a long list of planned contributions and we are hiring.
> Contact me if interested.
>
> On Thu, Aug 19, 2021 at 11:33 AM Hannu Krosing <hannuk@google.com> wrote:
> >
> > Need for this does come up quite often so I very much support this.
> >
> > In addition to keeping a registry there likely need to be some other
> > "generally agreed" rules as well, like
> > * it being safe to ignore any and all of the middleware messages (at
> > least with no degradation from the state of not having them)
> > * and maybe even a standard way to turn them on and off.
> >
> >   On/Off switch could be of course done using flags for each
> > individual use case, but it would be good to agree conventions.
> >
> > Another thing to agree would be a set of standard messages, like "I am
> > overloaded, consider moving some load away" or "Planning to switch
> > over to replica x.x.x.x, please follow"
> >
> > -----
> > Hannu Krosing
> > Google Cloud - We have a long list of planned contributions and we are hiring.
> > Contact me if interested.
> >
> >
> > On Thu, Aug 19, 2021 at 10:33 AM Simon Riggs
> > <simon.riggs@enterprisedb.com> wrote:
> > >
> > > The current FE/BE protocol assumes that the client is talking to the
> > > server directly. If middleware wants to do something like load
> > > balancing, the only current option is to inspect the incoming commands
> > > and take action from that. That approach performs poorly and has
> > > proven difficult to maintain, limiting the functionality in Postgres
> > > ecosystem middleware.
> > >
> > > It would be useful to have a way to speak to middleware within a
> > > session or prior to each command. There are ways to frig this and
> > > obviously it can always be done out-of-core, but there is a clear
> > > requirement for various client tools to agree a standard way for them
> > > to send messages to middleware rather than the database server. If we
> > > get PostgreSQL Project approval for this, then authors of client and
> > > middleware tools will know how to interoperate and can begin adding
> > > features to take advantage of this, allowing the Postgres ecosystem to
> > > improve and extend its middleware.
> > >
> > > Byte1('M')
> > > Identifies the message as a middleware message. (Or perhaps use 'U'
> > > for User Message?).
> > >
> > > Int32
> > > Length of message contents in bytes, including self.
> > >
> > > Int64
> > > Routing/class identifier, allowing middleware to quickly ignore whole
> > > classes of message if not appropriate. We would run some kind of
> > > allocation scheme to ensure unique meaning, probably via the Postgres
> > > Wiki. The first 2 billion values would be reserved for allocation by
> > > the PostgreSQL Project itself, values beyond that open for external
> > > allocation.
> > >
> > > Byten
> > > The message itself, where n is the remaining bytes in the message.
> > >
> > > The message is intended for middleware only. The server always ignores
> > > these messages, with an optional debug facility that can be enabled to
> > > allow printing NOTICEs to allow testing.
> > >
> > > I will supply a patch to any agreed message format, together with a
> > > new libpq call to utilise this.
> > >
> > > In summary: the patch is easy, the point is we need agreement to allow
> > > and encourage interoperation between clients and middleware.
> > >
> > > Thoughts?
> > >
> > > --
> > > Simon Riggs                http://www.EnterpriseDB.com/
> > >
> > >

On Thu, 19 Aug 2021 at 10:33, Hannu Krosing <hannuk@google.com> wrote:
>
> Need for this does come up quite often so I very much support this.
>
> In addition to keeping a registry there likely need to be some other
> "generally agreed" rules as well, like
> * it being safe to ignore any and all of the middleware messages (at
> least with no degradation from the state of not having them)
> * and maybe even a standard way to turn them on and off.
>
>   On/Off switch could be of course done using flags for each
> individual use case, but it would be good to agree conventions.

Good feedback and thoughts, thanks.

Based on your interest, I will begin to code, but won't be ready for Sept CF.

> Another thing to agree would be a set of standard messages, like "I am
> overloaded, consider moving some load away" or "Planning to switch
> over to replica x.x.x.x, please follow"

That sounds like a server->middleware message and existing mechanisms
might work for that already.

-- 
Simon Riggs                http://www.EnterpriseDB.com/

On Thu, 19 Aug 2021 at 10:58, Hannu Krosing <hannuk@google.com> wrote:
>
> Actually the way I have been thinking about this would be a message
>
>  "Hey replica x.x.x.x, please take over the write head role"
>
> Which would
>
> 1. cause the replica  x.x.x.x to self-promote at this point-in-WAL
> 1A - optionally the old write head becomes a replica (it should
> mention it in message if it plans to stay around as a replica)
> 2. tells other replicas to switch over and start replicating from it
> 3. tells all connected clients to switch, if they need read-write access.
>
> This could be even done without timeline switch, as it should
> guarantee that there is no split brain writing
> Of course there are (and should be) ways to still use the WALs
> normally for cases where replica x.x.x.x does not exists, like PITR
>
> And making this play nicely with Logical Decoding is another can of
> worms needing to be solved, but it is a start to becoming "Cloud
> Native" :)

Good ideas!

-- 
Simon Riggs                http://www.EnterpriseDB.com/

Hi Simon,

On 8/19/21 4:33 AM, Simon Riggs wrote:
> The current FE/BE protocol assumes that the client is talking to the
> server directly. If middleware wants to do something like load
> balancing, the only current option is to inspect the incoming commands
> and take action from that. That approach performs poorly and has
> proven difficult to maintain, limiting the functionality in Postgres
> ecosystem middleware.
>
> It would be useful to have a way to speak to middleware within a
> session or prior to each command. There are ways to frig this and
> obviously it can always be done out-of-core, but there is a clear
> requirement for various client tools to agree a standard way for them
> to send messages to middleware rather than the database server. If we
> get PostgreSQL Project approval for this, then authors of client and
> middleware tools will know how to interoperate and can begin adding
> features to take advantage of this, allowing the Postgres ecosystem to
> improve and extend its middleware.
>
> Byte1('M')
> Identifies the message as a middleware message. (Or perhaps use 'U'
> for User Message?).
>
> Int32
> Length of message contents in bytes, including self.
>
> Int64
> Routing/class identifier, allowing middleware to quickly ignore whole
> classes of message if not appropriate. We would run some kind of
> allocation scheme to ensure unique meaning, probably via the Postgres
> Wiki. The first 2 billion values would be reserved for allocation by
> the PostgreSQL Project itself, values beyond that open for external
> allocation.
>
> Byten
> The message itself, where n is the remaining bytes in the message.
>
> The message is intended for middleware only. The server always ignores
> these messages, with an optional debug facility that can be enabled to
> allow printing NOTICEs to allow testing.
>
> I will supply a patch to any agreed message format, together with a
> new libpq call to utilise this.
>
> In summary: the patch is easy, the point is we need agreement to allow
> and encourage interoperation between clients and middleware.
>
> Thoughts?
>

I would say that this is a PostgreSQL protocol v4 thing, as there is a 
bit more to it.


There is a need to trigger middleware functionality, but you need to 
query the middleware stack first on what it supports - failover, load 
balancing, ... And, what type of language is that ? SQL query ? Not all 
middleware support general SQL queries.


Once you have established what is supported and what isn't you need the 
client driver (libpq is the very easy part) - so lets include Java, C#, 
Rust, golang, ... - to understand their environment to trigger the 
correct message in the configured scenario. F.ex. how is multiple 
application clusters connecting to the same middleware instance 
(connection pool) going to coordinate their 'M' message ?


If you are looking to configure the middleware instance then we can just 
use the existing protocol with FE/Q; that said pgagroal has its own 
binary protocol for admin payload.


Could you expand on typical scenarios that you want to see implemented ?


Best regards,

  Jesper

On Thu, 19 Aug 2021 at 17:42, Jesper Pedersen
<jesper.pedersen@redhat.com> wrote:

> I would say that this is a PostgreSQL protocol v4 thing, as there is a
> bit more to it.
>
>
> There is a need to trigger middleware functionality, but you need to
> query the middleware stack first on what it supports - failover, load
> balancing, ... And, what type of language is that ? SQL query ? Not all
> middleware support general SQL queries.

Agreed. No part of my proposal mentioned SQL queries.

We need to be able to send the middleware a message. Replies can be
sent as NoticeResponse, if needed, which already exists.

If you want to invent a new grammar for your middleware, cool, I'm not
stopping you.

> Once you have established what is supported and what isn't you need the
> client driver (libpq is the very easy part) - so lets include Java, C#,
> Rust, golang, ... - to understand their environment to trigger the
> correct message in the configured scenario. F.ex. how is multiple
> application clusters connecting to the same middleware instance
> (connection pool) going to coordinate their 'M' message ?

Each component can speak to the middleware to discover that, if it
wishes. The middleware can coordinate.

> If you are looking to configure the middleware instance then we can just
> use the existing protocol with FE/Q;

Which is exactly the thing we already have and the very thing I said I
was trying to avoid.

> that said pgagroal has its own binary protocol for admin payload.

I want to empower all future middleware, not just support one. Perhaps
we can take code or ideas from pgagroal and put it in core? Please
explain the binary protocol some more.

> Could you expand on typical scenarios that you want to see implemented ?

I see this as a generalized interface that can be used for a great many things.
* Failover managers
* Load Balancers
* Routing agents
* Things I haven't thought of yet, but others may have
etc..

We are currently limited by the messages we can send efficiently.

-- 
Simon Riggs                http://www.EnterpriseDB.com/

Hi,

On 8/19/21 1:13 PM, Simon Riggs wrote:
> We need to be able to send the middleware a message. Replies can be
> sent as NoticeResponse, if needed, which already exists.
> 

Yeah, but you need the client driver - of all languages - to understand 
that notice. And, if different middleware uses different codes then the 
problem is just pushed to the client;

Java: M|12|0 (Do you support failover ?) MW1: N|6|-|0 (meaning: yes)
Rust: M|12|0 (Do you support load balancing ?) MW2: N|6|-|0 (meaning: no)

Should the middleware guess the client driver based on the startup 
message ? Or are you thinking about having a class identifier group for 
each client driver and then a massive "switch/case" inside each middleware ?

>> Once you have established what is supported and what isn't you need the
>> client driver (libpq is the very easy part) - so lets include Java, C#,
>> Rust, golang, ... - to understand their environment to trigger the
>> correct message in the configured scenario. F.ex. how is multiple
>> application clusters connecting to the same middleware instance
>> (connection pool) going to coordinate their 'M' message ?
> 
> Each component can speak to the middleware to discover that, if it
> wishes. The middleware can coordinate.
> 

So, application cluster 1 (written in Java) detects that a failover is 
needed, and application cluster 2..N (written in Rust, C# and golang) 
should detect that mid-transaction - or rollback and send their 'M' for 
the same thing ?

> I want to empower all future middleware, not just support one. Perhaps
> we can take code or ideas from pgagroal and put it in core? Please
> explain the binary protocol some more.
> 

There is nothing fancy there (see management.c) - so the official 
protocol should be the focus.

>> Could you expand on typical scenarios that you want to see implemented ?
> 
> I see this as a generalized interface that can be used for a great many things.
> * Failover managers
> * Load Balancers
> * Routing agents
> * Things I haven't thought of yet, but others may have
> etc..
>

Moving control to the client will make some of this harder. Maybe 
PeterE, Andrey and Tatsuo-san have additional feedback based on their 
experience.

> We are currently limited by the messages we can send efficiently.
> 

There are a lot of good ideas for the PostgreSQL protocol v4 already.

Best regards,
  Jesper

On Thu, 19 Aug 2021 at 19:04, Jesper Pedersen
<jesper.pedersen@redhat.com> wrote:

> Should the middleware guess the client driver based on the startup
> message ? Or are you thinking about having a class identifier group for
> each client driver and then a massive "switch/case" inside each middleware ?

The question is how does a client communicate with middleware?

The message that would be sent would relate to the features of the
middleware, while being agnostic as to the client driver.

-- 
Simon Riggs                http://www.EnterpriseDB.com/

Jesper, please don't confuse my ramblings with Simon's initial proposal.

As I understand, the original proposal was just about adding a new
wire protocol message type, which then could be used for emitting
custom messages by middleware support extension - likely loaded as a
preloaded shared library - and consumed by some middleware, which
could either be some proxy or connection pooler or something compiled
as part of the libpq, JDBC or other client driver. So it was just
about providing extensibility to the protocol (the same way almost
everything else in PostgreSQL is extensible).

But at least initially each middleware would process only it's own
class, so a single if() and not a big switch/case :)

The things I talked about were some forward-looking proposals on how
to use this capability and that some of the messages may at some point
become used / usable by more than a single middleware component at
which point they should be standardised .

-----
Hannu Krosing
Google Cloud - We have a long list of planned contributions and we are hiring.
Contact me if interested.

On Thu, Aug 19, 2021 at 8:29 PM Simon Riggs
<simon.riggs@enterprisedb.com> wrote:
>
> On Thu, 19 Aug 2021 at 19:04, Jesper Pedersen
> <jesper.pedersen@redhat.com> wrote:
>
> > Should the middleware guess the client driver based on the startup
> > message ? Or are you thinking about having a class identifier group for
> > each client driver and then a massive "switch/case" inside each middleware ?
>
> The question is how does a client communicate with middleware?
>
> The message that would be sent would relate to the features of the
> middleware, while being agnostic as to the client driver.
>
> --
> Simon Riggs                http://www.EnterpriseDB.com/
>
>

Hi Hannu,

On 8/19/21 3:52 PM, Hannu Krosing wrote:
> Jesper, please don't confuse my ramblings with Simon's initial proposal.
> 
> As I understand, the original proposal was just about adding a new
> wire protocol message type, which then could be used for emitting
> custom messages by middleware support extension - likely loaded as a
> preloaded shared library - and consumed by some middleware, which
> could either be some proxy or connection pooler or something compiled
> as part of the libpq, JDBC or other client driver. So it was just
> about providing extensibility to the protocol (the same way almost
> everything else in PostgreSQL is extensible).
> 

Yeah, but it is a change to the protocol which means that the client 
drivers and middleware ecosystem needs to be updated to support that 
message.

So, if the message was added to the protocol we could add another 
message with the response to the request and make the protocol stricter, say

FE/M
Int32 - Length
Int16 - Request type (0 == Query capability, 1 == Execute capability)
Int32 - Capability type (0 == Failover, 1 == ..., ...)

BE/?
Int32 - Length
Int32 - Capability type
Byte  - Support (0 == No, 1 == Yes)
Byten - Additional information

None of the client driver interfaces (specification API, like JDBC) has 
functionality for this currently, so the client would need to type cast 
the interface in all cases in order to get access to the trigger method, say

  org.postgresql.jdbc.PgConnection.failover()

There could be custom capability type codes that targets specific 
middleware like Simon suggested.

> But at least initially each middleware would process only it's own
> class, so a single if() and not a big switch/case :)
>

With a defined baseline of capabilities the client drivers and 
middleware wouldn't have to invent their own codes.

> The things I talked about were some forward-looking proposals on how
> to use this capability and that some of the messages may at some point
> become used / usable by more than a single middleware component at
> which point they should be standardised .
> 

It would be good to see a PoC of one capability implemented from the 
client to the middleware that deals with coordination at the client level.

Best regards,
  Jesper

On Fri, 20 Aug 2021 at 13:59, Jesper Pedersen
<jesper.pedersen@redhat.com> wrote:
>
> On 8/19/21 3:52 PM, Hannu Krosing wrote:
> > Jesper, please don't confuse my ramblings with Simon's initial proposal.
> >
> > As I understand, the original proposal was just about adding a new
> > wire protocol message type, which then could be used for emitting
> > custom messages by middleware support extension - likely loaded as a
> > preloaded shared library - and consumed by some middleware, which
> > could either be some proxy or connection pooler or something compiled
> > as part of the libpq, JDBC or other client driver. So it was just
> > about providing extensibility to the protocol (the same way almost
> > everything else in PostgreSQL is extensible).
> >
>
> Yeah, but it is a change to the protocol which means that the client
> drivers and middleware ecosystem needs to be updated to support that
> message.

No, because FE messages don't need to be handled by the client, they
just send them. It is the server that needs to be updated to
understand that these messages might be received and to ignore them,
which is simple enough.

If a client doesn't know about a message it COULD send, but doesn't,
then there is no update required.

> So, if the message was added to the protocol we could add another
> message with the response to the request and make the protocol stricter, say
>
> FE/M
> Int32 - Length
> Int16 - Request type (0 == Query capability, 1 == Execute capability)
> Int32 - Capability type (0 == Failover, 1 == ..., ...)

This much detail is optional. It is up to the middleware to define if
it supports capability requests, or how it handles requests that it
cannot satisfy.

I'm trying to come up with something generic that people can use for
decades to come, not restrict their choices to a very small subset
based upon our current vision.

> BE/?
> Int32 - Length
> Int32 - Capability type
> Byte  - Support (0 == No, 1 == Yes)
> Byten - Additional information

If we add a new message from BE, then yes, we need to modify all
drivers, which would be an immediate fail for this proposal.

The message replies you foresee are optional; they are not required by
all middleware.

I've already suggested you use NoticeResponse, which is already
defined to ignore unknown field types, so is suitable and extensible.
We could add a new field type of 'm' to represent a message sent from
middleware to the client.

-- 
Simon Riggs                http://www.EnterpriseDB.com/

Hi Simon,

On 8/20/21 10:39 AM, Simon Riggs wrote:
>> Yeah, but it is a change to the protocol which means that the client
>> drivers and middleware ecosystem needs to be updated to support that
>> message.
> 
> No, because FE messages don't need to be handled by the client, they
> just send them.

Yes, but the middleware need to parse them in order to send a response.

> It is the server that needs to be updated to
> understand that these messages might be received and to ignore them,
> which is simple enough.
> 
> If a client doesn't know about a message it COULD send, but doesn't,
> then there is no update required.
> 

Agreed.

>> So, if the message was added to the protocol we could add another
>> message with the response to the request and make the protocol stricter, say
>>
>> FE/M
>> Int32 - Length
>> Int16 - Request type (0 == Query capability, 1 == Execute capability)
>> Int32 - Capability type (0 == Failover, 1 == ..., ...)
> 
> This much detail is optional. It is up to the middleware to define if
> it supports capability requests, or how it handles requests that it
> cannot satisfy.
> 
> I'm trying to come up with something generic that people can use for
> decades to come, not restrict their choices to a very small subset
> based upon our current vision.
> 
>> BE/?
>> Int32 - Length
>> Int32 - Capability type
>> Byte  - Support (0 == No, 1 == Yes)
>> Byten - Additional information
> 
> If we add a new message from BE, then yes, we need to modify all
> drivers, which would be an immediate fail for this proposal.
> 
> The message replies you foresee are optional; they are not required by
> all middleware.
> 
> I've already suggested you use NoticeResponse, which is already
> defined to ignore unknown field types, so is suitable and extensible.
> We could add a new field type of 'm' to represent a message sent from
> middleware to the client.
> 

When doing the PoC just keep in mind that middleware acting on a 'M' 
message on a user authenticated connection could lead to a DoS attack.

Best regards,
  Jesper