Re: Add minor version to v3 protocol to allow changes without breaking backwards compatibility - Mailing list pgsql-hackers

On 11/30/2011 06:52 PM, Merlin Moncure wrote:
> On Mon, Nov 28, 2011 at 9:18 AM, Mikko Tiihonen
> <mikko.tiihonen@nitorcreations.com>  wrote:
>> Hi,
>>
>> As discussed few days ago it would be beneficial if we could change the v3
>> backend<->client protocol without breaking backwards compatibility.
>>
>> Here is a working patch that exports a supported_binary_minor constant and a
>> binary_minor session variable and a that can be used by clients to enable
>> newer features.
>>
>> I also added an example usage where the array encoding for constant size
>> elements is optimized if binary_minor version is new enough.
>>
>> I have coded the client side support for binary_minor for jdbc driver and
>> tested that it worked. But lets first discuss if this is an acceptable path
>> forward.
>
> Regarding your TODO in the code comments about interactions with
> replication:  I think it should be removed.  WAL streaming depends on
> more things being identical than what is guaranteed here which is
> basically the protocol + data formats.

OK. I'll remove the comments about replication.

> I think all references to
> 'protocol' should be removed;  Binary wire formats != protocol: the
> protocol could bump to v4 but the wire formats (by happenstance) could
> all still continue to work -- therefore the version isn't minor (it's
> not dependent on protocol version but only on itself).  Therefore,
> don't much like the name 'supported_binary_minor'.  How about
> binary_format_version?

I was thinking that it would be possible use the new minor version to
introduce also new protocol messages such as streaming of large data
into database without knowing it's size beforehand.

> Also, is a non granular approach really buying us anything here?  ISTM
> *something* is likely to change format on most releases of the server
> so I'm wondering what's the point (if you don't happen to be on the
> same x.x release of the server, you might as well assume to not match
> or at least 'go on your own risk). The value added to the client
> version query is quite low.

You have a very good point about changes in every new postgres
version. Either text or the binary encoding is likely to change for
some types in each new release.

There needs to be decision on official policy about breaking backwards
compatibility of postgresql clients. Is it:

A) Every x.y postgres release is free to break any parts of the   * protocol   * text encoding   * binary protocol   as
longas it is documented   -> all client libraries should be coded so that they refuse to      support version x.y+1 or
newer(they might have a option to      override this but there are no guarantees that it would work)   Good: no random
bugswhen using old client library   Bad: initial complaints from users before they understand that        this is the
bestoption for everyone
 

B) Every x.y postgres release must guarantee that no client visible   parts of protocol, text encoding or binary
encodingwill change   from previous release in the v3 protocol. If any changes are   needed then a new protocol version
mustbe created.   -> very high barrier for new features   Good: can upgrade server without updating clients   Bad: new
featuresare only introduced very rarely after enough        have accumulated   Bad: many feature/change patches will
rotwhile waiting for the        upcoming new version
 

C) There is effort to try avoiding incompatible changes. Some   changes are blocked because it is detected that they
canbreak   backwards compatibility while others are let through (often with   some compatibility option on server side
tofall back to   previous functionality (f.ex. bytea hex encoding).   -> As far as I understand this is the current
situation.  Good: has worked so far   Bad: accumulates compatibility flags in the server
 

D) My proposed compromise where there is one minor_version setting   and code in server to support all different minor
versions.  The client requests the minor version so that all releases   default to backwards compatible version.   When
therecombinations starts to create too much maintenance   overhead a new clean v4 version of protocol is specified.
Good:Keeps full backwards compatibility   Good: Allows introducing changes at any time   Bad: Accumulates conditional
codeto server and clients until        new version of protocol is released
 


I'd like the official policy to be A, otherwise I'll push for D.

-Mikko