Thread: Documenting the Frontend/Backend Protocol update criteria

<div id="OWAFontStyleDivID"
style="font-size:12pt;color:#000000;background-color:#FFFFFF;font-family:Calibri,Arial,Helvetica,sans-serif;"><p>Hi,<br
/><br/> Currently the criteria on updating the F/B protocol is undefined. This makes it hard to update the protocol
goingforward. It makes it also hard to write library/driver/application implementations that will be more future proof
tofuture server versions.<br /><br /> Ideally the documentation for 9.4 (backport?) would say what kind of things are
allowedto change within the v3 protocol, and thus implies what kind of changes need a new v4 protocol. Is there some
wishlistpage of items to do differently for v4 already?<br /><br /> I did find the following text in the documentation:
"binaryrepresentations for complex data types might change across server versions". But having more specific rules
wouldhelp, especially since it seems to be there just to scare: so far changes have been strongly discouraged.<br /><br
/>An example to consider: some binary formats have flags (arrays) or version (jsonb) field. We should explicitly say
thatclients must reject any unknown bits/versions that they do not know about. This guarantees they detect small format
updatesinstead of silently accepting then and possibly returning corrupt data.<br /><br /> My motivation:<br /><br />
Twoyears ago accidentally I opened a discussion on how to do updates to the binary encoding of data in the protocol
[1].I would like to reopen the discussion now since the jsonb 'binary' encoding is just a version '1' + text json. The
resultfrom the last discussion was that having a version or flags as part of the binary format is not enough, since
drivers/libraries(fixable) and applications (unfixable) are depending on the current encoding.<br /> And if we add a
newbit to the flags or bump the version number we will break backward compatibility.<br /><br /> To summarize the
previousdiscussion:<br /> - there are currently no written rules for modifying the binary encoding formats<br /> -
byteamodification was done with a GUC, but GUC was seen as a bad solution in general<br /> - my proposal was to add a
minorformat version number was not good either since any per session state would be problematic for connection
poolers<br/><br /> [1]:
http://grokbase.com/t/postgresql/pgsql-hackers/11bwhv1esa/add-minor-version-to-v3-protocol-to-allow-changes-without-breaking-backwards-compatibility<br
/></div>

Jan Urbański made a presentation titled 'Postgres on the wire',
subtitle 'A look at the PostgreSQL wire protocol'.   I hope this
covers some of your interest.   Presentation slide deck is available
at
http://www.pgcon.org/2014/schedule/attachments/330_postgres-for-the-wire.pdf

Hope it helps;
---
Koichi Suzuki


2014-06-02 7:22 GMT+09:00 Mikko Tiihonen <Mikko.Tiihonen@nitorcreations.com>:
> Hi,
>
> Currently the criteria on updating the F/B protocol is undefined. This makes
> it hard to update the protocol going forward. It makes it also hard to write
> library/driver/application implementations that will be more future proof to
> future server versions.
>
> Ideally the documentation for 9.4 (backport?) would say what kind of things
> are allowed to change within the v3 protocol, and thus implies what kind of
> changes need a new v4 protocol. Is there some wishlist page of items to do
> differently for v4 already?
>
> I did find the following text in the documentation: "binary representations
> for complex data types might change across server versions". But having more
> specific rules would help, especially since it seems to be there just to
> scare: so far changes have been strongly discouraged.
>
> An example to consider: some binary formats have flags (arrays) or version
> (jsonb) field. We should explicitly say that clients must reject any unknown
> bits/versions that they do not know about. This guarantees they detect small
> format updates instead of silently accepting then and possibly returning
> corrupt data.
>
> My motivation:
>
> Two years ago accidentally I opened a discussion on how to do updates to the
> binary encoding of data in the protocol [1]. I would like to reopen the
> discussion now since the jsonb 'binary' encoding is just a version '1' +
> text json. The result from the last discussion was that having a version or
> flags as part of the binary format is not enough, since drivers/libraries
> (fixable) and applications (unfixable) are depending on the current
> encoding.
> And if we add a new bit to the flags or bump the version number we will
> break backward compatibility.
>
> To summarize the previous discussion:
> - there are currently no written rules for modifying the binary encoding
> formats
> - bytea modification was done with a GUC, but GUC was seen as a bad solution
> in general
> - my proposal was to add a minor format version number was not good either
> since any per session state would be problematic for connection poolers
>
> [1]:
>
http://grokbase.com/t/postgresql/pgsql-hackers/11bwhv1esa/add-minor-version-to-v3-protocol-to-allow-changes-without-breaking-backwards-compatibility