Re: Wire protocol compression - Mailing list pgsql-hackers

From Craig Ringer
Subject Re: Wire protocol compression
Date
Msg-id CAMsr+YH0N7TaAtvS2hu-y8LBkA25fJs0oungGe-tNhwr7scLSQ@mail.gmail.com
Whole thread Raw
In response to Re: Wire protocol compression  (Tom Lane <tgl@sss.pgh.pa.us>)
List pgsql-hackers
On 21 April 2016 at 22:07, Tom Lane <tgl@sss.pgh.pa.us> wrote:
 

Yeah.  I think this should definitely be in the list of things we want
to add when we do the fabled 4.0 protocol revision (and, indeed, it's
been in the above-cited list for awhile).  Whether we've yet gotten to
the point of having critical mass for a revision ... meh, I doubt it.


Well, the critical mass is determined as much as anything by who's willing to put the large amount of work into implementing the protocol rev, testing clients, verifying performance, etc etc etc. So far I don't think anyone's leaping to volunteer.

We could use various hacks to enable wire compression without a protocol rev, like sending a gzip header before the startup packet so a server that doesn't understand it just sees a nonsensical startup packet and borks. But they all either boil down to uglier versions of genuine protocol bump or are potentially subject to the issues you raised earlier with GUC-controlled protocol changes.


In retrospect it's a huge pity there was no set of client-library-only key/value fields defined in the startup packet in addition to GUCs that libpq lets clients set via PGOPTIONS etc. That would've let us extend the protocol much more easily by sending capabilities like "I understand protocol compression".

Personally I think we should do that anyway - allow a startup-only GUC like proto_compression=gzip .

To help protect against abuse via 

  PGOPTIONS="-c proto_compression=gzip" ./exploitable-old-libpq-based-binary

we could do something like sending an ErrorResponse with a newly defined SQLSTATUS like 08P02 Client must support gzip compression . A client that supports gzip may ignore this error and proceed normally. A client that doesn't will not recognise the SQLSTATUS and since it's in a fatal category 08 (or rather, not the two non-fatal categories 00, 01 and 02) it'll die and won't see any upsetting new protocol messages. Clients that don't support it will even get a nice error message:

psql: FATAL:  unrecognized configuration parameter "proto_compression"

It's really just a hack around bumping the protocol to add capability negotiation though, and it'd scale very poorly if it was then used for more things. Lots of reconnections needed because we can't do a two-way negotiation.

I think retrofitting protocol compression might be compelling enough to justify that. But it's not really significantly better than just sending a v4 protoversion field and reconnecting if the server gets upset with that.


--
 Craig Ringer                   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services

pgsql-hackers by date:

Previous
From: Michael Paquier
Date:
Subject: Re: Why doesn't src/backend/port/win32/socket.c implement bind()?
Next
From: Aleksander Alekseev
Date:
Subject: Re: Parser extensions (maybe for 10?)