Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs - Mailing list pgsql-hackers
From | Jacob Burroughs |
---|---|
Subject | Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs |
Date | |
Msg-id | CACzsqT5oe7MTYdD9yNRssnm=WcsSz=tL2HjzGz33R_dbAYqiDg@mail.gmail.com Whole thread Raw |
In response to | Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs (Robert Haas <robertmhaas@gmail.com>) |
Responses |
Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs
|
List | pgsql-hackers |
On Thu, May 16, 2024 at 6:57 AM Robert Haas <robertmhaas@gmail.com> wrote: > > Ugh, it's so hard to communicate clearly about this stuff. I didn't > really have any thought that we'd ever try to handle something as > complicated as compression using ParameterSet. I tend to agree that > for compression I'd like to see the startup packet contain more than > _pq_.compression=1, but I'm not sure what would happen after that > exactly. If the client asks for _pq_.compression=lz4 and the server > tells the client that it doesn't understand _pq_.compression at all, > then everybody's on the same page: no compression. But, if the server > understands the option but isn't OK with the proposed value, what > happens then? Does it send a NegotiateCompressionType message after > the NegotiateProtocolVersion, for example? That seems like it could > lead to the client having to be prepared for a lot of NegotiateX > messages somewhere down the road. > > I think at some point in the past we had discussed having the client > list all the algorithms it supported in the argument to > _pq_.compression, and then the server would respond with the algorithm > it wanted use, or maybe a list of algorithms that it could allow, and > then we'd go from there. But I'm not entirely sure if that's the right > idea, either. As currently implemented [1], the client sends the server the list of all compression algorithms it is willing to accept, and the server can use one of them. If the server knows what `_pq_.compression` means but doesn't actually support any compression, it will both send the client its empty list of supported algorithms and just never send any compressed messages, and everyone involved will be (relatively) happy. There is a libpq function that a client can use to check what compression is in use if a client *really* doesn't want to continue with the conversation without compression, but 99% of the time I can't see why a client wouldn't prefer to continue using a connection with whatever compression the server supports (or even none) without more explicit negotiation. (Unlike TLS, where automagically picking between using and not using TLS has strange security implications and effects, compression is a convenience feature for everyone involved.) > Changing compression algorithms in mid-stream is tricky, too. If I > tell the server "hey, turn on server-to-client compression!" then I > need to be able to identify where exactly that happens. Any messages > already sent by the server and not yet processed by me, or any > messages sent after that but before the server handles my request, are > going to be uncompressed. Then, at some point, I'll start getting > compressed data. If the compressed data is framed inside some message > type created for that purpose, like I get a CompressedMessage message > and then I decompress to get the actual message, this is simpler to > manage. But even then, it's tricky if the protocol shifts. If I tell > the server, you know what, gzip was a bad choice, I want lz4, I'll > need to know where the switch happens to be able to decompress > properly. > > I don't know if we want to support changing compression algorithms in > mid-stream. I don't think there's any reason we can't, but it might be > a bunch of work for something that nobody really cares about. Not > sure. As the protocol layer is currently designed [1], it explicitly makes it very easy to change/restart compression streams, specifically for this use case (and in particular for the general connection pooler use case). Compressed data is already framed in a `CompressedData` message, and that message has a header byte that corresponds to an enum value for which algorithm is currently in use. Any time the compression stream was restarted by the sender, the first `CompressedData` message will set that byte, and then the client will restart its decompression stream with the chosen algorithm from that point. For `CompressedData` messages that continue using the already-established stream, the byte is simply set to 0. (This is also how the "each side sends a list" form of negotiation is able to work without additional round trips, as the `CompressedData` framing itself communicates which compression algorithm has been selected.) [1] https://www.postgresql.org/message-id/CACzsqT5-7xfbz%2BSi35TBYHzerNX3XJVzAUH9AewQ%2BPp13fYBoQ%40mail.gmail.com -- Jacob Burroughs | Staff Software Engineer E: jburroughs@instructure.com
pgsql-hackers by date: