Re: libpq compression (part 3) - Mailing list pgsql-hackers

From Jacob Burroughs
Subject Re: libpq compression (part 3)
Date
Msg-id CACzsqT7pirbDjVPrYL4fciFDOkByXn+HkKHss-48_6mgMsY5gA@mail.gmail.com
Whole thread Raw
In response to Re: libpq compression (part 3)  (Jacob Champion <jacob.champion@enterprisedb.com>)
Responses Re: libpq compression (part 3)
List pgsql-hackers
On Tue, May 21, 2024 at 1:39 PM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
>
> If the server doesn't reject compressed packets pre-authentication,
> then case 2 isn't mitigated. (I haven't proven how risky that case is
> yet, to be clear.) In other words: if the threat model is that a
> client can attack us, we shouldn't assume that it will attack us
> politely.

I think I thought I was writing about something else when I wrote that
:sigh:.  I think what I really should have written was a version of
the part below, which is that we use streaming decompression, only
decompress 8kb at a time, and for pre-auth messages limit them to
`PG_MAX_AUTH_TOKEN_LENGTH` (65535 bytes), which isn't really enough
data to actually cause any real-world pain by needing to decompress vs
the equivalent pain of sending invalid uncompressed auth packets.

> > Because we are using streaming decompression, this is much less of an
> > issue than for things that decompress wholesale onto disk/into memory.
>
> (I agree in general, but since you're designing a protocol extension,
> IMO it's not enough that your implementation happens to mitigate
> risks. We more or less have to bake those mitigations into the
> specification of the extension, because things that aren't servers
> have to decompress now. Similar to RFC security considerations.)

We own both the canonical client and server, so those are both covered
here.  I would think it would be the responsibility of any other
system that maintains its own implementation of the postgres protocol
and chooses to support the compression protocol to perform its own
mitigations against potential compression security issues.  Should we
put the fixed message size limits (that have de facto been part of the
protocol since 2021, even if they weren't documented as such) into the
protocol documentation?  That would give implementers of the protocol
numbers that they could actually rely on when implementing the
appropriate safeguards because they would be able to actually have
explicit guarantees about the size of messages. I think it would make
more sense to put the limits on the underlying messages rather than
adding an additional limit that only applies to compressed messages.
( I don't really see how one could implement other tooling that used
pg compression without using streaming compression, as the protocol
never hands over a standalone blob of compressed data: all compressed
data is always part of a stream, but even with streaming decompression
you still need some kind of limits or you will just chew up memory.)

--
Jacob Burroughs | Staff Software Engineer
E: jburroughs@instructure.com



pgsql-hackers by date:

Previous
From: Robert Haas
Date:
Subject: Re: First draft of PG 17 release notes
Next
From: Nathan Bossart
Date:
Subject: Re: problems with "Shared Memory and Semaphores" section of docs