Re: 12.1 not useable: clientlib fails after a dozen queries (GSSAPI?) - Mailing list pgsql-hackers

From Stephen Frost
Subject Re: 12.1 not useable: clientlib fails after a dozen queries (GSSAPI?)
Date
Msg-id 20200114204517.GD3195@tamriel.snowman.net
Whole thread Raw
In response to Re: 12.1 not useable: clientlib fails after a dozen queries (GSSAPI ?)  (Tom Lane <tgl@sss.pgh.pa.us>)
List pgsql-hackers
Greetings,

* Tom Lane (tgl@sss.pgh.pa.us) wrote:
> Stephen Frost <sfrost@snowman.net> writes:
> > * Tom Lane (tgl@sss.pgh.pa.us) wrote:
> >> ... We must remember how much data we encrypted
> >> and then discount that much of the caller's supplied data next time.
> >> There are hints in the existing comments that somebody understood
> >> this at one point, but the code isn't acting that way today.
>
> > That's a case I hadn't considered and you're right- the algorithm
> > certainly wouldn't work in such a case.  I don't recall specifically if
> > the code had handled it better previously, or not, but I do recall there
> > was something previously about being given a buffer and then having the
> > API defined as "give me back the exact same buffer because I had to
> > stop" and I recall finding that to ugly, but I get it now, seeing this
> > issue.  I'd certainly be happier if there was a better alternative but I
> > don't know that there really is.
>
> Yeah.  The only bright spot is that there's no reason for the caller
> to change its mind about what it wants to write, so that this restriction
> doesn't really affect anything.  (The next call might potentially add
> *more* data at the end, but that's fine.)

Right, makes sense.

> I realized when I got into it that my sketch above also considered only
> part of the problem.  In the general case, we might've encrypted some data
> from the current write request and successfully sent it, and then
> encrypted some more data but been unable to (fully) send that packet.
> In this situation, it's best to report that we wrote however much data
> corresponds to the fully sent packet(s).  That way the caller can discard
> that data from its buffer.  We can't report the data corresponding to the
> in-progress packet as being written, though, or we have the
> might-not-get-another-call problem.  Fortunately the API already has the
> notion of a partial write, since the underlying socket calls do.

Yeah, I see how that's also an issue and agree that it makes sense to
report back what's been written and sent as a partial write, and not
report back everything we've "consumed" since we might not get called
again in that case.

Thanks!

Stephen

Attachment

pgsql-hackers by date:

Previous
From: Stephen Frost
Date:
Subject: Re: backup manifests
Next
From: Tomas Vondra
Date:
Subject: Re: planner support functions: handle GROUP BY estimates ?