Re: Proposal: http2 wire format - Mailing list pgsql-hackers

From Craig Ringer
Subject Re: Proposal: http2 wire format
Date
Msg-id CAMsr+YHp+TA0U2GPorns1rsx5Ffry3yOmEHwF0qHx3RN6nbNog@mail.gmail.com
Whole thread Raw
In response to Re: Proposal: http2 wire format  (Andres Freund <andres@anarazel.de>)
Responses Re: Proposal: http2 wire format  (Andres Freund <andres@anarazel.de>)
List pgsql-hackers
On 28 March 2018 at 16:02, Andres Freund <andres@anarazel.de> wrote:
On 2018-03-26 22:44:09 +0200, Damir Simunic wrote:
> > *NONE* of the interesting problems are solved by HTTP2. You *still*
> > need a full blown protocol ontop of it. So no, this doesn't change that.
>
> If you had to nominate only one of those problems, which one would you consider the most interesting?

A few random, very tired, points:

- consolidated message for common tasks:
  - (bind, [describe?,] execute) to reduce overhead of prepared
    statement execution (both in messages, as well as branches)
  - (anonymous parse, bind, describe, execute) to make it cheaper to
    send statements with out-of-line parameters
- get rid of length limits of individual fields, probably w/ some variable
  length encoding (simple 7 bit?)

In preparation for the eventually-inevitable 64-bit field sizes, yes.

This should be on the protocol todo wiki.
 
- allow *streaming* of large datums
 
Yes, very much +1 there. That's already on the wiki. Yeah:

* Permit lazy fetches of large values, at least out-of-line TOASTED values


- type-level decisions about binary type transport, right now it's a lot
  of effort (including potentially additional roundtrips), to get the
  few important types to be transported in binary fashion. E.g. floating
  points are really expensive to stringify, bytea as text gets a lot
  bigger etc, but a lot of other types don't benefit a lot

Yeah, as distinct from now, where the client has specify param-by-param, and where libpq doesn't support mixing text and binary formats in result sets at all.
 
Again, needs wiki. I'll add.

- annotate COMMIT, PREPARE TRANSACTION, COMMIT PREPARED with LSN of
  associated WAL record

Already on the wiki, as is the related job of sending the xid of a txn to the client when one is assigned.
 
- have a less insane cancellation handling
 
+100

- nested table support


Can you elaborate on that one?


A few other points that come to mind for me are:

* labeled result sets (useful for stored procs, etc, as came up recently with trying to figure out how to let stored procs have OUT params and multiple result sets)

* room for other resultset formats later. Like Damir, I really want to add protobuf or json serializations of result sets at some point, mainly so we can return "entity graphs" in graph representation rather than left-join projection.

* Robert Haas was talking about some issues relating to sync and the COPY BOTH protocol a while ago, which we'd want to address.

--
 Craig Ringer                   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services

pgsql-hackers by date:

Previous
From: Fabien COELHO
Date:
Subject: Re: Re: csv format for psql
Next
From: Teodor Sigaev
Date:
Subject: Re: Cast jsonb to numeric, int, float, bool