Proposal: http2 wire format - Mailing list pgsql-hackers

Hello hackers,



I’d like to propose the implementation of new wire protocol using http2 framing.

It appears to me that http2 solves many of the issues on the TODO list under “Wire Protocol Changes / v4 Protocol,“
withoutany obvious downsides.  

The implementation I have in mind has zero impact on existing clients. No changes to the format of existing v3
protocol.The new protocol works through a few small additions to postmaster.c to intercept TLS requests, and the rest
innew source files, linked through PQcommMethods. 

I’d like to emphasize that this proposal is empathically NOT about “let’s handle REST in the database” or some such.
It’sabout upgrading the framing, where http2 offers many benefits: content negotiation, concurrent bidirectional
streams,extensible frame types, metadata/data split into headers/trailers and data frames, flow control, etc. It’s at
leastas efficient as febe v3. A lot of research is going into it to make it even more efficient and latency friendly.
Themechanisms it provides for content negotiation, (and with ALPN, protocol negotiation), offers us a future-friendly
wayto evolve without the burden of backward compatibility compromises. 

Before writing this proposal, I set out to create a proof of concept. My goal for the PoC is to be able to connect to
theserver using an existing http2 client and get json back: 

curl -k https://localhost:5432/some_func \
--http2-prior-knowledge --tlsv1.2 \
-H 'pg-database: postgres' \
-H 'pg-user: web'  \
-H ‘authorization: ….’
-H ‘accept: application/json’

{ result: [ … ] }

After spending a week getting up to speed with C, libpq internals, http2 standard, libnghttp2 interface, etc., I’m
fairlyconvinced that pg/http2 is feasible. 

Sadly, my experience with C and Postgres internals is non-existent, and I am not yet able to finalize a live demo. The
abovecurl request does establish the connection, receives the settings frame and queries the database, but I’m still
strugglingwith writing code to return the http2 response. At this stage, it’s purely an issue of mechanically writing
thecode, I think I solved how it all works in principle. 

If anyone finds the idea of Postgres speaking http2 appealing, I’d welcome guidance/mentoring/coding help (or just
plaintaking over). I a put up a repo with the results so far and a longer writeup: https://github.com/dsimunic/pg_h2  

All changes I made to the codebase are in a single commit, hopefully easy to understand what is happening. You’ll need
libnghttp2and openssl 1.0.2 or newer to compile. 

My hope is that this post leads to a conversation and gets a few people excited about the idea the way I am. Maybe even
someof the GSoC students would take the implementation further? 


Damir