Re: Proposal: http2 wire format - Mailing list pgsql-hackers
| From | Craig Ringer |
|---|---|
| Subject | Re: Proposal: http2 wire format |
| Date | |
| Msg-id | CAMsr+YEE3_6eOqxE3mWVKVq+Vy+CBsCX=dHVc5L362nhUP+Z3Q@mail.gmail.com Whole thread Raw |
| In response to | Re: Proposal: http2 wire format (Damir Simunic <damir.simunic@wa-research.ch>) |
| Responses |
Re: Proposal: http2 wire format
(Jacob Champion <pchampion@pivotal.io>)
Re: Proposal: http2 wire format (Damir Simunic <damir.simunic@wa-research.ch>) Re: Proposal: http2 wire format (Vladimir Sitnikov <sitnikov.vladimir@gmail.com>) |
| List | pgsql-hackers |
On 26 March 2018 at 06:00, Damir Simunic <damir.simunic@wa-research.ch> wrote:
> - Overhead for all clients. It may be tiny, but it needs to be
> measured and that cost needs to be weighed against the benefits.
> Maybe a cache miss in the context of a network connection is
> negligible, but we do need to know.
Important point. If h2 is to be seriously considered, then it must be an improvement in absolutely every aspect.
The core part of this proposal is that h2 is parallel to v3. Something one can opt into by compiling `--with_http2`.
IMO, a new protocol intended to supersede an old one must be a core, non-optional feature. It won't reach critical mass of adoption if people can't reasonably rely on it being there. There'll still be a multi-year lead time as versions that support it become widespread enough to interest non-libpq-based driver authors.
My PoC strategy is to touch existing code as little as possible. Yet if the ProcessStartupPacket can somehow return the consumed bytes back to the TLS lib for negotiation, then there’s zero cost to protocol detection for v2/v3 clients and only h2 clients pay the price of the extra check.
As others have noted, you'll want to find a way to handle this in the least SSL-implementation-specific manner possible. IMO if it can't work with OpenSSL, Windows's SSL implementation and OS X's SSL framework it's a non-starter.
> - Dependency on a new external library. Fortunately, it's MIT
> licensed, so it's PostgreSQL compatible, but what happens if it
> becomes unmaintained? This has happened a couple of times, and it
> causes overhead that needs to be taken into account.
I chose nghttp because it gave me a quick start, it’s well designed, a good fit for this kind of work, and fortunately indeed, the license is compatible. (Also, curl links to it as well, so am pretty confident it’ll be around). Very possible that over time h2 parsing code migrates into pg codebase. There are so much similarities to v3 architecture, we might find a way to generalize both into a single codebase. Then h2 frame parser/state machine becomes only a handful of .c files.
h2 is a standard; however you decide to parse it, your code will eventually converge to a stable state in the same manner that febe v3 code did. Once we master the protocol, I don’t think there’ll be much need to touch the framing code. IOW even if we just import what we need, it won’t be a big issue.
While I'm a big fan of code reuse and using existing libraries, I understand others' hesitance here. Look at what happened with ossp-uuid; that was painful and it was just a contrib.
It's a difficult balance between NIH and maintaining a stable core.
* Is there merit in the idea of a completely new v4 protocol—one that freezes the v3 and takes a new path?
Likely so... but it has to be pretty compelling IMO. And more importantly, offer a smooth backwards- and forwards-compatible path.
* What are the criteria for getting this into the core?
Mine would be:
- No new/separate port required. Works on existing port.
- Doesn't break old clients connecting to new servers
- Doesn't break new clients connecting to old servers
- No extra round trips for new client -> old server . I don't personally care about old client -> new server so much, but should be able to offer a pg_hba.conf option to ensure v3 proto only or otherwise prevent extra round trips in this case too.
- Offers significant, concrete benefits and solves the outstanding set of issues with v3 comprehensively
- Offers a really strong extensibility path for client-requested and server-requested optional protocol features as well as protocol version negotiation, with no extra round trips whenever possible.
- Has a wireshark dissector
- Is practical to implement in connection pooler proxies like pgbouncer, pgpool
- Can be made wholly transparent to clients of libpq, i.e. no extra headers or libraries to link
- Works on windows and osx too
- Any libraries used are widespread enough that they're present in at least RHEL7 and Debian Stable. We *can't* just bundle extras in our sources, and packagers are unlikely to be at all happy packaging an extra lib or backport for us. They'll probably just disable the new protocol.
- No regressions for support of SASL / SCRAM, GSSAPI, TLS with X.509 client certs, various other auth methods.
Now, a protocol that cannot satisfy these is IMO not a complete non-starter. It just has to be treated as an optional feature to help out webapps, with quite different design criteria as a result, and cannot be allowed to be as intrusive. Where changes to core protocol logic paths are required it'd have to add plugin mechanisms/hooks instead of adding its own new logic directly.
Make sense?
* Is it better to develop in an experimental fork until the architecture is stable and than patch onto the master, or are we supposed to keep proposing patches for inclusion in the master? Even if not all details are fully fleshed out?
Protocol support doesn't change fast.
I strongly advise you to work on git master at all times, and become familiar with:
- git rebase
- git cherry-pick
- git merge
- git reflog (for when you make mistakes with the above)
Consider maintaining a public git repo with the current working branch. Tag versions if you refer to them in mailing list posts etc, so that people know the exact code you were referring to.
pgsql-hackers by date: