Re: HTTP Frontend? (and a brief thought on materialized views) - Mailing list pgsql-hackers

From Dobes Vandermeer
Subject Re: HTTP Frontend? (and a brief thought on materialized views)
Date
Msg-id CADbG_jY_HOfn4LJ-hGxqrc7ww6DxHC93Ndqhp3CKCs27-P2R0Q@mail.gmail.com
Whole thread Raw
In response to Re: HTTP Frontend? (and a brief thought on materialized views)  (Daniel Farina <daniel@heroku.com>)
Responses Re: HTTP Frontend? (and a brief thought on materialized views)  (Daniel Farina <daniel@heroku.com>)
List pgsql-hackers
On Fri, Mar 30, 2012 at 10:55 AM, Daniel Farina <daniel@heroku.com> wrote:
On Thu, Mar 29, 2012 at 6:37 PM, Dobes Vandermeer <dobesv@gmail.com> wrote:
> On Fri, Mar 30, 2012 at 3:59 AM, Daniel Farina <daniel@heroku.com> wrote:
>>
>> On Thu, Mar 29, 2012 at 12:57 PM, Daniel Farina <daniel@heroku.com>
>> wrote:
>>
>> More technical concerns:
>> > * Protocol compression -- but a bit of sand in the gears is *which*
>> > compression -- for database workloads, the performance of zlib can be
>> > a meaningful bottleneck.
>
> I think if performance is the issue, people should use the native protocol.

No. I do not think so. I think a reasonable solution (part of what MS
is actually proposing to the IETF) is to make compression optional, or
have clients support an LZ77 family format like Snappy.  I get the
impression that SPDY is waffling a little on this fact, it mandates
compression, and definitely zlib, but is less heavy handed about
pushing, say Snappy.  Although I can understand why a
Google-originated technology probably doesn't want to push another
Google-originated implementation so hard, it would have been
convenient for me for Snappy to have become a more common format.

> Isn't the URL good enough (/databases/<dbname>) or are you talking about
> having some some of "virtual host" setup where you have multiple sets of
> databases available on the same port?

Virtual hosts. Same port.

In that case, the frontend would not be tied to a specific PostgreSQL server, then?  I think initially this might complicate things a bit, and you could solve it by putting an HTTP proxy in front to do the virtual hosts for you.
 
>> > * A standard frame extension format.  For example, last I checked
>> > Postgres-XC, it required snapshot information to be passed, and it'd
>> > be nice that instead of having to hack the protocol that they could
>> > attach an X-Snapshot-Info header, or whatever. This could also be a
>> > nice way to pass out-of-band hint information of all sorts.
>
>
> I am sorry to admit I don't understand the terms "frame extension format" or
> "Postgres-XC" in this paragraph ... help?

It'd be nice if it wasn't
necessary to do that and they had a much easier path to multiplex
additional information into the connection.

Ah, I get it - you want a way to add some extra information to the protocol in a backwards compatible way.  HTTP (and SPDY) provides a "standard" way to do that.  Makes sense.
 

For my own purposes, I'm intensely interest in lacing the connection with:

* EXPLAIN ANALYZE
* Partition IDs
* Read-only vs. Write workload

I'll make a note of these and hash out the details a bit more once there's something working to add them to.
 
> As for SPDY I can see how it may be helpful but as it is basically a
> different way to send HTTP requests

I think SPDY or like-protocols [...] give a crisp treatment to interactive, stateful workloads involving
back-and-forth between client and server with multiplexing, fixing
some problems with the hacks in HTTP-land from before.

It sounds like at some level you're really talking about replacing the built-in protocol with SPDY because SPDY is possibly a better baseline than updating the existing protocol.  That's an interesting idea, I think this project could evolve in that direction if there's demand for it.

pgsql-hackers by date:

Previous
From: Kyotaro HORIGUCHI
Date:
Subject: Re: Speed dblink using alternate libpq tuple storage
Next
From: Daniel Farina
Date:
Subject: Re: HTTP Frontend? (and a brief thought on materialized views)