Re: xReader, double-effort - Mailing list pgsql-hackers

Greg Smith <greg@2ndQuadrant.com> wrote:
> On 04/26/2012 12:57 PM, Robert Haas wrote:
>> I don't recall seeing any previous postings about the 2Q project,
>> either.  We should try to keep these discussions on-list.
> 
> First brought up two months ago to the list it seemed most
> relevant to at the time: 
>
http://archives.postgresql.org/pgsql-cluster-hackers/2012-02/msg00001.php
> and the solid commentary started coming out yesterday, right near
> the approach of May as promised.  There hasn't been much posted on
> hackers about this out of respect for the extended CommitFest.
> 
> I personally suggested Aakash vent ideas out on pgsql-students (or
> here) at the beginning of the month: 
> http://archives.postgresql.org/pgsql-students/2012-04/msg00002.php
> but the next time I heard about the subject was the acceptance
> notice.
That's largely my fault.  Aakash's original proposal was naively
assuming that logical records were available in our xlog stream and
wanted to work on tools to do something useful with that data. 
There was some confusion among the various core team members,
committers, and other contributors who volunteered to review the
proposals about whether he was talking about analyzing the server
log or xlog.  We had a wide-ranging discussion on the freenode
#postgresql channel about it, and brainstormed to the current idea
-- at which point Aakash had about a week and a half to rewrite his
proposal along the lines discussed on irc.  A week after that the
acceptance notices came out.
If we had had any idea that 2Q was working on something along these
lines, we would have coordinated or guided him in another direction.
The post you cite from a couple months ago was way too vague to
have even an inkling that it had anything to do with providing feeds
of the logical interpretation of the WAL stream.  I don't recall
having heard the idea even mentioned before.
The timing in regard to the release process seems to have had an
effect on both groups here, since I suggested he read up on
walsender and walreceiver and a few other areas, put together a Wiki
page with the plan, and *then* describe the preliminary plan to the
community (a month or so before starting to code), so that details
could be hashed out in public before time was spent coding.  Had
there not been so many people heads-down trying to get the release
out, I might have advised him to post a hand-wavy description of the
effort before he got the Wiki page set up and reviewed the relevant
code.
As Josh points out, his first choice was to write utilities to *do*
something with the logical xlog information, and he only switched
over to working on *producing* it when he heard that no such thing
was currently available.  I think there's a lot of room to
coordinate these efforts so that there *isn't* a waste of effort as
long as communication is open enough.
-Kevin