Re: Re: xReader, double-effort (was: Temporary tables under hot standby) - Mailing list pgsql-hackers

<div class="gmail_extra"><span style="style">> Aakash, when you get a chance, could you fill in the
"inch-stones"</span><brstyle="style" /><span style="style">> from the GSoC proposal page onto the Wiki
page?</span></div><divclass="gmail_extra"><font color="#222222" face="arial, sans-serif"><br /></font></div><div
class="gmail_extra">Sure, <a
href="http://wiki.postgresql.org/wiki/XReader">http://wiki.postgresql.org/wiki/XReader</a> updated.<fontcolor="#222222"
face="arial,sans-serif"><br /></font><br /><div class="gmail_quote">On Sat, Apr 28, 2012 at 3:48 AM, Kevin Grittner
<spandir="ltr"><<a href="mailto:Kevin.Grittner@wicourts.gov"
target="_blank">Kevin.Grittner@wicourts.gov</a>></span>wrote:<br /><blockquote class="gmail_quote" style="margin:0 0
0.8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="im">Andres Freund <<a
href="mailto:andres@2ndquadrant.com">andres@2ndquadrant.com</a>>wrote:<br /><br /> > In the current, prototypal,
statethere is one component thats<br /> > integrated into the server (because it needs information thats<br /> >
onlyavailable there).<br /><br /></div>The xReader design was based on the idea that it would be nice not<br /> to
causeload on the master machine, and that by proxying the WAL<br /> stream to the HS, using synchronous replication
styleto write from<br /> xReader to the HS, you could use the HS for a source for that data<br /> with it being at
exactlythe right point in time to query it.<br /><br /> I'm not convinced that I would rather see the logic fixed
insidethe<br /> master as opposed to being deployable on the master's machine, the<br /> slave machine, or even on its
ownmachine in between.<br /><div class="im"><br /> > That component is layered ontop of a totally generic xlog<br />
>reading/parsing library that doesn't care at all where its<br /> > running.<br /><br /></div>That's cool.<br
/><divclass="im"><br /> > Its also used in another cluster to read the received (filtered)<br /> > stream.<br
/><br/></div>I don't quite follow what you're saying there.<br /><div class="im"><br /> > I plan to submit the
XLogReader(thats what its called atm)<br /> > before everything else, so everybody can take a look as soon as<br />
>possible.<br /><br /></div>Great!  That will allow more discussion and planning.<br /><div class="im"><br /> > I
tooka *very* short glance over the current wiki description of<br /> > xReader and from that it seems to me it would
benefitfrom trying<br /> > to make it architecturally more similar to the rest of pg.<br /><br /></div>We're
planningon using existing protocol to talk between pieces.<br /> Other than breaking it out so that it can run
somewhereother than<br /> inside the server, and allowing clients to connect to xReader to<br /> listen to WAL events
ofinterest, are you referring to anything<br /> else?<br /><div class="im"><br /> > I also would suggest reviewing
howthe current walreceiver/sender,<br /> > and their protocol, work.<br /><br /></div>Of course!  The first
"inch-stone"in the GSoC project plan<br /> basically consists of creating an executable that functions as a<br />
walreceiverand a walsender to just pass things through from the<br /> master to the slave.  We build from there by
allowingclients to<br /> connect (again, over existing protocol) and register for events of<br /> interest, and then
recognizingdifferent WAL records to generate<br /> events.  The project was just going to create a simple client to<br
/>dump the information to disk, but with the time saved by adopting<br /> what you've already done, that might leave
moretime for generating<br /> a useful client.<br /><br /> Aakash, when you get a chance, could you fill in the
"inch-stones"<br/> from the GSoC proposal page onto the Wiki page?  I think the<br /> descriptions of those interim
stepswould help people understand<br /> your proposal better.  Obviously, some of the particulars of tasks<br /> and
thedates may need adjustment based on the new work which is<br /> expected to appear before you start, but what's there
nowwould be a<br /> helpful reference.<br /><span class="HOEnZb"><font color="#888888"><br /> -Kevin<br
/></font></span></blockquote></div><br/></div>