Re: Re: xReader, double-effort (was: Temporary tables under hot standby) - Mailing list pgsql-hackers

<div class="gmail_extra">Hello Andres,</div><div class="gmail_extra"><br /></div><div class="gmail_extra"><div
class="im"style="style">>> The xReader design was based on the idea that it would be nice not<br />>> to
causeload on the master machine, and that by proxying the WAL<br /> >> stream to the HS, using synchronous
replicationstyle to write from<br />>> xReader to the HS, you could use the HS for a source for that data<br
/>>>with it being at exactly the right point in time to query it.<br /></div><span style="style">>Yes, that
doesmake sense for some workloads. I don't think its viable for</span><br style="style" /><span
style="style">>everythingthough, thats why were not aiming for that ourselves atm.</span> </div><div
class="gmail_extra"><br/></div><div class="gmail_extra">Regarding the above, what would be a case where querying the HS
willnot suffice?<br /><br /><div class="gmail_quote">On Sat, Apr 28, 2012 at 4:02 AM, Andres Freund <span
dir="ltr"><<ahref="mailto:andres@2ndquadrant.com" target="_blank">andres@2ndquadrant.com</a>></span> wrote:<br
/><blockquoteclass="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi Kevin, Hi
Aakash,<br/><div class="im"><br /> On Saturday, April 28, 2012 12:18:38 AM Kevin Grittner wrote:<br /> > Andres
Freund<<a href="mailto:andres@2ndquadrant.com">andres@2ndquadrant.com</a>> wrote:<br /> > > In the current,
prototypal,state there is one component thats<br /> > > integrated into the server (because it needs information
thats<br/> > > only available there).<br /> > The xReader design was based on the idea that it would be nice
not<br/> > to cause load on the master machine, and that by proxying the WAL<br /> > stream to the HS, using
synchronousreplication style to write from<br /> > xReader to the HS, you could use the HS for a source for that
data<br/> > with it being at exactly the right point in time to query it.<br /></div>Yes, that does make sense for
someworkloads. I don't think its viable for<br /> everything though, thats why were not aiming for that ourselves
atm.<br/><div class="im"><br /> > I'm not convinced that I would rather see the logic fixed inside the<br /> >
masteras opposed to being deployable on the master's machine, the<br /> > slave machine, or even on its own machine
inbetween.<br /></div>I don't think that you can do everything apart from the master. We currently<br /> need shared
memoryfor coordination between the moving parts, thats why we<br /> have it inside the master.<br /> It also have the
advantageof being easier to setup.<br /><div class="im"><br /> > > That component is layered ontop of a totally
genericxlog<br /> > > reading/parsing library that doesn't care at all where its<br /> > > running.<br />
>That's cool.<br /><br /> > > Its also used in another cluster to read the received (filtered)<br /> > >
stream.<br/> > I don't quite follow what you're saying there.<br /></div>To interpret the xlog back into something
thatcan be used for replication you<br /> need to read it again. After filtering we again write valid WAL, so we can
use<br/> the same library on the sending|filtering side and on the receiving side.<br /> But thats actually off topic
forthis thread ;)<br /><div class="im"><br /><br /> > > I took a *very* short glance over the current wiki
descriptionof<br /> > > xReader and from that it seems to me it would benefit from trying<br /> > > to make
itarchitecturally more similar to the rest of pg.<br /> > We're planning on using existing protocol to talk between
pieces.<br/> > Other than breaking it out so that it can run somewhere other than<br /> > inside the server, and
allowingclients to connect to xReader to<br /> > listen to WAL events of interest, are you referring to anything<br
/>> else?<br /></div>It sounds like the xReader is designed to be one multiplexing process. While<br /> this
definitelyhas some advantages resource-usage-wise it doesn't seem to be<br /> fitting the rest of the design that well.
Theadvantages might outweigh<br /> everything else, but I am not sure about that.<br /> Something like
registering/deregisteringalso doesn't fit that well with the<br /> way walsender works as far as I understand it.<br
/><br/> Greetings,<br /><div class="HOEnZb"><div class="h5"><br /> Andres<br /> --<br />  Andres Freund                
   <a href="http://www.2ndQuadrant.com/" target="_blank">http://www.2ndQuadrant.com/</a><br />  PostgreSQL Development,
24x7Support, Training & Services<br /></div></div></blockquote></div><br /></div>