Re: Streaming replication and non-blocking I/O - Mailing list pgsql-hackers

<p>I'm interested in abstracting out features of replication from libpq too. It would be nice if we could implement
differentcommunication bus modules. <p>For example if you have dozens of replicas you may want to use something like
spreadto distribute the records using multicast. <p>Sorry for top posting -- I haven't yet figured out how not to in
thisclient. <p><blockquote type="cite">On 16 Dec 2009 09:54, "Heikki Linnakangas" <<a
href="mailto:heikki.linnakangas@enterprisedb.com">heikki.linnakangas@enterprisedb.com</a>>wrote:<br /><br /><p><font
color="#500050">FujiiMasao wrote: > On Tue, Dec 15, 2009 at 3:47 AM, Heikki Linnakangas >
<heikki.linnakangas@enter...</font>Great!The logical next step is move the handling of TimelineID and<br /> system
identifierout of libpq as well.<br /><br /><br /> I'm thinking of refactoring the protocol along these lines:<br /><br
/>0. Begin by connecting to the master just like a normal backend does. We<br /> don't necessarily need the new
ProtocolVersioncode either, though it's<br /> probably still a good idea to reject connections to older server
versions.<br/><br /> 1. Get the system identifier of the master.<br /><br /> Slave -> Master: Query message, with a
querystring like<br /> "GET_SYSTEM_IDENTIFIER"<br /><br /> Master -> Slave: RowDescription, DataRow CommandComplete,
and<br/> ReadyForQuery messages. The system identifier is returned in the DataRow<br /> message.<br /><br /> This is
identicalto what happens when a query is executed against a<br /> normal backend using the simple query protocol, so
walsendercan use<br /> PQexec() for this.<br /><br /> 2. Another query exchange like above, for timeline ID. (or these
two<br/> steps can be joined into one query, to eliminate one round-trip).<br /><br /> 3. Request a backup history
file,if needed:<br /><br /> Slave -> Master: Query message, with a query string like<br /> "GET_BACKUP_HISTORY_FILE
XXX"where XXX is XLogRecPtr or file name.<br /><br /> Master -> Slave: RowDescription, DataRow CommandComplete
and<br/> ReadyForQuery messages as usual. The file contents are returned in the<br /> DataRow message.<br /><br /><br
/>4. Start replication<br /><br /> Slave -> Master: Query message, with query string "START REPLICATION:<br />
XXXX",where XXXX is the RecPtr of the starting point.<br /><br /> Master -> Slave: CopyOutResponse followed by a
continuousstream of<br /> CopyData messages with WAL contents.<br /><br /><br /> This minimizes the changes to the
protocoland libpq, with a clear way<br /> of extending by adding new commands. Similar to what you did a long time<br
/>ago, connecting as an actual backend at first and then switching to<br /> walsender mode after running a few queries,
butthis would all be<br /> handled in a separate loop in walsender instead of running as a<br /> full-blown backend.
We'llstill need small changes to libpq to allow<br /> sending messages back to the server in COPY_IN mode (maybe add a
new<br/> COPY_IN_OUT mode for that).<br /><br /> Thoughts?<br /><p><font color="#500050"> -- Heikki Linnakangas
EnterpriseDB<a href="http://www.enterprisedb.com">http://www.enterprisedb.com</a> -- </font><p><font
color="#500050">Sentvia pgsql-hackers mailing list (<a
href="mailto:pgsql-hackers@postgresql.org">pgsql-hackers@postgresql.org</a>)To make changes to your
subscript...</font></blockquote>