Thread: About the structure of WAL Files.

I want to write a function to expose the content of WAL Files as streams,
and then send to somewhere else, record by record.



Could you tell me something about the internal structure of the WAL Files?



Thank you very much!

Charlie Wang wrote:
> I want to write a function to expose the content of WAL Files as streams,
> and then send to somewhere else, record by record.

This is not a patch -- please do not write to the pgsql-patches list.
Thanks.

Re: the WAL records, most likely they are useless outside the server
that generated them, because they refer to relations using Oids, and to
specific page addresses which are probably not going to be exactly the
same anywhere else (consider VACUUM commands running at different
times).

--
Alvaro Herrera                                http://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

On Wed, 2006-03-15 at 09:12 -0400, Alvaro Herrera wrote:
> Charlie Wang wrote:
> > I want to write a function to expose the content of WAL Files as streams,
> > and then send to somewhere else, record by record. 
> 
> Re: the WAL records, most likely they are useless outside the server
> that generated them, because they refer to relations using Oids, and to
> specific page addresses which are probably not going to be exactly the
> same anywhere else (consider VACUUM commands running at different
> times).

Yet it should be pointed out that there is a commercial product that
says it uses this information to provide a replication system for
PostgreSQL, so presumably that problem can be solved...

Best Regards, Simon Riggs

Simon Riggs wrote:
> On Wed, 2006-03-15 at 09:12 -0400, Alvaro Herrera wrote:
> > Charlie Wang wrote:
> > > I want to write a function to expose the content of WAL Files as streams,
> > > and then send to somewhere else, record by record. 
> > 
> > Re: the WAL records, most likely they are useless outside the server
> > that generated them, because they refer to relations using Oids, and to
> > specific page addresses which are probably not going to be exactly the
> > same anywhere else (consider VACUUM commands running at different
> > times).
> 
> Yet it should be pointed out that there is a commercial product that
> says it uses this information to provide a replication system for
> PostgreSQL, so presumably that problem can be solved...

Are you referring to Mammoth Replicator?  Let me point out that it
doesn't really use the WAL.

Now that I think of it, maybe the problem _can_ be solved.  Consider
e.g. the WAL-dispatcher resolving the relation Oids to relnames before
shipping, and the WAL-receiver resolving that back to a local Oid before
"applying".  And it only needs to get the tuple contents; it can ignore
the position, and it can resolve the indexing by itself.

-- 
Alvaro Herrera                                http://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support