Re: intercepting WAL writes - Mailing list pgsql-hackers

From Hannu Krosing
Subject Re: intercepting WAL writes
Date
Msg-id 1212044273.7129.12.camel@huvostro
Whole thread Raw
In response to intercepting WAL writes  ("Mike" <mike@fonolo.com>)
Responses Re: intercepting WAL writes
List pgsql-hackers
On Wed, 2008-05-28 at 19:11 -0400, Mike wrote:

> 
> Can somebody point to the most logical place in the code to intercept
> the WAL writes? (just a rough direction would be enough)- or if this
> doesn’t make sense at all, another suggestion on where to get the
> data? 

I don't think that intercepting (and then decoding ) WAL is very
productive. It is too low level to be of much help.

The way I'd do it would be using pgQ from SkyTools package where change
events can be queued when happening and then moved in bulk to memcached
with not too much effort.

Marko Kreen, the primary author of pgQ did a presentation on it at this
years pgcon :

http://www.pgcon.org/2008/schedule/events/79.en.html

You can download the presentation slides to get the overview.


> (I’m trying to avoid doing it using triggers).

I can't see a good point in avoiding triggers, if you end up doing a lot
more work to reconstruct the data which would have been easy to get
using a trigger.

I've heard that there was and add-on replication solution for Oracle
which extracted data from WAL and I was told that it maxed out at about
200 tps. pgQ approach can do much better.

---------------
Hannu




pgsql-hackers by date:

Previous
From: Hannu Krosing
Date:
Subject: Re: BUG #4204: COPY to table with FK has memory leak
Next
From: Sabbiolina
Date:
Subject: [PERFORM] Memory question on win32 systems