Re: Synchronous Log Shipping Replication - Mailing list pgsql-hackers

From Markus Wanner
Subject Re: Synchronous Log Shipping Replication
Date
Msg-id 48C51026.2090003@bluegap.ch
Whole thread Raw
In response to Re: Synchronous Log Shipping Replication  (ITAGAKI Takahiro <itagaki.takahiro@oss.ntt.co.jp>)
Responses Re: Synchronous Log Shipping Replication  ("Fujii Masao" <masao.fujii@gmail.com>)
List pgsql-hackers
Hi,

ITAGAKI Takahiro wrote:
>   1. Is process-switching approach the best way to share one socket?
>         Both Postgres-R and the log-shipping prototype use the approach now.
>         Can I think there is no objection here?

I don't see any appealing alternative. The postmaster certainly 
shouldn't need to worry with any such socket for replication. Threading 
falls pretty flat for Postgres. So the socket must be held by one of the 
child processes of the Postmaster.

>   2. If 1 is reasonable, how should we add a new WAL sender process?
>         Just add a new process using a core-patch?

Seems feasible to me, yes.

>         Merge into WAL writer?

Uh.. that would mean you'd loose parallelism between WAL writing to disk 
and WAL shipping via network. That does not sound appealing to me.

>         Consider framework to add any of user-defined auxiliary process?

What for? What do you miss in the existing framework?

>   3. If 1 is reasonable, what should we use for the process-switching
>      primitive?
>         Postgres-R uses signals and locking and the log-shipping prototype
>         uses multi-threads and POSIX message queues now.

AFAIK message queues are problematic WRT portability. At least Postgres 
doesn't currently use them and introducing dependencies on those might 
lead to problems, but I'm not sure. Others certainly know more about 
issues involved.

A multi-threaded approach is certainly out of bounds, at least within 
the Postgres core code.

> Signals and locking is possible choice for 3, but I want to use better
> approach if any. Faster is always better.

I think the approach can reach better throughput than POSIX message 
queues or unix pipes, because of the mentioned savings in copying around 
between system and application memory. However, that hasn't been proved, 
yet.

> I guess we could invent a new semaphore-like primitive at the same layer
> as LWLocks using spinlock and PGPROC directly...

Sure, but in what way would that differ from what I do with imessages?

Regards

Markus Wanner



pgsql-hackers by date:

Previous
From: Devrim GÜNDÜZ
Date:
Subject: Re: [GENERAL] New shapshot RPMs (Sep 7 2008) are ready for testing
Next
From: Tom Lane
Date:
Subject: Re: reducing statistics write overhead