Thread: Walsender process patch v1 for Synch Rep
Hi, > To be reviewed easily, I'm splitting Synch Rep patch into some pieces. Attached is a patch only to start and manage walsender process. This patch don't include replication itself, walreceiver and signal handling patch, etc. The outline of this patch is as follow: Authentication ----------------------- As pointed out at another thread, for authentication, I defined the database only for replication (named "walsender" tentatively). walsender database is not pseudo but created by initdb like postgres database, because the user can re-create it easily even if it is lost accidentally. If the startup packet specifies walsender database, a backend declares postmaster working as walsender. Then, the backend is removed from BackendList and managed as background process by postmaster. Replication message --------------------------------- I defined new message type 'R', which means the start of replication. If the message is received, walsender will perform the main routine. Of course, a backend who is not walsender cannot perform the routine. Shutdown ---------------- I arranged the shutdown timing of walsender. For example, in smart shutdown case, walsender should exit after bgwriter at least in order to replicate a shutdown checkpoint xlog. Initialization -------------------- In the main routine, walsender sets up signal handlers, etc again. And some bug fixes. Regards, -- Fujii Masao NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center
Attachment
On Wed, 2008-11-05 at 23:17 +0900, Fujii Masao wrote: > Authentication > ----------------------- > As pointed out at another thread, for authentication, I defined the database > only for replication (named "walsender" tentatively). walsender database is > not pseudo but created by initdb like postgres database, because the user > can re-create it easily even if it is lost accidentally. > > If the startup packet specifies walsender database, a backend declares > postmaster working as walsender. Then, the backend is removed from > BackendList and managed as background process by postmaster. > > Replication message > --------------------------------- > I defined new message type 'R', which means the start of replication. If the > message is received, walsender will perform the main routine. Of course, > a backend who is not walsender cannot perform the routine. I don't understand why you've done it this way. Can you explain? This stuff about a walsender database sounds like it has a purpose, but I'm not sure what it is. The route I would have taken would be to add walsender and walreceiver as new auxiliary processes. They would start via AuxiliaryProcessMain() in bootstrap/bootstrap.c. ISTM this would be slightly less code as well and not too much change from what you have now. After a quick look, most of the rest of the patch looks correct. I would hope that walsender and walreceiver would start like that. walsender would start at same time as walwriter. walreceiver can start earlier, for later discussion. > Shutdown > ---------------- > I arranged the shutdown timing of walsender. For example, in smart > shutdown case, walsender should exit after bgwriter at least in order to > replicate a shutdown checkpoint xlog. Agreed. -- Simon Riggs www.2ndQuadrant.comPostgreSQL Training, Services and Support
Hi, Simon Thank you for the review. On Fri, Nov 7, 2008 at 5:49 PM, Simon Riggs <simon@2ndquadrant.com> wrote: > > On Wed, 2008-11-05 at 23:17 +0900, Fujii Masao wrote: > >> Authentication >> ----------------------- >> As pointed out at another thread, for authentication, I defined the database >> only for replication (named "walsender" tentatively). walsender database is >> not pseudo but created by initdb like postgres database, because the user >> can re-create it easily even if it is lost accidentally. >> >> If the startup packet specifies walsender database, a backend declares >> postmaster working as walsender. Then, the backend is removed from >> BackendList and managed as background process by postmaster. >> >> Replication message >> --------------------------------- >> I defined new message type 'R', which means the start of replication. If the >> message is received, walsender will perform the main routine. Of course, >> a backend who is not walsender cannot perform the routine. > > I don't understand why you've done it this way. Can you explain? This > stuff about a walsender database sounds like it has a purpose, but I'm > not sure what it is. > > The route I would have taken would be to add walsender and walreceiver > as new auxiliary processes. They would start via AuxiliaryProcessMain() > in bootstrap/bootstrap.c. ISTM this would be slightly less code as well > and not too much change from what you have now. After a quick look, most > of the rest of the patch looks correct. > > I would hope that walsender and walreceiver would start like that. > walsender would start at same time as walwriter. walreceiver can start > earlier, for later discussion. Yeah, I also add walsender as new auxiliary process at first. But, during coding, I made out that this is more complicated for code and user. First problem : Which process accepts the connection from standby? IMO, *postmaster* should accept it like normal database access. Since we can use the existing connection establishment logic, the change of the code is smaller and it's easier to use. So, I added walsender as a special backend which is forked when standby connects to postmaster. Is there any advantage which walsender or other processes accept the connection from standby? Second problem : What should walsender do after the termination of the connection from standby? should die?, or remain alive and wait for next connection? IMO, we should handle it like normal database access, i.e. exit walsender. This and adding walsender as an auxiliary process seldom meet, I think. Does that answer you? Am I missing something? Regards, -- Fujii Masao NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center
On Mon, 2008-11-10 at 18:22 +0900, Fujii Masao wrote: > Yeah, I also add walsender as new auxiliary process at first. But, > during coding, > I made out that this is more complicated for code and user. > > First problem : Which process accepts the connection from standby? > IMO, *postmaster* should accept it like normal database access. Since > we > can use the existing connection establishment logic, the change of the > code > is smaller and it's easier to use. So, I added walsender as a special > backend > which is forked when standby connects to postmaster. Is there any > advantage > which walsender or other processes accept the connection from standby? > Second problem : What should walsender do after the termination of the > connection from standby? should die?, or remain alive and wait for > next > connection? IMO, we should handle it like normal database access, i.e. > exit walsender. This and adding walsender as an auxiliary process > seldom > meet, I think. > > Does that answer you? Am I missing something? It's good to see your reasons written down. OK, I think I could like this way around. The "walsender" database allows us to enforce restrictions in pg_hba.conf. Also avoids needing to run a long running transaction to initiate wal sending feature, if we do it just with user function. I'd like to see a longer README explaining these design aspects though. -- Simon Riggs www.2ndQuadrant.comPostgreSQL Training, Services and Support
On Tue, Nov 11, 2008 at 9:12 AM, Simon Riggs <simon@2ndquadrant.com> wrote: > > On Mon, 2008-11-10 at 18:22 +0900, Fujii Masao wrote: > >> Yeah, I also add walsender as new auxiliary process at first. But, >> during coding, >> I made out that this is more complicated for code and user. >> >> First problem : Which process accepts the connection from standby? >> IMO, *postmaster* should accept it like normal database access. Since >> we >> can use the existing connection establishment logic, the change of the >> code >> is smaller and it's easier to use. So, I added walsender as a special >> backend >> which is forked when standby connects to postmaster. Is there any >> advantage >> which walsender or other processes accept the connection from standby? > >> Second problem : What should walsender do after the termination of the >> connection from standby? should die?, or remain alive and wait for >> next >> connection? IMO, we should handle it like normal database access, i.e. >> exit walsender. This and adding walsender as an auxiliary process >> seldom >> meet, I think. >> >> Does that answer you? Am I missing something? > > It's good to see your reasons written down. > > OK, I think I could like this way around. The "walsender" database > allows us to enforce restrictions in pg_hba.conf. Also avoids needing to > run a long running transaction to initiate wal sending feature, if we do > it just with user function. I'd like to see a longer README explaining > these design aspects though. OK, thanks. I'll try to write them. -- Fujii Masao NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center