Thread: Re: [GENERAL] [streaming replication] 9.1.3 streaming replication bug ?
On 4/11/12, 乔志强 <qiaozhiqiang@leadcoretech.com> wrote: > >> Yes, increase wal_keep_segments. Even if you set wal_keep_segments to 64, >> the amount of disk space for WAL files is only 1GB, so there is no need to >> worry so much, I think. No? > > But when a transaction larger than 1GB... Then you may need WAL space larger than 1GB as well. For replication to work, it seems likely that you may need to have sufficient WAL space to handle a row, possibly the entire transaction.. But since a single statement can update thousands or millions of rows, do you always need enough WAL space to hold the entire transaction? > So in sync streaming replication, if master delete WAL before sent to the > only standby, all transaction will fail forever, > "the master tries to avoid a PANIC error rather than termination of > replication." but in sync replication, termination of replication is THE > bigger PANIC error. That's somewhat debatable. Would I rather have a master that PANICED or a slave that lost replication? I would choose the latter. A third option, which may not even be feasible, would be to have the master fail the transaction if synchronous replication cannot be achieved, although that might have negative consequences as well. > Another question: > Does master send WAL to standby before the transaction commit ? That's another question for the core team, I suspect. A related question is what happens if there is a rollback? -- Mike Nolan
Michael Nolan <htfoot@gmail.com> wrote: > On 4/11/12, 乔志强 <qiaozhiqiang@leadcoretech.com> wrote: >> But when a transaction larger than 1GB... > > Then you may need WAL space larger than 1GB as well. For > replication to work, it seems likely that you may need to have > sufficient WAL space to handle a row, possibly the entire > transaction.. But since a single statement can update thousands > or millions of rows, do you always need enough WAL space to hold > the entire transaction? No. >> Does master send WAL to standby before the transaction commit ? Yes. > A related question is what happens if there is a rollback? PostgreSQL doesn't use a rollback log; WAL files can be reclaimed as soon as the work they represent has been persisted to the database by a CHECKPOINT, even if it is not committed. Because there can be multiple versions of each row in the base table, each with its own xmin (telling which transaction committed it) and xmax (telling which transaction expired it) visibiliity checking can handle the commits and rollbacks correctly. It also uses a commit log (CLOG), hint bits, and other structures to help resolve visibility. It is a complex topic, but it does work. -Kevin
On 4/11/12, Kevin Grittner <Kevin.Grittner@wicourts.gov> wrote: > Michael Nolan <htfoot@gmail.com> wrote: >> On 4/11/12, 乔志强 <qiaozhiqiang@leadcoretech.com> wrote: > >>> But when a transaction larger than 1GB... >> >> Then you may need WAL space larger than 1GB as well. For >> replication to work, it seems likely that you may need to have >> sufficient WAL space to handle a row, possibly the entire >> transaction.. But since a single statement can update thousands >> or millions of rows, do you always need enough WAL space to hold >> the entire transaction? > > No. > >>> Does master send WAL to standby before the transaction commit ? > > Yes. > >> A related question is what happens if there is a rollback? > > PostgreSQL doesn't use a rollback log; WAL files can be reclaimed as > soon as the work they represent has been persisted to the database > by a CHECKPOINT, even if it is not committed. Because there can be > multiple versions of each row in the base table, each with its own > xmin (telling which transaction committed it) and xmax (telling > which transaction expired it) visibiliity checking can handle the > commits and rollbacks correctly. It also uses a commit log (CLOG), > hint bits, and other structures to help resolve visibility. It is a > complex topic, but it does work. Thanks, Kevin. That does lead to a question about the problem that started this thread, though. How does one determine how big the WAL space needs to be to not cause streaming replication to fail? Or maybe this is a bug after all? -- Mike Nolan