Thread: PD_ALL_VISIBLE flag error on 9.0 alpha 4
All, What I did: 1. Set up 9.0a4 doing SR replication with a 2nd 9.0a4 2. Ran pgbench for a while. 3. Aborted pgbench with Ctl-C 4. Changed vacuum_defer_cleanup_age in postgresql.conf and reloaded 5. Ran pgbench again, and got: Sidney-Stratton:pg90 josh$ pgbench -c 2 -T 300 bench starting vacuum...WARNING: PD_ALL_VISIBLE flag was incorrectly set in relation "pgbench_branches" page 0 WARNING: PD_ALL_VISIBLE flag was incorrectly set in relation "pgbench_branches" page 1 WARNING: PD_ALL_VISIBLE flag was incorrectly set in relation "pgbench_tellers" page 0 WARNING: PD_ALL_VISIBLE flag was incorrectly set in relation "pgbench_tellers" page 1 ... not one I'm familiar with. Issues? --Josh Berkus
On Tue, 2010-03-09 at 22:02 -0800, Josh Berkus wrote: > 1. Set up 9.0a4 doing SR replication with a 2nd 9.0a4 > 2. Ran pgbench for a while. > 3. Aborted pgbench with Ctl-C > 4. Changed vacuum_defer_cleanup_age in postgresql.conf and reloaded > 5. Ran pgbench again, and got: > > Sidney-Stratton:pg90 josh$ pgbench -c 2 -T 300 bench > starting vacuum...WARNING: PD_ALL_VISIBLE flag was incorrectly set in > relation "pgbench_branches" page 0 > WARNING: PD_ALL_VISIBLE flag was incorrectly set in relation > "pgbench_branches" page 1 > WARNING: PD_ALL_VISIBLE flag was incorrectly set in relation > "pgbench_tellers" page 0 > WARNING: PD_ALL_VISIBLE flag was incorrectly set in relation > "pgbench_tellers" page 1 Understandable. Time to remove vacuum_defer_cleanup_age, I think. -- Simon Riggs www.2ndQuadrant.com
Simon Riggs wrote: > On Tue, 2010-03-09 at 22:02 -0800, Josh Berkus wrote: > >> 1. Set up 9.0a4 doing SR replication with a 2nd 9.0a4 >> 2. Ran pgbench for a while. >> 3. Aborted pgbench with Ctl-C >> 4. Changed vacuum_defer_cleanup_age in postgresql.conf and reloaded >> 5. Ran pgbench again, and got: >> >> Sidney-Stratton:pg90 josh$ pgbench -c 2 -T 300 bench >> starting vacuum...WARNING: PD_ALL_VISIBLE flag was incorrectly set in >> relation "pgbench_branches" page 0 >> WARNING: PD_ALL_VISIBLE flag was incorrectly set in relation >> "pgbench_branches" page 1 >> WARNING: PD_ALL_VISIBLE flag was incorrectly set in relation >> "pgbench_tellers" page 0 >> WARNING: PD_ALL_VISIBLE flag was incorrectly set in relation >> "pgbench_tellers" page 1 > > Understandable. > > Time to remove vacuum_defer_cleanup_age, I think. Umm, so what's the bug? -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com
On Wed, 2010-03-10 at 23:08 +0200, Heikki Linnakangas wrote: > > > > Time to remove vacuum_defer_cleanup_age, I think. > > Umm, so what's the bug? Whether you call it a bug or just an annoyance is debatable, but the source of it is clear. Given the lack of effectiveness, I propose removing it. Would you agree or disagree with the suggested removal? -- Simon Riggs www.2ndQuadrant.com
Simon Riggs <simon@2ndQuadrant.com> writes: >>> Time to remove vacuum_defer_cleanup_age, I think. >> >> Umm, so what's the bug? > Whether you call it a bug or just an annoyance is debatable, but the > source of it is clear. Maybe to you, but the rest of us would like to know. > Given the lack of effectiveness, I propose > removing it. I read Josh's recent report at http://archives.postgresql.org/message-id/4B973C3F.9070501@agliodbs.com to say that it's quite effective. I think you're being way too hasty to decide that it can just be dropped. regards, tom lane
On Wed, 2010-03-10 at 17:55 -0500, Tom Lane wrote: > Simon Riggs <simon@2ndQuadrant.com> writes: > >>> Time to remove vacuum_defer_cleanup_age, I think. > >> > >> Umm, so what's the bug? > > > Whether you call it a bug or just an annoyance is debatable, but the > > source of it is clear. > > Maybe to you, but the rest of us would like to know. If vacuum_defer_cleanup_age is set higher this causes the xmin to go backwards, leading to the "PD_ALL_VISIBLE flag was incorrectly set" warning. Having this false xmin move backwards doesn't endanger the standby, since the xids arrive and are checked normally. If they stop arriving that is fine. Having the false xmin going backwards is not a serious issue on primary because the actual xmin does not go backwards. No observer loses information as a result of this, it is only about whether cleanup records are generated later than normal, or not. > > Given the lack of effectiveness, I propose > > removing it. > > I read Josh's recent report at > http://archives.postgresql.org/message-id/4B973C3F.9070501@agliodbs.com > to say that it's quite effective. I think you're being way too hasty to > decide that it can just be dropped. OK, that's enough to not remove it. I was aware of more negative thoughts and conscious of my own feelings about it being a kluge. -- Simon Riggs www.2ndQuadrant.com
On 3/10/10 3:26 PM, Simon Riggs wrote: > OK, that's enough to not remove it. I was aware of more negative > thoughts and conscious of my own feelings about it being a kluge. Well, it *is* a kludge, but it may be the best one for people who want to use HS/SR to support web applications. So I think we should work on making it less kludgy. Ultimately we're going to need publish-XID-to-master, but that's not realistic for 9.0. --Josh Berkus
On Wed, 2010-03-10 at 17:12 -0800, Josh Berkus wrote: > On 3/10/10 3:26 PM, Simon Riggs wrote: > > OK, that's enough to not remove it. I was aware of more negative > > thoughts and conscious of my own feelings about it being a kluge. > > Well, it *is* a kludge, but it may be the best one for people who want > to use HS/SR to support web applications. So I think we should work on > making it less kludgy. That's better, I was worried you'd gone all complimentary on me. > Ultimately we're going to need publish-XID-to-master, but that's not > realistic for 9.0. Still some open items that I'm working on, but not that. -- Simon Riggs www.2ndQuadrant.com
> That's better, I was worried you'd gone all complimentary on me. <grin>Never fear that! Was that setting originally part of your design for HS? If so, why did you back off from it? --Josh
On Sat, 2010-03-13 at 11:29 -0800, Josh Berkus wrote: > > That's better, I was worried you'd gone all complimentary on me. > > <grin>Never fear that! > > Was that setting originally part of your design for HS? If so, why did > you back off from it? We all agreed its a kluge, that's why. It's also my 3rd choice of solution behind fine-grained lock conflicts (1st) which would avoid many issues and master/standby in lock step (2nd). Having said that I'm much in favour of providing a range of options and then letting users tell us what works for them. -- Simon Riggs www.2ndQuadrant.com
> It's also my 3rd choice of solution behind fine-grained lock conflicts > (1st) which would avoid many issues and master/standby in lock step > (2nd). Yeah, I just can't imagine you hunting down all of the corner cases for fine-grained lock conflicts in time for 9.0. Given what I've been looking at, it seems like a LOT of work. --Josh Berkus