On 28 October 2014 23:25, Andres Freund <andres@2ndquadrant.com> wrote:
> On 2014-10-28 20:17:57 +0000, Simon Riggs wrote:
>> On 28 October 2014 17:47, Andres Freund <andres@2ndquadrant.com> wrote:
>> > On 2014-10-28 17:45:36 +0000, Simon Riggs wrote:
>> >> I'd like to avoid all of the pain by making persistent AMs that are
>> >> recoverable after a crash, rather than during crash recovery.
>> >
>> > Besides the actual difficulities of supporting this, imo not being
>> > available on HS and directly after a failover essentially makes them
>> > next to useless.
>>
>> Broken WAL implementations are worse than useless.
...because the indexes are also unavailable during HS.
>> I'm saying we should work on how to fix broken indexes first, before
>> we allow a crop of new code that might cause them.
>
> Why do we presume all of them will be that buggy? And why is that
> different for nbtree, gin, gist? And how is any form of automated
> invalidation changing anything fundamentally?
The current system does not allow for the possibility of a corruption
bug. If one occurs, the only thing an AM can do is PANIC. It has no
mechanism to isolate the problem and deal with it, which affects the
whole server.
So the issue is one of risk of PANIC or data loss - things we have
always taken strong measures against. That is all I have requested as
a first step. And I request it because I remember and dealt with many
bugs and user problems in earlier times of 6-9 years ago.
You are also right: btree, GIN and GIST will benefit from this also.
> To me this is a pretty independent issue.
Initial users of GiST and GIN were rare. The clear target here is
indexing of JSONB data. I don't expect the users of that to be rare. I
expect adoption to be rapid and the effect of bugs to be widespread.
So I see the ability to report bugs and prevent data loss as essential
in the context of new AMs. Automatic fixing of the problem could be
fairly easy, but might be regarded as nice to have, as long as manual
fixing of the problem is easily possible.
I don't regard any of this as an insult or comment that certain people
write buggy code, while others are better people. Everybody has bugs
and WAL code in complex new index types is complex enough that it is
more likely than other places. I salute those who write innovative new
code for Postgres.
-- Simon Riggs http://www.2ndQuadrant.com/PostgreSQL Development, 24x7 Support, Training & Services
