I wrote:
> ... What I think is happening is that there's some inadequately
> locked page-changing logic in there, and if you can get autovacuum to
> happen on msginfo concurrently with the data reload step, it'll mess up.
> Given that we've not seen reports of this on 8.3, I'll bet lunch that
> the bug is somewhere in the "fast insert" code. I haven't found it
> yet, though.
Well, I found at least part of the problem:
http://archives.postgresql.org/message-id/20090606023940.BD4B875331E@cvs.postgresql.org
This is in perfectly sequential code. The reason it has
nondeterministic effects is that (so far as I can tell) the only
non-crash case where there would be duplicate TIDs to eliminate is if
two backends are concurrently flushing an index's pending-inserts list.
The code is designed to let that happen and suppress the duplicates at
the very end of the process, in addItemPointersToTuple(); but the
duplicate-removal logic was broken and allowed extra garbage TIDs to
creep in. So at least in the case I'm testing, this happens when
autovacuum fires on the table concurrently with a large insertion.
Please update to CVS HEAD, reindex that index, and then see if you see
any more strange behavior. I'm not entirely convinced that this is the
only problem ...
regards, tom lane
