Putting the split-in-progress flag in the new bucket's primary page makes a lot of sense. I don't have any problem with dumping the rest of it for a first cut if we have a different long-term plan for how to improve concurrency, but I don't see much point in going to a lot of work to implement a system for WAL logging if we're going to end up having to afterwards throw it out and start from scratch to get rid of the heavyweight locks - and it's not obvious to me how what you have here could be extended to do that.
+1 I don't think we have to improve concurrency at the same time as WAL logging, but we at least have to implement WAL logging in a way that doesn't foreclose future improvements to concurrency.
I've been tempted to implement a new type of hash index that allows both WAL and high concurrency, simply by disallowing bucket splits. At the index creation time you use a storage parameter to specify the number of buckets, and that is that. If you mis-planned, build a new index with more buckets, possibly concurrently, and drop the too-small one.
I think it would be easier to add bucket splitting to something like this than it would be to add WAL logging and concurrency to what we already have--mostly because I think that route would be more amenable to incremental programming efforts, and also because if people had an actually useful hash index type they would use it and see that it worked well (assuming of course that it does work well), and then be motivated to improve it.
If this could be done as an extension I would just go (attempt to) do it. But since WAL isn't pluggable, it can't go in as an extension.
