On Fri, Mar 11, 2016 at 9:09 AM, Bruce Momjian <bruce@momjian.us> wrote:
3. I have tried to encourage others to get involved, with limited success. I do think the FDW is perhaps the only reasonable way to get _built-in_ sharding. The external sharding solutions are certainly viable, but external. It is possible we will make all the FDW improvements, find out it doesn't work, but find out the improvements allow us to go in another direction.
I remember last summer emails and we really wanted to participate in development, but it happens all slots were occupied by edb and ntt people. We wanted to work on distributed transactions and proposed our XTM. Our feeling that time from discussion was that we were invited, but all doors were closed. It was very bad experience. Hopefully, we understand our misunderstanding.
There seems to be serious interest in how this idea came about, so let me say what I remember.
I think the idea was so obvious, so let's don't discuss this.
As for why there is so much hostility, I think this is typical for any ill-defined feature development. There was simmering hostility to the Windows port and pg_upgrade for many years because those projects were not easy to define and risky, and had few active developers. The agreement was that work could continue as long as destabilization wasn't introduced. Ideally everything would have a well-defined plan, it is sometimes hard to do. Similar to our approach on parallelism (which is also super-important and doesn't many active developers), sometimes you just need to create infrastructure and see how well it solves problems.
Our XTM is the yet another example of infrastructure we need to work on clustering. Should we wait other smart guy starts thinking on distributed transactions ? We described in https://wiki.postgresql.org/wiki/DTM our API, which is just a wrapper on existed functions, but it will allow us and fortunately others to play with their ideas. We did several prototypes, including FDW, to demonstrate viability of API, and plan to continue our work on built-in high availability, multi-master. Of course, there will be a lot to learn, but it will be much easier if XTM will exists not as separate patch, which is really small.
