Re: Pglogical questions and problems - Mailing list pgsql-hackers

On 14/04/16 20:46, Robert Haas wrote:
> I believe
> logical replication is a fundamental database technology that should
> be considered just as much within the score of the core product as
> physical replication, parallel query, or UPSERT.

Agreed, I believed we need this for very long time as well (pglogical is 
not my first or even second logical replication project).

>
>> At present, I don't understand why we would do sharding via FDWs, i.e. an
>> out-of-core solution and yet replication as an in-core solution. Sharding
>> desires/requires a single system image, so tight coupling is sensible (for
>> example, defining a distribution key column on a distributed table). For
>> replication between disparate loosely coupled systems, tight coupling is
>> exactly what you do not want. So doing it that way round would give an an
>> out-of-core solution for something that is best done in-core and an in-core
>> solution for something best done out-of-core.
>
> First, I think that replication can be either loosely-coupled or
> tightly-coupled.  There are interesting cases with intermittently
> connected networks where you really don't want too much coupling, and
> then there are cases where you are doing load-balancing across a
> cluster and tight coupling is fine, even desirable.   Similarly,
> although I agree that a sharding solution intrinsically requires
> fairly tight coupling, I think that one of the strengths of FDWs is
> that they do not.  I'm not very interested in seeing a sharding
> solution in PostgreSQL that limits what you can do to a particular
> network topology and enforces tight coupling whether you want it or
> not.  I'm more interested in seeing how we can build something that
> *permits* a tightly-coupled system but also lets people build other
> kinds of systems if they wish.

I think we need both tightly and loosely coupled and I think we should 
aim to solution that will make it reasonably possible to do both, ie 
without reimplementing whole thing again to get the other variant. In 
pglogical I was more concerned with loosely-coupled use-cases btw. 
Mainly because I've seen more of them in the wild and also they are more 
interesting for me personally (partially because many of the tightly 
coupled use-cases can be solved by physical replication). I don't think 
there is anything there that would fundamentally prevent us building 
something that can work for both scenarios though.

About doing things with SQL. I think some stuff would be better done 
using SQL personally, like the nodes table, since I dislike the fact 
that it's not shared catalog currently. But many other things I'd prefer 
to manage using functions at least in the first iteration. I think it's 
preferable to have simple function interface and make things work 
correctly and then maybe try to replace it with DDL where it makes sense 
because DDL just needs much more boiler plate but otherwise is mostly 
mechanical. One thing that worries me with SQL is that we do have some 
history of bike-shedding any SQL syntax that's not part of standard to 
death halting the actual feature development for prolonged time periods 
as a result.

Finally a side note about sharding - I have strong believe that sharding 
needs to be tightly coupled to be effective and maintainable in production.

--   Petr Jelinek                  http://www.2ndQuadrant.com/  PostgreSQL Development, 24x7 Support, Training &
Services