Re: Horizontal scalability/sharding - Mailing list pgsql-hackers

From Etsuro Fujita
Subject Re: Horizontal scalability/sharding
Date
Msg-id 55E691A2.7000706@lab.ntt.co.jp
Whole thread Raw
In response to Re: Horizontal scalability/sharding  (Amit Langote <Langote_Amit_f8@lab.ntt.co.jp>)
Responses Re: Horizontal scalability/sharding  (Amit Kapila <amit.kapila16@gmail.com>)
List pgsql-hackers
On 2015/09/02 14:28, Amit Langote wrote:
> On 2015-09-02 PM 01:28, Amit Kapila wrote:
>>> On Tue, Sep 1, 2015 at 9:48 PM, Robert Haas <robertmhaas@gmail.com> wrote:
>>>> I'm not averse to making the "connect to the remote nodes" part of
>>>> this solution use something other than the FDW infrastructure at some
>>>> point in time if somebody's prepared to build something better.  On
>>>> the other hand, I think it's extremely clear that the FDW
>>>> infrastructure has a large amount of potential upon which we have
>>>> thoroughly failed to capitalize.  Patches have already been written
>>>> for UPDATE/DELETE pushdown and for join pushdown.

>> Will pushing down writes (Update/Delete) sufficient to maintain sane locking
>> behaviour and deadlock detection that can occur during writes on multiple
>> shards?  For example it could easily be the case where a single Update
>> statement could effect multiple shards and cause deadlock due to waits
>> across the nodes.  Now unless we have some distributed lock manager or
>> some other way to know the information of locks that happens across
>> shards, it could be difficult to detect deadlocks.

> I wonder if Ashutosh's atomic foreign transactions patch would address any
> issues inherent in such cases...

The UPDATE/DELETE pushdown, which I've proposed, would ensure the sane 
behaviour for inherited UPDATEs/DELETEs, as existing non-pushed-down 
UPDATE/DELETE does, because inheritance_planner guarantees that all 
backends lock inheritance children in the same order to avoid needless 
deadlocks.

Best regards,
Etsuro Fujita




pgsql-hackers by date:

Previous
From: Stefan Kaltenbrunner
Date:
Subject: Re: Proposing COPY .. WITH PERMISSIVE
Next
From: Jim Nasby
Date:
Subject: Re: Fwd: Core dump with nested CREATE TEMP TABLE