Re: I'd like to discuss scaleout at PGCon - Mailing list pgsql-hackers

From MauMau
Subject Re: I'd like to discuss scaleout at PGCon
Date
Msg-id 19921046097C4E049E570BF003B830EC@tunaPC
Whole thread Raw
In response to Re: I'd like to discuss scaleout at PGCon  (Simon Riggs <simon@2ndquadrant.com>)
List pgsql-hackers
From: Simon Riggs
On 1 June 2018 at 16:56, Ashutosh Bapat
<ashutosh.bapat@enterprisedb.com> wrote:
>> I think partitioning + FDW provide basic infrastructure for
>> distributing data, planning queries working with such data. We need
>> more glue to support node management, cluster configuration. So, I
>> agree with your statement. But I think it was clear from the
beginning
>> that we need more than FDW and partitioning.
>
> No, it wasn't clear. But I'm glad to hear it. It might actually work
then.

I found a possibly interesting description in the XL manual.  Although
XL performs various pushdowns like FDW, XL seems to perform some kind
of joins with datanode-to-datanode communication.  Doesn't this prove
that the FDW approach can't handle those joins optimally?  What kind
of joins use the shared queue?



https://www.postgres-xl.org/documentation/pg-xc-specifics.html
--------------------------------------------------
shared_queues (integer)
Datanode Only

For some joins that occur in queries, data from one Datanode may need
to be joined with data from another Datanode. Postgres-XL uses shared
queues for this purpose. During execution each Datanode knows if it
needs to produce or consume tuples, or both.

Note that there may be mulitple shared_queues used even for a single
query. So a value should be set taking into account the number of
connections it can accept and expected number of such joins occurring
simultaneously.
--------------------------------------------------


Regards
MauMau



pgsql-hackers by date:

Previous
From: Andres Freund
Date:
Subject: Re: Spilling hashed SetOps and aggregates to disk
Next
From: Serge Rielau
Date:
Subject: Re: Variable-length FunctionCallInfoData