Re: Horizontal scalability/sharding - Mailing list pgsql-hackers

From Bruce Momjian
Subject Re: Horizontal scalability/sharding
Date
Msg-id 20150901005451.GA24993@momjian.us
Whole thread Raw
In response to Re: Horizontal scalability/sharding  (Josh Berkus <josh@agliodbs.com>)
Responses Re: Horizontal scalability/sharding  (Alvaro Herrera <alvherre@2ndquadrant.com>)
Re: Horizontal scalability/sharding  (Etsuro Fujita <fujita.etsuro@lab.ntt.co.jp>)
Re: Horizontal scalability/sharding  (Andres Freund <andres@anarazel.de>)
List pgsql-hackers
On Mon, Aug 31, 2015 at 05:10:11PM -0700, Josh Berkus wrote:
> > As far as (3) is concerned, why
> > wouldn't we use the foreign data wrapper interface, and specifically
> > postgres_fdw?  That interface was designed for the explicit purpose of
> > allowing access to remote data sources, and a lot of work has been put
> > into it, so it would be highly surprising if we decided to throw that
> > away and develop something completely new from the ground up.
> 
> Well, query hooks are also a capability which we already have, and is
> mature.  Citus has already posted about why they chose to use them instead.
> 
> As long as you recognize that the FDW API (not just the existing fdws)
> will need to expand to make this work, it's a viable path.

Uh, we already have a list of things we need to add to FDWs to make them
work, and Citus Data has provided a document of more things that are
needed, https://goo.gl/vJWF85.  I am not sure how much bigger a red flag
you want to confirm that everyone agrees that major FDW improvements are
a requirement for this.  

My hope is that many FDW improvements will benefit sharding and
non-sharding workloads, but I bet some improvements are going to be
sharding-specific.  I would say we are still in the exploratory stage,
but based on the number of people who care about this feature and want
to be involved, I think we are off to a very good start.  :-)

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + Everyone has their own god. +



pgsql-hackers by date:

Previous
From: Marc Munro
Date:
Subject: Re: Horizontal scalability/sharding
Next
From: Peter Eisentraut
Date:
Subject: Re: exposing pg_controldata and pg_config as functions