Thread: Reformatting the FDW wiki page

Reformatting the FDW wiki page

From
damien clochard
Date:
Hey guys,

Last time I checked, there's now more that 40 Foreign Data Wrappers in
the wild and the wiki that tries to keep track of everything is kind of
a mess :

https://wiki.postgresql.org/wiki/Foreign_data_wrappers


* The page is way too long to scroll
* Separating the "native" FDW from the "multicorn" FDW doesn't really
make sense for the end user.
* For some data source, they are several FDW living their own lives
(mongo for instance)
* For each FDW the data provided can easily be standardized ( data
source, url, PGXN link, comment, etc.) so that each project can be
treated in the same way
* We need to find a unified format to update and maintain this data easily

So I'd like to start a complete makeover of this page, but before I go
bonkers on the wiki syntax, here's a preview :

Here's the NoSQL section as it is now :
https://wiki.postgresql.org/wiki/Foreign_data_wrappers#NoSQL_Database_Wrappers

Here's the NoSQL section as I'd like it to be :
https://wiki.postgresql.org/wiki/FDW_v2

Any comments ?


Re: Reformatting the FDW wiki page

From
Dave Page
Date:
On Sun, Feb 15, 2015 at 4:06 PM, damien clochard <damien@dalibo.info> wrote:
> Hey guys,
>
> Last time I checked, there's now more that 40 Foreign Data Wrappers in
> the wild and the wiki that tries to keep track of everything is kind of
> a mess :
>
> https://wiki.postgresql.org/wiki/Foreign_data_wrappers
>
>
> * The page is way too long to scroll
> * Separating the "native" FDW from the "multicorn" FDW doesn't really
> make sense for the end user.
> * For some data source, they are several FDW living their own lives
> (mongo for instance)
> * For each FDW the data provided can easily be standardized ( data
> source, url, PGXN link, comment, etc.) so that each project can be
> treated in the same way
> * We need to find a unified format to update and maintain this data easily
>
> So I'd like to start a complete makeover of this page, but before I go
> bonkers on the wiki syntax, here's a preview :
>
> Here's the NoSQL section as it is now :
> https://wiki.postgresql.org/wiki/Foreign_data_wrappers#NoSQL_Database_Wrappers
>
> Here's the NoSQL section as I'd like it to be :
> https://wiki.postgresql.org/wiki/FDW_v2
>
> Any comments ?

I think that's much better - thanks.

Just to note though - EDB took over maintenance of the Mongo FDW from
Citus Data, with their approval. You should probably only list that
one once.

--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


Re: Reformatting the FDW wiki page

From
damien clochard
Date:

Le 15/02/2015 22:18, Dave Page a écrit :
> On Sun, Feb 15, 2015 at 4:06 PM, damien clochard <damien@dalibo.info> wrote:
>> Hey guys,
>>
>> Last time I checked, there's now more that 40 Foreign Data Wrappers in
>> the wild and the wiki that tries to keep track of everything is kind of
>> a mess :
>>
>> https://wiki.postgresql.org/wiki/Foreign_data_wrappers
>>
>>
>> * The page is way too long to scroll
>> * Separating the "native" FDW from the "multicorn" FDW doesn't really
>> make sense for the end user.
>> * For some data source, they are several FDW living their own lives
>> (mongo for instance)
>> * For each FDW the data provided can easily be standardized ( data
>> source, url, PGXN link, comment, etc.) so that each project can be
>> treated in the same way
>> * We need to find a unified format to update and maintain this data easily
>>
>> So I'd like to start a complete makeover of this page, but before I go
>> bonkers on the wiki syntax, here's a preview :
>>
>> Here's the NoSQL section as it is now :
>> https://wiki.postgresql.org/wiki/Foreign_data_wrappers#NoSQL_Database_Wrappers
>>
>> Here's the NoSQL section as I'd like it to be :
>> https://wiki.postgresql.org/wiki/FDW_v2
>>
>> Any comments ?
>
> I think that's much better - thanks.
>
> Just to note though - EDB took over maintenance of the Mongo FDW from
> Citus Data, with their approval. You should probably only list that
> one once.
>

good to know, I'll update the page.

Does this means that EDB is maintaining the PGXN package as well ?


Re: Reformatting the FDW wiki page

From
Dave Page
Date:
On Sun, Feb 15, 2015 at 4:21 PM, damien clochard <damien@dalibo.info> wrote:
>
>
> Le 15/02/2015 22:18, Dave Page a écrit :
>> On Sun, Feb 15, 2015 at 4:06 PM, damien clochard <damien@dalibo.info> wrote:
>>> Hey guys,
>>>
>>> Last time I checked, there's now more that 40 Foreign Data Wrappers in
>>> the wild and the wiki that tries to keep track of everything is kind of
>>> a mess :
>>>
>>> https://wiki.postgresql.org/wiki/Foreign_data_wrappers
>>>
>>>
>>> * The page is way too long to scroll
>>> * Separating the "native" FDW from the "multicorn" FDW doesn't really
>>> make sense for the end user.
>>> * For some data source, they are several FDW living their own lives
>>> (mongo for instance)
>>> * For each FDW the data provided can easily be standardized ( data
>>> source, url, PGXN link, comment, etc.) so that each project can be
>>> treated in the same way
>>> * We need to find a unified format to update and maintain this data easily
>>>
>>> So I'd like to start a complete makeover of this page, but before I go
>>> bonkers on the wiki syntax, here's a preview :
>>>
>>> Here's the NoSQL section as it is now :
>>> https://wiki.postgresql.org/wiki/Foreign_data_wrappers#NoSQL_Database_Wrappers
>>>
>>> Here's the NoSQL section as I'd like it to be :
>>> https://wiki.postgresql.org/wiki/FDW_v2
>>>
>>> Any comments ?
>>
>> I think that's much better - thanks.
>>
>> Just to note though - EDB took over maintenance of the Mongo FDW from
>> Citus Data, with their approval. You should probably only list that
>> one once.
>>
>
> good to know, I'll update the page.
>
> Does this means that EDB is maintaining the PGXN package as well ?

Yes, I believe that's all been sorted out with David now. I don't know
if a new release has been pushed yet though.

--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


Re: Reformatting the FDW wiki page

From
Devrim Gündüz
Date:
A new version was released last week.

On February 15, 2015 11:23:05 PM EET, Dave Page <dpage@pgadmin.org> wrote:
On Sun, Feb 15, 2015 at 4:21 PM, damien clochard <damien@dalibo.info> wrote:


Le 15/02/2015 22:18, Dave Page a écrit :
On Sun, Feb 15, 2015 at 4:06 PM, damien clochard <damien@dalibo.info> wrote:
Hey guys,

Last time I checked, there's now more that 40 Foreign Data Wrappers in
the wild and the wiki that tries to keep track of everything is kind of
a mess :

https://wiki.postgresql.org/wiki/Foreign_data_wrappers


* The page is way too long to scroll
* Separating the "native" FDW from the "multicorn" FDW doesn't really
make sense for the end user.
* For some data source, they are several FDW living their own lives
(mongo for instance)
* For each FDW the data provided can easily be standardized ( data
source, url, PGXN link, comment, etc.) so that each project can be
treated in the same way
* We need to find a unified format to update and maintain this data easily

So I'd like to start a complete makeover of this page, but before I go
bonkers on the wiki syntax, here's a preview :

Here's the NoSQL section as it is now :
https://wiki.postgresql.org/wiki/Foreign_data_wrappers#NoSQL_Database_Wrappers

Here's the NoSQL section as I'd like it to be :
https://wiki.postgresql.org/wiki/FDW_v2

Any comments ?

I think that's much better - thanks.

Just to note though - EDB took over maintenance of the Mongo FDW from
Citus Data, with their approval. You should probably only list that
one once.


good to know, I'll update the page.

Does this means that EDB is maintaining the PGXN package as well ?

Yes, I believe that's all been sorted out with David now. I don't know
if a new release has been pushed yet though.

--
Sent from my Android device with K-9 Mail. Please excuse my brevity.

Re: Reformatting the FDW wiki page

From
damien clochard
Date:

Done :
https://wiki.postgresql.org/wiki/Foreign%20data%20wrappers?nocache=1


If you're a FDW author, please read take a few seconds to read this page
and modify it if necessary.

If you're not a FDW author, you can also help ! Just go to the "TO DO"
section, pick up a link, check the FDW and put in the right place on the
page:

https://wiki.postgresql.org/wiki/Foreign_data_wrappers?nocache=1#TO_DO

For the record, there's now more than 70 Foreign Data Wrappers listed on
this page. To me, it shows clearly that PostgreSQL has more connectors
to external data than any other storage system. Not sure how we can
communicate about this but it's a clear advantage.

In fact, if you look at it this way : Oracle introduced DBLINK years ago
 and DB2 implemented SQL/MED very early too. In the other hand, FDW
appeared in PostgreSQL in 2011 and we're already way ahead of everyone :)



Re: Reformatting the FDW wiki page

From
Michael Banck
Date:
Hi,

Am Mittwoch, den 18.02.2015, 12:55 +0100 schrieb damien clochard:
> Done :
> https://wiki.postgresql.org/wiki/Foreign%20data%20wrappers?nocache=1

One thing that would be useful to add: a feature matrix for "write
support", "predicate push-down", "join push-down", "schema import" etc.
I think the difference between a basic read-only FDW and one which
implements all of the above (or more) is rather big.

Mentioning the maintainer wouldn't hurt either, but that might make the
table even wider I guess.

As a nitpick, there's some inconsistency in the spelling of github:
"github", "Github" and "GitHub".


Michael

--
Michael Banck
Projektleiter / Berater
Tel.: +49 (2161) 4643-171
Fax:  +49 (2161) 4643-100
Email: michael.banck@credativ.de

credativ GmbH, HRB Mönchengladbach 12080
USt-ID-Nummer: DE204566209
Hohenzollernstr. 133, 41061 Mönchengladbach
Geschäftsführung: Dr. Michael Meskes, Jörg Folz, Sascha Heuer



Re: Reformatting the FDW wiki page

From
damien clochard
Date:

Le 24/08/2015 13:13, Michael Banck a écrit :
> Hi,
>
> Am Mittwoch, den 18.02.2015, 12:55 +0100 schrieb damien clochard:
>> Done :
>> https://wiki.postgresql.org/wiki/Foreign%20data%20wrappers?nocache=1
>
> One thing that would be useful to add: a feature matrix for "write
> support", "predicate push-down", "join push-down", "schema import" etc.
> I think the difference between a basic read-only FDW and one which
> implements all of the above (or more) is rather big.
>

I agree. Something like "production ready" or "stable" won't hurt either
because some FDW in the list are merely proof of concept, while others
can be used without hesitation

> Mentioning the maintainer wouldn't hurt either, but that might make the
> table even wider I guess.
>

yes that's one problem. Feature Matrix are a pain to write with a wiki
syntax (although mediawiki is not the worse). The more columns you add
to the matrix, the more difficult it is to maintain.

Now we could find a smarter way to organize these data. Like storing
them in a robust open source RDBMS for instance... I'm pretty sure we
have people able to do that around here :)

FTR my colleague Ronan Dunklau does something similar but only for the
multicorn-based FDW :

http://multicorn.readthedocs.org/en/latest/third-party-fdw.html

And the PGXN repository also has a list with some information
(maintainer, status, last release) but many FDW are not available on PGXN :

http://pgxn.org/tag/fdw/


Anyway the main problem is : how do you keep the data up-to-date ? I try
to maintain the wiki page once every few weeks but there's probably some
errors in it. The best approach is to let the maintainers update the
wiki page themselves and therefore I want to keep it as simple as possible.


Re: Reformatting the FDW wiki page

From
"Charles Clavadetscher"
Date:
> -----Original Message-----
> From: pgsql-advocacy-owner@postgresql.org [mailto:pgsql-advocacy-
> owner@postgresql.org] On Behalf Of damien clochard
> Sent: Montag, 24. August 2015 14:22
> To: Michael Banck <michael.banck@credativ.de>
> Cc: PostgreSQL Advocacy <pgsql-advocacy@postgresql.org>
> Subject: Re: [pgsql-advocacy] Reformatting the FDW wiki page
>
>
>
> Le 24/08/2015 13:13, Michael Banck a écrit :
> > Hi,
> >
> > Am Mittwoch, den 18.02.2015, 12:55 +0100 schrieb damien clochard:
> >> Done :
> >>
> https://wiki.postgresql.org/wiki/Foreign%20data%20wrappers?nocache=1
> >
> > One thing that would be useful to add: a feature matrix for "write
> > support", "predicate push-down", "join push-down", "schema import" etc.
> > I think the difference between a basic read-only FDW and one which
> > implements all of the above (or more) is rather big.
> >
>
> I agree. Something like "production ready" or "stable" won't hurt either
> because some FDW in the list are merely proof of concept, while others can
> be used without hesitation
>
> > Mentioning the maintainer wouldn't hurt either, but that might make
> > the table even wider I guess.
> >
>
> yes that's one problem. Feature Matrix are a pain to write with a wiki syntax
> (although mediawiki is not the worse). The more columns you add to the
> matrix, the more difficult it is to maintain.
>
> Now we could find a smarter way to organize these data. Like storing them in
> a robust open source RDBMS for instance... I'm pretty sure we have people
> able to do that around here :)

You could use the MediaWiki ExternalData Extension: https://www.mediawiki.org/wiki/Extension:External_Data
That's how we populate dynamically our (unfortunately still very short) members' list on:
http://www.swisspug.org/wiki/index.php/Members(you can look at the source to get an idea). 

Bye
Charles





Re: Reformatting the FDW wiki page

From
Jay Payne
Date:
I've always found the best way to deal with a situation like this is to only have a brief description and link to the maintainer's page for details.    It becomes an editing nightmare to keep information current and relevant.     So I would suggest keeping as little on the main page as possible.  

Just a thought
--J

On Mon, Aug 24, 2015 at 7:50 AM, Charles Clavadetscher <clavadetscher@swisspug.org> wrote:
> -----Original Message-----
> From: pgsql-advocacy-owner@postgresql.org [mailto:pgsql-advocacy-
> owner@postgresql.org] On Behalf Of damien clochard
> Sent: Montag, 24. August 2015 14:22
> To: Michael Banck <michael.banck@credativ.de>
> Cc: PostgreSQL Advocacy <pgsql-advocacy@postgresql.org>
> Subject: Re: [pgsql-advocacy] Reformatting the FDW wiki page
>
>
>
> Le 24/08/2015 13:13, Michael Banck a écrit :
> > Hi,
> >
> > Am Mittwoch, den 18.02.2015, 12:55 +0100 schrieb damien clochard:
> >> Done :
> >>
> https://wiki.postgresql.org/wiki/Foreign%20data%20wrappers?nocache=1
> >
> > One thing that would be useful to add: a feature matrix for "write
> > support", "predicate push-down", "join push-down", "schema import" etc.
> > I think the difference between a basic read-only FDW and one which
> > implements all of the above (or more) is rather big.
> >
>
> I agree. Something like "production ready" or "stable" won't hurt either
> because some FDW in the list are merely proof of concept, while others can
> be used without hesitation
>
> > Mentioning the maintainer wouldn't hurt either, but that might make
> > the table even wider I guess.
> >
>
> yes that's one problem. Feature Matrix are a pain to write with a wiki syntax
> (although mediawiki is not the worse). The more columns you add to the
> matrix, the more difficult it is to maintain.
>
> Now we could find a smarter way to organize these data. Like storing them in
> a robust open source RDBMS for instance... I'm pretty sure we have people
> able to do that around here :)

You could use the MediaWiki ExternalData Extension: https://www.mediawiki.org/wiki/Extension:External_Data
That's how we populate dynamically our (unfortunately still very short) members' list on: http://www.swisspug.org/wiki/index.php/Members (you can look at the source to get an idea).

Bye
Charles





--
Sent via pgsql-advocacy mailing list (pgsql-advocacy@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-advocacy

Re: Reformatting the FDW wiki page

From
David Fetter
Date:
On Mon, Aug 24, 2015 at 08:52:01AM -0500, Jay Payne wrote:
> I've always found the best way to deal with a situation like this is
> to only have a brief description and link to the maintainer's page
> for details.    It becomes an editing nightmare to keep information
> current and relevant.     So I would suggest keeping as little on
> the main page as possible.

My experience runs counter to yours.  When there's only minimal
information, what you get is useless and incoherent.

That reducing entropy locally takes some energy, i.e. time commitment
by actual people, should not surprise anybody who passed high school
physics.

> Just a thought

Please don't top post.

Cheers,
David.
--
David Fetter <david@fetter.org> http://fetter.org/
Phone: +1 415 235 3778  AIM: dfetter666  Yahoo!: dfetter
Skype: davidfetter      XMPP: david.fetter@gmail.com

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate