Thread: RFC: Remove contrib entirely

Hello,

This is a topic that has come up in various ways over the years. After 
the long thread on pg_audit, I thought it might be time to bring it up 
again.

Contrib according to the docs is:

"These include porting tools, analysis utilities, and plug-in features 
that are not part of the core PostgreSQL system, mainly because they 
address a limited audience or are too experimental to be part of the 
main source tree. This does not preclude their usefulness."

It has also been mentioned many times over the years that contrib is a 
holding tank for technology that would hopefully be pushed into core 
someday.

What I am suggesting:

1. Analyze the current contrib modules for inclusion into -core. A few 
of these are pretty obvious:
pg_stat_statementscitextpostgres_fdwhstorepg_crypto[...]

I am sure there will be plenty of fun to be had with what should or 
shouldn't be merged into core. I think if we argue about the guidelines 
of how to analyze what should be in core versus the merits of any 
particular module, life will be easier. Here are some for a start:
A. Must have been in contrib for at least two releasesB. Must have visible community (and thus use case)

2. Push the rest out into a .Org project called contrib. Let those who 
are interested in the technology work on them or use them. This project 
since it is outside of core proper can work just like other extension 
projects. Alternately, allow the maintainers push them wherever they 
like (Landscape, Github, Savannah, git.postgresql.org ...).

Why I am suggesting this:

1. Less code to maintain in core
2. Eliminates the mysticism of contrib
3. Removal of experimental code from core
4. Most of the distributions package contrib separately anyway
5. Some of core is extremely small use case (sepgsql, tsearch2, lo ...)
6. Finding utilities for PostgreSQL used to be harder. It is rather dumb 
simple teenage snapchat user easy now.
8. Isn't this what pgxs is for?
9. Everybody hates cleaning the closet until the end result.
10. Several of these modules would make PostgreSQL look good anyway 
(default case insensitive index searching with citext? It is a gimme)
11. Contrib has been getting smaller and smaller. Let's cut the cord.
12. Isn't this the whole point of extensions?

Sincerely,

jD

-- 
Command Prompt, Inc. - http://www.commandprompt.com/  503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.

JD,

* Joshua D. Drake (jd@commandprompt.com) wrote:
> Contrib according to the docs is:
>
> "These include porting tools, analysis utilities, and plug-in
> features that are not part of the core PostgreSQL system, mainly
> because they address a limited audience or are too experimental to
> be part of the main source tree. This does not preclude their
> usefulness."

We certainly seem to have moved on from that definition.  If nothing
else, it'd be valuable to clarify what contrib is and then rearrange
things accordingly.

> It has also been mentioned many times over the years that contrib is
> a holding tank for technology that would hopefully be pushed into
> core someday.

I continue to feel this is a good use-case for contrib.

> What I am suggesting:
>
> 1. Analyze the current contrib modules for inclusion into -core. A
> few of these are pretty obvious:
>
>     pg_stat_statements
>     citext
>     postgres_fdw
>     hstore
>     pg_crypto
>     [...]

We don't really have a place in "-core" for them..  There has been
ongoing discussion about that but nothing has changed in that regard, as
far as I'm aware.  We have moved a few things out of contrib and into
bin, but that doesn't make sense for the above.

What we would need for this is an 'extensions' directory, or similar,
and a clear definition of what the requirements are around getting into
it are.  With that, we could decide for each module currently in contrib
if it should go into the 'extensions' directory.  I'm not sure that we
would necessairly have to remove the contrib module or any modules which
are deemed to not be appropriate for the 'extensions' directory.

> I am sure there will be plenty of fun to be had with what should or
> shouldn't be merged into core. I think if we argue about the
> guidelines of how to analyze what should be in core versus the
> merits of any particular module, life will be easier. Here are some
> for a start:
>
>     A. Must have been in contrib for at least two releases
>     B. Must have visible community (and thus use case)

I don't particularly like having a time-based requirement drive what's
in which area, especially one that's double the length of our major
release cycle.

> 2. Push the rest out into a .Org project called contrib. Let those
> who are interested in the technology work on them or use them. This
> project since it is outside of core proper can work just like other
> extension projects. Alternately, allow the maintainers push them
> wherever they like (Landscape, Github, Savannah, git.postgresql.org
> ...).

That's an interesting idea, but it's unclear how this would be any
different from PGXN..?
Thanks!
    Stephen

On 05/28/2015 12:35 PM, Stephen Frost wrote:
> JD,

> What we would need for this is an 'extensions' directory, or similar,
> and a clear definition of what the requirements are around getting into
> it are.  With that, we could decide for each module currently in contrib
> if it should go into the 'extensions' directory.  I'm not sure that we
> would necessairly have to remove the contrib module or any modules which
> are deemed to not be appropriate for the 'extensions' directory.

I am suggesting that things like citext be made a type proper. No 
install required. For things like pg_stat_statements of course there 
could be configuration required (need to add it to the preload) but it 
is automatically installed with the packages.

>
>> I am sure there will be plenty of fun to be had with what should or
>> shouldn't be merged into core. I think if we argue about the
>> guidelines of how to analyze what should be in core versus the
>> merits of any particular module, life will be easier. Here are some
>> for a start:
>>
>>     A. Must have been in contrib for at least two releases
>>     B. Must have visible community (and thus use case)
>
> I don't particularly like having a time-based requirement drive what's
> in which area, especially one that's double the length of our major
> release cycle.

I think there is only one or two contrib modules that don't actually fit 
requirement A.

>
>> 2. Push the rest out into a .Org project called contrib. Let those
>> who are interested in the technology work on them or use them. This
>> project since it is outside of core proper can work just like other
>> extension projects. Alternately, allow the maintainers push them
>> wherever they like (Landscape, Github, Savannah, git.postgresql.org
>> ...).
>
> That's an interesting idea, but it's unclear how this would be any
> different from PGXN..?

PGXN is CPAN not Perl or DBD::Pg.

It is actually a compliment to PGXN because it is still needed and 
needed more because that is where you are going to get the "latest and 
greatest" of the modules.

Sincerely,

JD

-- 
Command Prompt, Inc. - http://www.commandprompt.com/  503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.

On 05/28/2015 04:05 PM, Joshua D. Drake wrote:
>
> On 05/28/2015 12:35 PM, Stephen Frost wrote:
>> JD,
>
>> What we would need for this is an 'extensions' directory, or similar,
>> and a clear definition of what the requirements are around getting into
>> it are.  With that, we could decide for each module currently in contrib
>> if it should go into the 'extensions' directory.  I'm not sure that we
>> would necessairly have to remove the contrib module or any modules which
>> are deemed to not be appropriate for the 'extensions' directory.
>
> I am suggesting that things like citext be made a type proper. No 
> install required. 


This seems to come up regularly. Maybe we should put it in an FAQ 
somewhere. The barriers to making non-core types into core types are 
very very high, possibly insurmountable. This is pretty much not an option.

cheers

andrew

On 05/28/2015 01:11 PM, Andrew Dunstan wrote:

>
> This seems to come up regularly. Maybe we should put it in an FAQ
> somewhere. The barriers to making non-core types into core types are
> very very high, possibly insurmountable. This is pretty much not an option.

O.k., then either:
 * We install it by default and document that it is available (and how 
to enable it)
 * Push it out of core and let it be happy wherever Theory wants it to be

JD



-- 
Command Prompt, Inc. - http://www.commandprompt.com/  503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.

On 05/28/2015 04:22 PM, Joshua D. Drake wrote:
>
> On 05/28/2015 01:11 PM, Andrew Dunstan wrote:
>
>>
>> This seems to come up regularly. Maybe we should put it in an FAQ
>> somewhere. The barriers to making non-core types into core types are
>> very very high, possibly insurmountable. This is pretty much not an 
>> option.
>
> O.k., then either:
>
>  * We install it by default and document that it is available (and how 
> to enable it)
>
>  * Push it out of core and let it be happy wherever Theory wants it to be
>


I'd be ok with installing it by default.

But the case that's a lot harder to require to be always installed is 
pgcrypto, as has often been discussed in the past.

In any case, we will always want to have some loadable modules, not 
least because it's a part of eating our own dog food.


cheers

andrew

On 5/28/15 3:35 PM, Stephen Frost wrote:
> What we would need for this is an 'extensions' directory, or similar,
> and a clear definition of what the requirements are around getting into
> it are.  With that, we could decide for each module currently in contrib
> if it should go into the 'extensions' directory.  I'm not sure that we
> would necessairly have to remove the contrib module or any modules which
> are deemed to not be appropriate for the 'extensions' directory.

This seems reasonable to me.  It's in line with the recent move from
contrib to bin.  It'll just be quite a bit bigger of an undertaking.
(50 threads to discuss the merits of each module separately?)  Maybe
start by picking the top 5 and sort those out.

On 05/28/2015 06:25 PM, Andrew Dunstan wrote:

>
> I'd be ok with installing it by default.
>
> But the case that's a lot harder to require to be always installed is
> pgcrypto, as has often been discussed in the past.
>

It used to be but IIRC we don't have those restrictions anymore. If so, 
then we need to pull it out and just call it the "Encryption Extension" 
or whatever....

JD


-- 
Command Prompt, Inc. - http://www.commandprompt.com/  503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.

On 05/28/2015 06:50 PM, Peter Eisentraut wrote:
>
> On 5/28/15 3:35 PM, Stephen Frost wrote:
>> What we would need for this is an 'extensions' directory, or similar,
>> and a clear definition of what the requirements are around getting into
>> it are.  With that, we could decide for each module currently in contrib
>> if it should go into the 'extensions' directory.  I'm not sure that we
>> would necessairly have to remove the contrib module or any modules which
>> are deemed to not be appropriate for the 'extensions' directory.
>
> This seems reasonable to me.  It's in line with the recent move from
> contrib to bin.  It'll just be quite a bit bigger of an undertaking.
> (50 threads to discuss the merits of each module separately?)  Maybe
> start by picking the top 5 and sort those out.

The thing is, we don't have that many to argue about now, in fact:

F.1. adminpack
F.2. auth_delay
F.3. auto_explain
F.4. btree_gin
F.5. btree_gist
F.6. chkpass
F.7. citext
F.8. cube
F.9. dblink
F.10. dict_int
F.11. dict_xsyn
F.12. earthdistance
F.13. file_fdw
F.14. fuzzystrmatch
F.15. hstore
F.16. intagg
F.17. intarray
F.18. isn
F.19. lo
F.20. ltree
F.21. pageinspect
F.22. passwordcheck
F.23. pg_buffercache
F.24. pgcrypto
F.25. pg_freespacemap
F.26. pg_prewarm
F.27. pgrowlocks
F.28. pg_stat_statements
F.29. pgstattuple
F.30. pg_trgm
F.31. postgres_fdw
F.32. seg
F.33. sepgsql
F.34. spi
F.35. sslinfo
F.36. tablefunc
F.37. tcn
F.38. test_decoding
F.39. tsearch2
F.40. tsm_system_rows
F.41. tsm_system_time
F.42. unaccent
F.43. uuid-ossp
F.44. xml2

Look at these, a lot of them are obvious... just include for goodness sakes:

pg_trgm has been in contrib for a decade of not more. Either rip it out 
or include it by default.

postgres_fdw (by the time we make the change it will be two releases)

sepgsql has no business being in core, it is:

1. An extension
2. About as linux specific as we can get

Adminpack:

It is for pgAdmin, give it back or push it into core proper

I just don't think this would be that hard if we were willing to put our 
minds to it.

Or.. another way:

Nobody has yet to provide an argument as to why we need contrib when we 
have a fully functioning extension capability.

Ego... is not a good reason.

Of course the other option:

Freeze contrib. What is in there now, is all there will ever be in there 
and the goal is to slowly reduce it to the point that it doesn't matter.


Sincerely,

jD


>
>
>


-- 
Command Prompt, Inc. - http://www.commandprompt.com/  503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.

On 2015-05-29 AM 11:14, Joshua D. Drake wrote:
> 
> pg_trgm has been in contrib for a decade of not more. Either rip it out or
> include it by default.
> 

There are jsonb gin operator class related files in src/backend/utils/adt/.
Perhaps, trgm_gin.c, trgm_gist.c, trgm_op.c could be moved there. Similarly
for other gin/gist operators classes - hstore, intarray, ltree maybe. Or each
can have its own directory (including jsonb).

But the original comment was about having these moved to an 'extensions'
directory if at all, so perhaps this suggestion is moot.

Thanks,
Amit

JD,

* Joshua D. Drake (jd@commandprompt.com) wrote:
> On 05/28/2015 06:50 PM, Peter Eisentraut wrote:
> >On 5/28/15 3:35 PM, Stephen Frost wrote:
> >>What we would need for this is an 'extensions' directory, or similar,
> >>and a clear definition of what the requirements are around getting into
> >>it are.  With that, we could decide for each module currently in contrib
> >>if it should go into the 'extensions' directory.  I'm not sure that we
> >>would necessairly have to remove the contrib module or any modules which
> >>are deemed to not be appropriate for the 'extensions' directory.
> >
> >This seems reasonable to me.  It's in line with the recent move from
> >contrib to bin.  It'll just be quite a bit bigger of an undertaking.
> >(50 threads to discuss the merits of each module separately?)  Maybe
> >start by picking the top 5 and sort those out.
>
> The thing is, we don't have that many to argue about now, in fact:

Alright, I'll bite. :)

> F.1. adminpack

Need it- pgAdmin can't senibly install it or even include it in some
way, and it'd be *very* painful to not have it for a lot of users.

> F.2. auth_delay

Should be a core feature.  Having this in a contrib module is silly.

> F.3. auto_explain

Move to extension directory in the repo.

> F.4. btree_gin
> F.5. btree_gist

Both of these should simply be in core.

> F.6. chkpass
> F.7. citext
> F.8. cube

Push out and/or keep it in contrib in repo.

> F.9. dblink

Move to extension directory in the repo.

> F.10. dict_int
> F.11. dict_xsyn

Looks like these are just examples?  Maybe move to an 'examples'
directory, or into src/test/modules, or keep in contrib.

> F.12. earthdistance

Depends on cube, so, same as whatever happens there.  I don't think
extensions-in-repo should depend on contrib modules, as a rule.

> F.13. file_fdw
> F.14. fuzzystrmatch
> F.15. hstore

Move to extension directory in the repo.

> F.16. intagg

Obsolute, per the docs.  Push out and deal with the break, or keep it in
contrib in repo.

> F.17. intarray

Move to extension directory in the repo.

> F.18. isn
> F.19. lo
> F.20. ltree
> F.21. pageinspect

Move to extension directory in the repo.

> F.22. passwordcheck

Should be an in-core capability and not shoved off into an extension.

> F.23. pg_buffercache

Pull it into core.

> F.24. pgcrypto

Move to extension directory in the repo.

> F.25. pg_freespacemap

Should be in core.

> F.26. pg_prewarm
> F.27. pgrowlocks

Should be in core.

> F.28. pg_stat_statements

I'd actually prefer that this be in core, but I'd be alright with it
being in extension directory in the repo.

> F.29. pgstattuple
> F.30. pg_trgm

Should be in core.

> F.31. postgres_fdw

Move to extension directory in the repo.
(actually, I'd be fine with both this and file_fdw being included in
core..  I'm just not 100% sure how that'd look)

> F.32. seg
> F.33. sepgsql

Move to extension directory in the repo.

> F.34. spi

Maybe pull some into core..  or maybe all, or move to an extension.

> F.35. sslinfo

Should be in core.

> F.36. tablefunc

My gut reaction is that it should be in core for crosstab(), but David's
talking about implementing PIVOT, so..

> F.37. tcn

Should be in core, imv, but not a position I hold very strongly.

> F.38. test_decoding

Should be in src/test/modules, or maybe some 'examples' dir.

> F.39. tsearch2

I'm inclined to just drop this..  Or we could keep it in contrib in the
repo.

> F.40. tsm_system_rows
> F.41. tsm_system_time

These should be in core.

> F.42. unaccent

Move to extension directory in the repo.

> F.43. uuid-ossp

This one probably deserves its own thread, heh..

> F.44. xml2

Push it out, or keep it in contrib in repo.

> Look at these, a lot of them are obvious... just include for goodness sakes:
>
> pg_trgm has been in contrib for a decade of not more. Either rip it
> out or include it by default.
>
> postgres_fdw (by the time we make the change it will be two releases)

Agreed.

> sepgsql has no business being in core, it is:
>
> 1. An extension
> 2. About as linux specific as we can get

Not sure that being platform agnostic has to be a requirement of being
in the repo or being an extension in the repo...  It does need some work
though and discussion has recently started about if the sepgsql types
defined in the SELinux reference policy should continue to exist or if
they should be changed.  I'm following that discussion with interest.

> Adminpack:
>
> It is for pgAdmin, give it back or push it into core proper

I'd keep it in the repo as an extension.  Pushing it out would just
cause lots of trouble for little gain.

> I just don't think this would be that hard if we were willing to put
> our minds to it.
>
> Or.. another way:
>
> Nobody has yet to provide an argument as to why we need contrib when
> we have a fully functioning extension capability.

pg_stat_statements is perhaps one of the best reasons.  Not something
that we necessairly want to force on everyone who installs PG
(presumably the additional shared memory and performance impact is an
issue for some people...  though I'm not sure I agree, though it's been
a while since that discussion and perhaps the impact looks like less
from a distance), but a capability that we absolutely want to have.

Not sure if we want the FDWs to be installed by default...  I had been
thinking there would be an issue with that at first, but the more I
consider it, the more I'm agreeing with you that we should just pull in
both postgres_fdw and file_fdw.

Still, there are extensions which we'll want to have because they simply
can't be included in core for one reason or another, they're very useful
bits of code, and we're willing and eager to maintain them moving
forward.

One of the big questions which will arise out of this though is- how do
we describe contrib vs. this set of extensions?  Do we treat security
issues in the 'new' contrib differently?  Do we only leave things which
are examples or are deprecated in the 'new' contrib?  I'm presuming that
the items which end up in 'extensions' are more formally considered part
of the project and therefore held to the overall project standard (as
contrib is now being, even though it clearly wasn't always that way).

> Freeze contrib. What is in there now, is all there will ever be in
> there and the goal is to slowly reduce it to the point that it
> doesn't matter.

I don't particularly like this option.
Thanks!
    Stephen

On 05/28/2015 08:10 PM, Stephen Frost wrote:
> JD,

>>> This seems reasonable to me.  It's in line with the recent move from
>>> contrib to bin.  It'll just be quite a bit bigger of an undertaking.
>>> (50 threads to discuss the merits of each module separately?)  Maybe
>>> start by picking the top 5 and sort those out.
>>
>> The thing is, we don't have that many to argue about now, in fact:
>
> Alright, I'll bite. :)

I knew somebody eventually would ;)

>
>> F.1. adminpack
>
> Need it- pgAdmin can't senibly install it or even include it in some
> way, and it'd be *very* painful to not have it for a lot of users.

Fair enough, although keep in mind we aren't telling people pgAdmin 
isn't useful. We are just pushing it out of core. Who runs from source 
except developers? Distributions would take care of this for us.

>
>> F.2. auth_delay
>
> Should be a core feature.  Having this in a contrib module is silly.
>

+1


>> F.3. auto_explain
>
> Move to extension directory in the repo.

+1

>
>> F.4. btree_gin
>> F.5. btree_gist
>
> Both of these should simply be in core.

+1

>
>> F.6. chkpass
>> F.7. citext
>> F.8. cube
>
>Push out and/or keep it in contrib in repo.
>

Agreed except citext which I think should install by default.


>> F.9. dblink
>
> Move to extension directory in the repo.
>

Agreed.

>> F.10. dict_int
>> F.11. dict_xsyn
>
> Looks like these are just examples?  Maybe move to an 'examples'
> directory, or into src/test/modules, or keep in contrib.
>

Agreed.

>> F.12. earthdistance
>
> Depends on cube, so, same as whatever happens there.  I don't think
> extensions-in-repo should depend on contrib modules, as a rule.
>
>> F.13. file_fdw
>> F.14. fuzzystrmatch
>> F.15. hstore
>
> Move to extension directory in the repo.

Disagree, hstore should be in core. Rest, fine.

>
>> F.16. intagg
>
> Obsolute, per the docs.  Push out and deal with the break, or keep it in
> contrib in repo.
>

Spelling mistake aside ;) agreed

>> F.17. intarray
>
> Move to extension directory in the repo.
>

Agreed

>> F.18. isn
>> F.19. lo
>> F.20. ltree
>> F.21. pageinspect
>
> Move to extension directory in the repo.
>

Except for maybe pageinspect, agreed.

>> F.22. passwordcheck
>
> Should be an in-core capability and not shoved off into an extension.
>

Agreed

>> F.23. pg_buffercache
>
> Pull it into core.
>

Agreed

>> F.24. pgcrypto
>
> Move to extension directory in the repo.
>

Sure.

>> F.25. pg_freespacemap
>
> Should be in core.
>

Agreed.

>> F.26. pg_prewarm
>> F.27. pgrowlocks
>
> Should be in core.
>

With a change to pg_rowlocks, agreed.

>> F.28. pg_stat_statements
>
> I'd actually prefer that this be in core, but I'd be alright with it
> being in extension directory in the repo.
>

Agreed just not enabled by default.

>> F.29. pgstattuple
>> F.30. pg_trgm
>
> Should be in core.

Agreed.

>
>> F.31. postgres_fdw
>
> Move to extension directory in the repo.
> (actually, I'd be fine with both this and file_fdw being included in
> core..  I'm just not 100% sure how that'd look)
>

I think they should be in core, not all FDWs of course but file and 
postgres are kind of obvious to me.

>> F.32. seg
>> F.33. sepgsql
>
> Move to extension directory in the repo.
>

Agreed.

>> F.34. spi
>
> Maybe pull some into core..  or maybe all, or move to an extension.
>

No opinion.

>> F.35. sslinfo
>
> Should be in core.
>

Agreed.

>> F.36. tablefunc
>
> My gut reaction is that it should be in core for crosstab(), but David's
> talking about implementing PIVOT, so..
>

Easy... give it 1 more release. If we get PIVOT, then we don't need it, 
if we don't... all the better for us.

>> F.37. tcn
>
> Should be in core, imv, but not a position I hold very strongly.

no opinion

>
>> F.38. test_decoding
>
> Should be in src/test/modules, or maybe some 'examples' dir.
>

agreed


>> F.39. tsearch2
>
> I'm inclined to just drop this..  Or we could keep it in contrib in the
> repo.

Release a "final release" as a pgxn capable extension and rip it out.

>
>> F.40. tsm_system_rows
>> F.41. tsm_system_time
>
> These should be in core.

Agreed

>
>> F.42. unaccent
>
> Move to extension directory in the repo.
>

Agreed

>> F.43. uuid-ossp
>
> This one probably deserves its own thread, heh..
>
>> F.44. xml2
>
> Push it out, or keep it in contrib in repo.
>
>> Look at these, a lot of them are obvious... just include for goodness sakes:
>>

Agreed.

>> pg_trgm has been in contrib for a decade of not more. Either rip it
>> out or include it by default.
>>
>> postgres_fdw (by the time we make the change it will be two releases)
>
> Agreed.
>
>> sepgsql has no business being in core, it is:
>>
>> 1. An extension
>> 2. About as linux specific as we can get
>
> Not sure that being platform agnostic has to be a requirement of being
> in the repo or being an extension in the repo...  It does need some work
> though and discussion has recently started about if the sepgsql types
> defined in the SELinux reference policy should continue to exist or if
> they should be changed.  I'm following that discussion with interest.
>
>> Adminpack:
>>
>> It is for pgAdmin, give it back or push it into core proper
>
> I'd keep it in the repo as an extension.  Pushing it out would just
> cause lots of trouble for little gain.
>
>> I just don't think this would be that hard if we were willing to put
>> our minds to it.
>>

See... we agree on pretty much all of it in principle.

Seriously guys, this particular argument is an argument of "what?". This 
is writing on the wall. If Frost and I are in this much agreement... :P

Sincerely,

jD


-- 
Command Prompt, Inc. - http://www.commandprompt.com/  503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.

<p dir="ltr">Le 29 mai 2015 5:33 AM, "Joshua D. Drake" <<a
href="mailto:jd@commandprompt.com">jd@commandprompt.com</a>>a écrit :<br /> ><br /> ><br /> > On 05/28/2015
08:10PM, Stephen Frost wrote:<br /> >><br /> >> JD,<br /> ><br /> ><br /> >>>> This seems
reasonableto me.  It's in line with the recent move from<br /> >>>> contrib to bin.  It'll just be quite a
bitbigger of an undertaking.<br /> >>>> (50 threads to discuss the merits of each module separately?) 
Maybe<br/> >>>> start by picking the top 5 and sort those out.<br /> >>><br /> >>><br />
>>>The thing is, we don't have that many to argue about now, in fact:<br /> >><br /> >><br />
>>Alright, I'll bite. :)<br /> ><br /> ><br /> > I knew somebody eventually would ;)<br /> ><br />
><br/> >><br /> >>> F.1. adminpack<br /> >><br /> >><br /> >> Need it- pgAdmin
can'tsenibly install it or even include it in some<br /> >> way, and it'd be *very* painful to not have it for a
lotof users.<br /> ><p dir="ltr">Painful? The adminpack allows pgadmin to change the config files remotely with a UI
thatdoesn't make it easy to say the least. You may well trash your pg_hba.conf file and not be able to connect again
afterreloading. It also allows you to read your log files remotely... if it only contains UTF8 characters (which
doesn'thappen much with my french customers). And loading a 1GB log file is definitely painful.<p dir="ltr">I would be
ofthe idea it doesn't give much (though it doesn't mean I want it to be dropped), but I'm pretty much telling my
customersto drop it whenever I can.<p dir="ltr">> Fair enough, although keep in mind we aren't telling people
pgAdminisn't useful. We are just pushing it out of core. Who runs from source except developers? Distributions would
takecare of this for us.<br /> ><p dir="ltr">Yeah. The way I see this is that distributions would make packages for
eachextension. And I don't see the difference between doing a<p dir="ltr">yum install postgresql94-contrib<p
dir="ltr">Anda<p dir="ltr">yum install postgresql94-adminpack<p dir="ltr">for example, except I would have to run
various"yum install" commands to install every extensions I need, but this is much better for me than bloating my
systemwith extensions I never use (earthdistance comes to mind for example).<p dir="ltr">FWIW, I don't mind which one
weput in core and which one we put out of core. But I like Joshua's idea of getting rid of contribs and pushing them
outas any other extensions.<p dir="ltr">>>> F.2. auth_delay<br /> >><br /> >><br /> >>
Shouldbe a core feature.  Having this in a contrib module is silly.<br /> >><br /> ><br /> > +1<br />
><br/> >>> F.3. auto_explain<br /> >><br /> >><br /> >> Move to extension directory in
therepo.<br /> ><br /> ><br /> > +1<br /> ><br /> ><br /> >><br /> >>> F.4. btree_gin<br
/>>>> F.5. btree_gist<br /> >><br /> >><br /> >> Both of these should simply be in core.<br
/>><br /> ><br /> > +1<br /> ><br /> ><br /> >><br /> >>> F.6. chkpass<br /> >>>
F.7.citext<br /> >>> F.8. cube<br /> >><br /> >><br /> >> Push out and/or keep it in contrib
inrepo.<br /> >><br /> ><br /> > Agreed except citext which I think should install by default.<br />
><br/> ><br /> ><br /> >>> F.9. dblink<br /> >><br /> >><br /> >> Move to extension
directoryin the repo.<br /> >><br /> ><br /> > Agreed.<br /> ><br /> ><br /> >>> F.10.
dict_int<br/> >>> F.11. dict_xsyn<br /> >><br /> >><br /> >> Looks like these are just
examples? Maybe move to an 'examples'<br /> >> directory, or into src/test/modules, or keep in contrib.<br />
>><br/> ><br /> > Agreed.<br /> ><br /> ><br /> >>> F.12. earthdistance<br /> >><br />
>><br/> >> Depends on cube, so, same as whatever happens there.  I don't think<br /> >>
extensions-in-reposhould depend on contrib modules, as a rule.<br /> >><br /> >>> F.13. file_fdw<br />
>>>F.14. fuzzystrmatch<br /> >>> F.15. hstore<br /> >><br /> >><br /> >> Move to
extensiondirectory in the repo.<br /> ><br /> ><br /> > Disagree, hstore should be in core. Rest, fine.<br />
><br/> ><br /> >><br /> >>> F.16. intagg<br /> >><br /> >><br /> >> Obsolute,
perthe docs.  Push out and deal with the break, or keep it in<br /> >> contrib in repo.<br /> >><br />
><br/> > Spelling mistake aside ;) agreed<br /> ><br /> ><br /> >>> F.17. intarray<br />
>><br/> >><br /> >> Move to extension directory in the repo.<br /> >><br /> ><br /> >
Agreed<br/> ><br /> ><br /> >>> F.18. isn<br /> >>> F.19. lo<br /> >>> F.20. ltree<br
/>>>> F.21. pageinspect<br /> >><br /> >><br /> >> Move to extension directory in the
repo.<br/> >><br /> ><br /> > Except for maybe pageinspect, agreed.<br /> ><br /> ><br />
>>>F.22. passwordcheck<br /> >><br /> >><br /> >> Should be an in-core capability and not
shovedoff into an extension.<br /> >><br /> ><br /> > Agreed<br /> ><br /> ><br /> >>> F.23.
pg_buffercache<br/> >><br /> >><br /> >> Pull it into core.<br /> >><br /> ><br /> >
Agreed<br/> ><br /> ><br /> >>> F.24. pgcrypto<br /> >><br /> >><br /> >> Move to
extensiondirectory in the repo.<br /> >><br /> ><br /> > Sure.<br /> ><br /> ><br /> >>>
F.25.pg_freespacemap<br /> >><br /> >><br /> >> Should be in core.<br /> >><br /> ><br />
>Agreed.<br /> ><br /> ><br /> >>> F.26. pg_prewarm<br /> >>> F.27. pgrowlocks<br />
>><br/> >><br /> >> Should be in core.<br /> >><br /> ><br /> > With a change to
pg_rowlocks,agreed.<br /> ><br /> ><br /> >>> F.28. pg_stat_statements<br /> >><br /> >><br
/>>> I'd actually prefer that this be in core, but I'd be alright with it<br /> >> being in extension
directoryin the repo.<br /> >><br /> ><br /> > Agreed just not enabled by default.<br /> ><br /> ><br
/>>>> F.29. pgstattuple<br /> >>> F.30. pg_trgm<br /> >><br /> >><br /> >> Should
bein core.<br /> ><br /> ><br /> > Agreed.<br /> ><br /> ><br /> >><br /> >>> F.31.
postgres_fdw<br/> >><br /> >><br /> >> Move to extension directory in the repo.<br /> >>
(actually,I'd be fine with both this and file_fdw being included in<br /> >> core..  I'm just not 100% sure how
that'dlook)<br /> >><br /> ><br /> > I think they should be in core, not all FDWs of course but file and
postgresare kind of obvious to me.<br /> ><br /> ><br /> >>> F.32. seg<br /> >>> F.33.
sepgsql<br/> >><br /> >><br /> >> Move to extension directory in the repo.<br /> >><br />
><br/> > Agreed.<br /> ><br /> ><br /> >>> F.34. spi<br /> >><br /> >><br /> >>
Maybepull some into core..  or maybe all, or move to an extension.<br /> >><br /> ><br /> > No opinion.<br
/>><br /> ><br /> >>> F.35. sslinfo<br /> >><br /> >><br /> >> Should be in core.<br
/>>><br /> ><br /> > Agreed.<br /> ><br /> ><br /> >>> F.36. tablefunc<br /> >><br />
>><br/> >> My gut reaction is that it should be in core for crosstab(), but David's<br /> >> talking
aboutimplementing PIVOT, so..<br /> >><br /> ><br /> > Easy... give it 1 more release. If we get PIVOT,
thenwe don't need it, if we don't... all the better for us.<br /> ><br /> ><br /> >>> F.37. tcn<br />
>><br/> >><br /> >> Should be in core, imv, but not a position I hold very strongly.<br /> ><br />
><br/> > no opinion<br /> ><br /> ><br /> >><br /> >>> F.38. test_decoding<br /> >><br
/>>><br /> >> Should be in src/test/modules, or maybe some 'examples' dir.<br /> >><br /> ><br />
>agreed<br /> ><br /> ><br /> ><br /> >>> F.39. tsearch2<br /> >><br /> >><br />
>>I'm inclined to just drop this..  Or we could keep it in contrib in the<br /> >> repo.<br /> ><br />
><br/> > Release a "final release" as a pgxn capable extension and rip it out.<br /> ><br /> ><br />
>><br/> >>> F.40. tsm_system_rows<br /> >>> F.41. tsm_system_time<br /> >><br />
>><br/> >> These should be in core.<br /> ><br /> ><br /> > Agreed<br /> ><br /> ><br />
>><br/> >>> F.42. unaccent<br /> >><br /> >><br /> >> Move to extension directory in
therepo.<br /> >><br /> ><br /> > Agreed<br /> ><br /> ><br /> >>> F.43. uuid-ossp<br />
>><br/> >><br /> >> This one probably deserves its own thread, heh..<br /> >><br />
>>>F.44. xml2<br /> >><br /> >><br /> >> Push it out, or keep it in contrib in repo.<br />
>><br/> >>> Look at these, a lot of them are obvious... just include for goodness sakes:<br />
>>><br/> ><br /> > Agreed.<br /> ><br /> ><br /> >>> pg_trgm has been in contrib for a
decadeof not more. Either rip it<br /> >>> out or include it by default.<br /> >>><br /> >>>
postgres_fdw(by the time we make the change it will be two releases)<br /> >><br /> >><br /> >>
Agreed.<br/> >><br /> >>> sepgsql has no business being in core, it is:<br /> >>><br />
>>>1. An extension<br /> >>> 2. About as linux specific as we can get<br /> >><br />
>><br/> >> Not sure that being platform agnostic has to be a requirement of being<br /> >> in the
repoor being an extension in the repo...  It does need some work<br /> >> though and discussion has recently
startedabout if the sepgsql types<br /> >> defined in the SELinux reference policy should continue to exist or
if<br/> >> they should be changed.  I'm following that discussion with interest.<br /> >><br />
>>>Adminpack:<br /> >>><br /> >>> It is for pgAdmin, give it back or push it into core
proper<br/> >><br /> >><br /> >> I'd keep it in the repo as an extension.  Pushing it out would
just<br/> >> cause lots of trouble for little gain.<br /> >><br /> >>> I just don't think this
wouldbe that hard if we were willing to put<br /> >>> our minds to it.<br /> >>><br /> ><br />
>See... we agree on pretty much all of it in principle.<br /> ><br /> > Seriously guys, this particular
argumentis an argument of "what?". This is writing on the wall. If Frost and I are in this much agreement... :P<br />
><br/> ><br /> > Sincerely,<br /> ><br /> > jD<br /> ><br /> ><br /> > -- <br /> > Command
Prompt,Inc. - <a href="http://www.commandprompt.com/">http://www.commandprompt.com/</a>  503-667-4564<br /> >
PostgreSQLCentered full stack support, consulting and development.<br /> > Announcing "I'm offended" is basically
tellingthe world you can't<br /> > control your own emotions, so everyone else should do it for you.<br /> ><br
/>><br /> > -- <br /> > Sent via pgsql-hackers mailing list (<a
href="mailto:pgsql-hackers@postgresql.org">pgsql-hackers@postgresql.org</a>)<br/> > To make changes to your
subscription:<br/> > <a
href="http://www.postgresql.org/mailpref/pgsql-hackers">http://www.postgresql.org/mailpref/pgsql-hackers</a><br/> 

> FWIW, I don't mind which one we put in core and which one we put out of
> core. But I like Joshua's idea of getting rid of contribs and pushing them
> out as any other extensions.

Hmmm.

I like the contrib directory as a living example of "how to do an 
extension" directly available in the source tree. It also allows to test 
in-tree that the extension mechanism works. So I think it should be kept 
at least with a minimum set of dummy examples for this purpose, even if 
all current extensions are moved out.

Also, removing a feature is a regression, and someone is always bound to 
complain... What is the real benefit? ISTM that it is a solution that 
fixes no important problem. Reaching a consensus about what to move here 
or there will consume valuable time that could be spent on more important 
tasks... Is it worth it?

Also moving things into postgresql main sources makes pg heavier for all 
and benefits only to some, so I think that some careful filtering must be 
done bevore moving large contribs there. I guess this is part of the 
argumentation.

-- 
Fabien.

<p dir="ltr">Le 29 mai 2015 8:10 AM, "Pavel Stehule" <<a
href="mailto:pavel.stehule@gmail.com">pavel.stehule@gmail.com</a>>a écrit :<br /> ><br /> > Hi<br /> ><br
/>> I am not sure if PGXN can substitute contrib - mainly due deployment - It doesn't helps with MS Windows.
Installingnecessary software for compilation there is terrible.<br /> ><p dir="ltr">I agree it's hard to compile an
extensionon Windows, but that's already what we have. And I'm sure EDB will put all interesting contrib modules in
theirwindows installer to help users. They already go way further than any Linux packages.<p dir="ltr">> Regards<br
/>><br /> > Pavel<br /> ><br /> > 2015-05-28 18:19 GMT+02:00 Joshua D. Drake <<a
href="mailto:jd@commandprompt.com">jd@commandprompt.com</a>>:<br/> >><br /> >><br /> >> Hello,<br
/>>><br /> >> This is a topic that has come up in various ways over the years. After the long thread on
pg_audit,I thought it might be time to bring it up again.<br /> >><br /> >> Contrib according to the docs
is:<br/> >><br /> >> "These include porting tools, analysis utilities, and plug-in features that are not
partof the core PostgreSQL system, mainly because they address a limited audience or are too experimental to be part of
themain source tree. This does not preclude their usefulness."<br /> >><br /> >> It has also been mentioned
manytimes over the years that contrib is a holding tank for technology that would hopefully be pushed into core
someday.<br/> >><br /> >> What I am suggesting:<br /> >><br /> >> 1. Analyze the current
contribmodules for inclusion into -core. A few of these are pretty obvious:<br /> >><br /> >>        
pg_stat_statements<br/> >>         citext<br /> >>         postgres_fdw<br /> >>         hstore<br />
>>        pg_crypto<br /> >>         [...]<br /> >><br /> >> I am sure there will be plenty of
funto be had with what should or shouldn't be merged into core. I think if we argue about the guidelines of how to
analyzewhat should be in core versus the merits of any particular module, life will be easier. Here are some for a
start:<br/> >><br /> >>         A. Must have been in contrib for at least two releases<br /> >>      
 B. Must have visible community (and thus use case)<br /> >><br /> >> 2. Push the rest out into a .Org
projectcalled contrib. Let those who are interested in the technology work on them or use them. This project since it
isoutside of core proper can work just like other extension projects. Alternately, allow the maintainers push them
whereverthey like (Landscape, Github, Savannah, <a href="http://git.postgresql.org">git.postgresql.org</a> ...).<br />
>><br/> >> Why I am suggesting this:<br /> >><br /> >> 1. Less code to maintain in core<br />
>>2. Eliminates the mysticism of contrib<br /> >> 3. Removal of experimental code from core<br /> >>
4.Most of the distributions package contrib separately anyway<br /> >> 5. Some of core is extremely small use
case(sepgsql, tsearch2, lo ...)<br /> >> 6. Finding utilities for PostgreSQL used to be harder. It is rather dumb
simpleteenage snapchat user easy now.<br /> >> 8. Isn't this what pgxs is for?<br /> >> 9. Everybody hates
cleaningthe closet until the end result.<br /> >> 10. Several of these modules would make PostgreSQL look good
anyway(default case insensitive index searching with citext? It is a gimme)<br /> >> 11. Contrib has been getting
smallerand smaller. Let's cut the cord.<br /> >> 12. Isn't this the whole point of extensions?<br /> >><br
/>>> Sincerely,<br /> >><br /> >> jD<br /> >><br /> >> -- <br /> >> Command Prompt,
Inc.- <a href="http://www.commandprompt.com/">http://www.commandprompt.com/</a>  503-667-4564<br /> >> PostgreSQL
Centeredfull stack support, consulting and development.<br /> >> Announcing "I'm offended" is basically telling
theworld you can't<br /> >> control your own emotions, so everyone else should do it for you.<br /> >><br
/>>><br /> >> -- <br /> >> Sent via pgsql-hackers mailing list (<a
href="mailto:pgsql-hackers@postgresql.org">pgsql-hackers@postgresql.org</a>)<br/> >> To make changes to your
subscription:<br/> >> <a
href="http://www.postgresql.org/mailpref/pgsql-hackers">http://www.postgresql.org/mailpref/pgsql-hackers</a><br/>
><br/> ><br /> 

<p dir="ltr">Le 29 mai 2015 8:01 AM, "Fabien COELHO" <<a
href="mailto:coelho@cri.ensmp.fr">coelho@cri.ensmp.fr</a>>a écrit :<br /> ><br /> ><br /> >> FWIW, I
don'tmind which one we put in core and which one we put out of<br /> >> core. But I like Joshua's idea of getting
ridof contribs and pushing them<br /> >> out as any other extensions.<br /> ><br /> ><br /> > Hmmm.<br
/>><br /> > I like the contrib directory as a living example of "how to do an extension" directly available in
thesource tree. It also allows to test in-tree that the extension mechanism works. So I think it should be kept at
leastwith a minimum set of dummy examples for this purpose, even if all current extensions are moved out.<br /> ><p
dir="ltr">Agreed.<pdir="ltr">> Also, removing a feature is a regression, and someone is always bound to complain...
Whatis the real benefit? ISTM that it is a solution that fixes no important problem. Reaching a consensus about what to
movehere or there will consume valuable time that could be spent on more important tasks... Is it worth it?<br />
><pdir="ltr">It would be less confusing for users. Contrib modules seem to be first class extensions, leaving doubt
onother extensions. But the fact they aren't in core make them not fully trusted by some users. Trying to explain all
thatin a training is a PITA. It would be much less confusing if they were either in core or in their own repository.<p
dir="ltr">>Also moving things into postgresql main sources makes pg heavier for all and benefits only to some, so I
thinkthat some careful filtering must be done bevore moving large contribs there. I guess this is part of the
argumentation.<br/> ><p dir="ltr">Agreed. 

On Fri, May 29, 2015 at 3:20 PM, Guillaume Lelarge
<guillaume@lelarge.info> wrote:
> Le 29 mai 2015 8:10 AM, "Pavel Stehule" <pavel.stehule@gmail.com> a écrit :
>>
>> Hi
>>
>> I am not sure if PGXN can substitute contrib - mainly due deployment - It
>> doesn't helps with MS Windows. Installing necessary software for compilation
>> there is terrible.
>>
>
> I agree it's hard to compile an extension on Windows, but that's already
> what we have. And I'm sure EDB will put all interesting contrib modules in
> their windows installer to help users. They already go way further than any
> Linux packages.

Speaking with my Windows packager hat on, compiling with MinGW would
not be that terrible I think for extensions, as all the existing
Makefile machinery could be used for this purpose. MSVC stuff is more
complicated though with what we have in-core, but still I think that
we could do something with them if we refactor a bit the code and make
easier the declaration of Project objects and have some proper
documentation in the extension chapter, the idea being that users
should not need to build more than a simple build.pl file linking to
some of .pm files of the in-core perl module scripts, basically with a
switch to src/tools/msvc. You would need to have those modules as well
as the compiled deliverables to compile the extensions, but that's the
same deal as any devel-* package on Linux.
--
Michael

On Fri, May 29, 2015 at 4:11 PM, Pavel Stehule wrote:
> 1. VS requires relatively new MS Windows - problem for people with Ms Win 7
> and older

Really, I use Win 2k8 stuff and Win7 quite a lot.

> 2. After installation you have to find and apply some critical fixes - some
> is bad documented.

Example? Perhaps we could improve the docs then.

> I have a few customers on MS Win - nobody was able to compile extension
> there - and I had lot of trables - (but I worked with MSVC ten years ago)

Well, FWIW, I have some extensions compiled on Windows using MSVC
modules with stuff like perl -I src/tools/msvc build.pl that are out
of the vanilla scripts and it works... The script is rather simple
even if a bit grotty.
-- 
Michael

On Fri, May 29, 2015 at 7:20 AM, Guillaume Lelarge
<guillaume@lelarge.info> wrote:
> Le 29 mai 2015 8:10 AM, "Pavel Stehule" <pavel.stehule@gmail.com> a écrit :
>>
>> Hi
>>
>> I am not sure if PGXN can substitute contrib - mainly due deployment - It
>> doesn't helps with MS Windows. Installing necessary software for compilation
>> there is terrible.
>>
>
> I agree it's hard to compile an extension on Windows, but that's already
> what we have. And I'm sure EDB will put all interesting contrib modules in
> their windows installer to help users. They already go way further than any
> Linux packages.

The only extra extension we ship is pl/debugger.


--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Fri, May 29, 2015 at 7:27 AM, Pavel Stehule <pavel.stehule@gmail.com> wrote:
>
>
> 2015-05-29 8:20 GMT+02:00 Guillaume Lelarge <guillaume@lelarge.info>:
>>
>> Le 29 mai 2015 8:10 AM, "Pavel Stehule" <pavel.stehule@gmail.com> a écrit
>> :
>> >
>> > Hi
>> >
>> > I am not sure if PGXN can substitute contrib - mainly due deployment -
>> > It doesn't helps with MS Windows. Installing necessary software for
>> > compilation there is terrible.
>> >
>>
>> I agree it's hard to compile an extension on Windows, but that's already
>> what we have. And I'm sure EDB will put all interesting contrib modules in
>> their windows installer to help users. They already go way further than any
>> Linux packages.
>
> I afraid so dependency on EDB in this case is wrong - I have respect to EDB
> due  work, but installation other extension from EDB stack is difficult,
> unclean, and nothing what I would to use as new base.

The five or six mouse clicks required to install something like Slony
or PostGIS (or both at once) is difficult and unclean?

--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Fri, May 29, 2015 at 8:50 AM, Pavel Stehule <pavel.stehule@gmail.com> wrote:
>
>
> 2015-05-29 9:42 GMT+02:00 Michael Paquier <michael.paquier@gmail.com>:
>>
>> On Fri, May 29, 2015 at 4:11 PM, Pavel Stehule wrote:
>> > 1. VS requires relatively new MS Windows - problem for people with Ms
>> > Win 7
>> > and older
>>
>> Really, I use Win 2k8 stuff and Win7 quite a lot.
>
>
> On Win 7 you have to search and install now unsupported VS EE 2010.

I've been running 2013 on Windows 7 since it came out. Works perfectly
well, and didn't require any unusual installation.

-- 
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Fri, May 29, 2015 at 9:35 AM, Dave Page <dpage@pgadmin.org> wrote:
> On Fri, May 29, 2015 at 7:20 AM, Guillaume Lelarge
> <guillaume@lelarge.info> wrote:
>> Le 29 mai 2015 8:10 AM, "Pavel Stehule" <pavel.stehule@gmail.com> a écrit :
>>>
>>> Hi
>>>
>>> I am not sure if PGXN can substitute contrib - mainly due deployment - It
>>> doesn't helps with MS Windows. Installing necessary software for compilation
>>> there is terrible.
>>>
>>
>> I agree it's hard to compile an extension on Windows, but that's already
>> what we have. And I'm sure EDB will put all interesting contrib modules in
>> their windows installer to help users. They already go way further than any
>> Linux packages.
>
> The only extra extension we ship is pl/debugger.

To clarify - that's the only one we ship in the PostgreSQL installer.
There are many more that we ship (both from EDB and other sources),
but users use StackBuilder to choose, download and install them.

--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Fri, May 29, 2015 at 5:52 PM, Pavel Stehule <pavel.stehule@gmail.com> wrote:
>
>
> 2015-05-29 10:40 GMT+02:00 Dave Page <dpage@pgadmin.org>:
>>
>> On Fri, May 29, 2015 at 8:50 AM, Pavel Stehule <pavel.stehule@gmail.com>
>> wrote:
>> >
>> >
>> > 2015-05-29 9:42 GMT+02:00 Michael Paquier <michael.paquier@gmail.com>:
>> >>
>> >> On Fri, May 29, 2015 at 4:11 PM, Pavel Stehule wrote:
>> >> > 1. VS requires relatively new MS Windows - problem for people with Ms
>> >> > Win 7
>> >> > and older
>> >>
>> >> Really, I use Win 2k8 stuff and Win7 quite a lot.
>> >
>> >
>> > On Win 7 you have to search and install now unsupported VS EE 2010.
>>
>> I've been running 2013 on Windows 7 since it came out. Works perfectly
>> well, and didn't require any unusual installation.
>
>
> I had different experience - When I did it year ago, I had to searching and
> install VS 2010.

And it is not mandatory to use or install visual basic, a simple
Windows SDK with perl installed is enough to build and install PG from
a command prompt.
-- 
Michael

On Fri, May 29, 2015 at 9:55 AM, Pavel Stehule <pavel.stehule@gmail.com> wrote:
>
>
> 2015-05-29 10:37 GMT+02:00 Dave Page <dpage@pgadmin.org>:
>>
>> On Fri, May 29, 2015 at 7:27 AM, Pavel Stehule <pavel.stehule@gmail.com>
>> wrote:
>> >
>> >
>> > 2015-05-29 8:20 GMT+02:00 Guillaume Lelarge <guillaume@lelarge.info>:
>> >>
>> >> Le 29 mai 2015 8:10 AM, "Pavel Stehule" <pavel.stehule@gmail.com> a
>> >> écrit
>> >> :
>> >> >
>> >> > Hi
>> >> >
>> >> > I am not sure if PGXN can substitute contrib - mainly due deployment
>> >> > -
>> >> > It doesn't helps with MS Windows. Installing necessary software for
>> >> > compilation there is terrible.
>> >> >
>> >>
>> >> I agree it's hard to compile an extension on Windows, but that's
>> >> already
>> >> what we have. And I'm sure EDB will put all interesting contrib modules
>> >> in
>> >> their windows installer to help users. They already go way further than
>> >> any
>> >> Linux packages.
>> >
>> > I afraid so dependency on EDB in this case is wrong - I have respect to
>> > EDB
>> > due  work, but installation other extension from EDB stack is difficult,
>> > unclean, and nothing what I would to use as new base.
>>
>> The five or six mouse clicks required to install something like Slony
>> or PostGIS (or both at once) is difficult and unclean?
>
>
> I had a problem with downloading isolated packages with stackbuilder - but
> it was specific, because comp where some packages was installed was not
> access to internet.

Run SB on a machine that is connected and:

- Select "remote server" on the first step. This will disable any
filtering based on database server version, which is normally applied
if you're installing to the local machine (it will only offer packages
compatible with the database servers you have installed). This is not
required if you have the appropriate DB server installed on the
internet connected machine as well.

- Make a note of the Download Directory on step 3

- Check "Skip installation" on step 4.

Copy the downloaded installed from the Download Directory to your
server machine, and run them.

--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On 05/28/2015 11:01 PM, Fabien COELHO wrote:

> Also, removing a feature is a regression, and someone is always bound to
> complain...

We aren't removing any features. These are all items that are NOT 
installed or functional by default.

Sincerely,

JD



-- 
The most kicking donkey PostgreSQL Infrastructure company in existence.
The oldest, the most experienced, the consulting company to the stars.
Command Prompt, Inc. http://www.commandprompt.com/ +1 -503-667-4564 -
24x7 - 365 - Proactive and Managed Professional Services!

On 05/28/2015 11:08 PM, Pavel Stehule wrote:
> Hi
>
> I am not sure if PGXN can substitute contrib - mainly due deployment -
> It doesn't helps with MS Windows. Installing necessary software for
> compilation there is terrible.

Anyone who is building for Windows won't have that problem. They already 
have the environment setup. Windows users for the most part are going to 
download our stackbuilder distro and go from there. They are not going 
to compile modules. What I am suggesting doesn't stop that.

JD

-- 
The most kicking donkey PostgreSQL Infrastructure company in existence.
The oldest, the most experienced, the consulting company to the stars.
Command Prompt, Inc. http://www.commandprompt.com/ +1 -503-667-4564 -
24x7 - 365 - Proactive and Managed Professional Services!

* Joshua D. Drake (jd@commandprompt.com) wrote:
> On 05/28/2015 11:01 PM, Fabien COELHO wrote:
> >Also, removing a feature is a regression, and someone is always bound to
> >complain...
>
> We aren't removing any features. These are all items that are NOT
> installed or functional by default.

Uh, we've already had reports by Debian users about problems upgrading
when they forgot to install the contrib package for the version they're
moving to, so I'm not sure it's quite that simple.  Or, at least, it
won't be for the packagers, who I do believe this would be creating a
fair bit of work for.

That work will be much less if we simply split what's in contrib now
into extension and contrib directories, as it's still all one source
repo to the packagers.  If we punt things out (unless they're being
formally deprecated/removed) to another repo somewhere, then the
packagers are going to have to deal with new source repos and related
complexity.  I'm not saying that's a horrible thing and it might make
sense in some cases, but generally it's a lot easier to go from one
soruce package to a bunch of binary ones than from lots of tiny source
packages to lots of tiny packages.

The versioning aspect of this does come into play though, as having
everything with one relatively slow versioning cycle (on the order of
months) is actually keeping the load on the packagers down.  Lots of
little releases, all at different times, from lots of different source
packages, would increase the workload.

I'm not sure where I ultimately come down on the question about in-repo
vs. out-of-repo.  My gut feeling is that if the community is willing to
continue maintaining contrib modules, then that's ultimately a good
thing and many of them are relatively low-maintenance anyway.  Having a
high barrier to entry for new modules looks a bit odd, given the
definition of contrib, but would be more understandable with a proper
'extensions' directory.  Of course, we'd have to collectivly agree that
we feel comfortable with a lower barrier for contrib that what is being
done now, if the distinction is going to have any meaning.
Thanks!
    Stephen

All,

So there are currently three kinds of things in contrib:

A. Extra commands and tools which aren't considered general enough, or
reliable enough, to be included by default, e.g. pg_standby, pgbench and
vacuumlo.

B. Developer tools, like spi, start-scripts, and oid2name.

C. "Core Extensions", which fall into three further groups:C1: encryption extensions we can't include in core
forlegal reasons (pg_crypto)C2: example extensions which show useful things about           how to build an
extensionC3:Admin extensions which are not core because they carry           risks (e.g. pgstattuple, auto_explain)
 C4: Extensions which are generally useful, used, and           maintained with Postgres (e.g. hstore, citext)
 

So A and B need to stay somewhere in the source distribution.  I would
like to see them go into /admin-tools and /developer-tools directories;
I believe that Greg Smith came up with a patch to do just this at
sometime in the past.  C2 could also go into /developer-tools, and C3
are really just more admin-tools.  C1 would need to stick around as a
special case.

To move C4, you'd have to solve the problem of "how do we turn a former
external extension into a core feature", which nobody yet has solved.

All of this ignores the critical part of this, which is packaging.
Right now packagers ship a "contrib" package which includes everything
in /contrib.  Shifting to having any other arrangement is going to
involve working with them and convincing them that a change to packaging
is worthwhile.  And then getting the news to our users.

Given that, there needs to be significant benefit to our users in the
change.  So, what's the benefit?

-- 
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com

On 05/29/2015 11:02 AM, Jeff Janes wrote:

>
>     Also, removing a feature is a regression, and someone is always
>     bound to complain... What is the real benefit? ISTM that it is a
>     solution that fixes no important problem. Reaching a consensus about
>     what to move here or there will consume valuable time that could be
>     spent on more important tasks... Is it worth it?
>
>
> Yeah, I don't really see the benefit either.  Some could be moved to
> core, and some could just be removed, but many of them it just seems
> like we would end up inventing a new 'contrib' to which is the same as
> the old, but with a different name.

Name one.

Sincerely,

JD
-- 
Command Prompt, Inc. - http://www.commandprompt.com/  503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.

On 05/29/2015 11:27 AM, Jeff Janes wrote:

>     It would be less confusing for users. Contrib modules seem to be
>     first class extensions, leaving doubt on other extensions.
>
>
> Presumably there are still going to be some extensions maintained by
> -hackers, and some not.  I don't think we are going to change that, so
> the difference will still need to be explained, regardless of what words
> are used.  And people *should* have doubts about other extensions.
> Couldn't any talented programmer write an extension which gives them a
> backdoor into the deployer's system, and then upload it to github?

Yes, it is called Open Source development.

>
> I would certainly be cautious about installing any old extension I found
> some some place on the internet.
>
>     But the fact they aren't in core make them not fully trusted by some
>     users.

No. This is completely wrong thinking. If you are compiling this stuff 
from source you are taking that risk on yourself.

Most people are not compiling from source, they are installing from a 
distribution (or downloading a distribution package).

>
>     Trying to explain all that in a training is a PITA. It would be much
>     less confusing if they were either in core or in their own repository.
>
> Several of the contrib modules should be kept in tight sync with src or
> else testing and debugging would be a disaster. Putting them in
> different git repositories wouldn't work well.  Unless those would among
> the ones moved to "core".
>

Note: I actually don't care if the current contrib gets pushed into core 
proper and is default installed.

I care about this idea that contrib exists. It isn't needed and leads to 
a discussion like this one (or the pg_audit), almost every release.

Contrib made sense years ago. It does not any longer. Let's put the old 
horse down and raise a new herd of ponies on a new pasture.

JD

-- 
Command Prompt, Inc. - http://www.commandprompt.com/  503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.

On 05/29/2015 12:30 PM, Pavel Stehule wrote:

>     Contrib made sense years ago. It does not any longer. Let's put the
>     old horse down and raise a new herd of ponies on a new pasture.
>
>
> Still there is strong sense - it is a referential implementation of our
> extension API. We need it to find regressions, changes. I don't believe

No, then we need a proper test suite for the extension API.

> so external extensions can do it. Only PostGIS is massively accepted and
> developed by more than few people. Personally I am thinking so removing
> contrib is not good idea.

Is there an extension/contrib module in the last decade that more than 
once has shown to help us with that?


Sincerely,

JD



-- 
Command Prompt, Inc. - http://www.commandprompt.com/  503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.

On 05/29/2015 02:08 PM, Peter Geoghegan wrote:
> On Fri, May 29, 2015 at 11:47 AM, Josh Berkus <josh@agliodbs.com> wrote:
>> A. Extra commands and tools which aren't considered general enough, or
>> reliable enough, to be included by default, e.g. pg_standby, pgbench and
>> vacuumlo.
>>
>> B. Developer tools, like spi, start-scripts, and oid2name.
>>
>> C. "Core Extensions", which fall into three further groups:
>>         C1: encryption extensions we can't include in core
>>             for legal reasons (pg_crypto)
>>         C2: example extensions which show useful things about
>>             how to build an extension
>>         C3: Admin extensions which are not core because they carry
>>             risks (e.g. pgstattuple, auto_explain)
>>         C4: Extensions which are generally useful, used, and
>>             maintained with Postgres (e.g. hstore, citext)
> 
> I always liked the idea of organizing contrib along these lines.
> 
> I know that I will never be successful in convincing people to remove,
> say, contrib/isn, which is total garbage, but the next best thing is
> to categorize it in a way that sets expectations very low.

Well, contrib/isn is still useful (I use it).  But there's no good
reason it couldn't be on pgxn.


-- 
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com

On Fri, May 29, 2015 at 11:47 AM, Josh Berkus <josh@agliodbs.com> wrote:
> A. Extra commands and tools which aren't considered general enough, or
> reliable enough, to be included by default, e.g. pg_standby, pgbench and
> vacuumlo.
>
> B. Developer tools, like spi, start-scripts, and oid2name.
>
> C. "Core Extensions", which fall into three further groups:
>         C1: encryption extensions we can't include in core
>             for legal reasons (pg_crypto)
>         C2: example extensions which show useful things about
>             how to build an extension
>         C3: Admin extensions which are not core because they carry
>             risks (e.g. pgstattuple, auto_explain)
>         C4: Extensions which are generally useful, used, and
>             maintained with Postgres (e.g. hstore, citext)

I always liked the idea of organizing contrib along these lines.

I know that I will never be successful in convincing people to remove,
say, contrib/isn, which is total garbage, but the next best thing is
to categorize it in a way that sets expectations very low.


-- 
Peter Geoghegan

Josh Berkus <josh@agliodbs.com> writes:
> On 05/29/2015 02:08 PM, Peter Geoghegan wrote:
>> I always liked the idea of organizing contrib along these lines.
>> 
>> I know that I will never be successful in convincing people to remove,
>> say, contrib/isn, which is total garbage, but the next best thing is
>> to categorize it in a way that sets expectations very low.

> Well, contrib/isn is still useful (I use it).  But there's no good
> reason it couldn't be on pgxn.

We already did one round of removal of low-grade contrib items.
Admittedly that was in 2006, and maybe some of the stuff that survived
that cut no longer looks good enough.  But I don't think there's all
that much low-hanging fruit there.

But let's get to the point: the real reason for keeping most of these
contrib modules in the core distribution is that they are essential test
cases for core's extensibility features.  contrib/isn is actually a good
example of that.  It made us realize that extensions that create types
that are physically equivalent to int8 or float8 were broken when we made
those types potentially pass-by-value; we had to add a CREATE TYPE option
to allow that to still work (cf commit 3f936aacc057e4b3).  If contrib/isn
had not been around and been getting built by the buildfarm, we would have
found that out only much later and with much more pain.

You could imagine some other way to address that, like generalizing the
buildfarm so that it can pull in extensions from other source repos for
testing purposes.  But that's going to be a lot of work and I'm not even
real sure we want it --- it'd increase the trust problem for buildfarm
owners quite a bit, for one thing.

I'm not particularly on board with renaming things just to get rid of the
term "contrib".  We have much better things to do with our time.
        regards, tom lane

On Fri, May 29, 2015 at 2:35 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> It made us realize that extensions that create types
> that are physically equivalent to int8 or float8 were broken when we made
> those types potentially pass-by-value; we had to add a CREATE TYPE option
> to allow that to still work (cf commit 3f936aacc057e4b3).  If contrib/isn
> had not been around and been getting built by the buildfarm, we would have
> found that out only much later and with much more pain.

Interesting.

FWIW, my concerns with contrib/isn are limited to the ISBN type and
related types. These types enforce that ISBNs are within certain
ranges known by the module to be valid. The first patch I reviewed for
Postgres back in 2010 extended this range, and I first raised the
issue then -- how many such patches can we expect in the future?

The problem here is that these ranges are controlled by a
decentralized patchwork of national standards bodies, and the ranges
are always subject to revision. I think that it's egregious that
contrib/isn imagines it can track that with a static array.

Since contrib is a place that example code is supposed to live,
perhaps contrib/isn could be held up as an example of what not to
do...
-- 
Peter Geoghegan

Peter Geoghegan <pg@heroku.com> writes:
> The problem here is that these ranges are controlled by a
> decentralized patchwork of national standards bodies, and the ranges
> are always subject to revision. I think that it's egregious that
> contrib/isn imagines it can track that with a static array.

Well, that module has already been rewritten once (which proves that
there's an audience out there for it).  Perhaps somebody will rewrite it
again to support a non-hardwired set of ranges.  Now that we have the
concept of an extension configuration table, that'd be one possible
way to fix it ...
        regards, tom lane

On Fri, May 29, 2015 at 2:54 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Well, that module has already been rewritten once (which proves that
> there's an audience out there for it).  Perhaps somebody will rewrite it
> again to support a non-hardwired set of ranges.  Now that we have the
> concept of an extension configuration table, that'd be one possible
> way to fix it ...

I wouldn't bother.

ISBNs already have a UPC-style weighted sum check digit that will
catch the vast majority of errors, including all transposition errors.
You'd have to try hard to fatfinger an ISBN in a way that produced
something that accidentally had a valid check digit.

contrib/isn suffers from a bad case of protecting against Machiavelli
rather than Murphy. The enforcement isn't just obviously wrong, it's
also ridiculous in principle.

You're right, though -- we have better things to worry about.
-- 
Peter Geoghegan

On 05/29/2015 02:54 PM, Tom Lane wrote:
> Peter Geoghegan <pg@heroku.com> writes:
>> The problem here is that these ranges are controlled by a
>> decentralized patchwork of national standards bodies, and the ranges
>> are always subject to revision. I think that it's egregious that
>> contrib/isn imagines it can track that with a static array.
> 
> Well, that module has already been rewritten once (which proves that
> there's an audience out there for it).  Perhaps somebody will rewrite it
> again to support a non-hardwired set of ranges.  Now that we have the
> concept of an extension configuration table, that'd be one possible
> way to fix it ...

FWIW, neither of the projects I know of which uses ISBN has had any
issues with the range since the 2010 updates.  While the ranges can be
updated in theory, in practice is happens glacially slowly.

-- 
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com

On 5/29/15 5:35 PM, Tom Lane wrote:
> But let's get to the point: the real reason for keeping most of these
> contrib modules in the core distribution is that they are essential test
> cases for core's extensibility features.  contrib/isn is actually a good
> example of that.  It made us realize that extensions that create types
> that are physically equivalent to int8 or float8 were broken when we made
> those types potentially pass-by-value; we had to add a CREATE TYPE option
> to allow that to still work (cf commit 3f936aacc057e4b3).  If contrib/isn
> had not been around and been getting built by the buildfarm, we would have
> found that out only much later and with much more pain.

But the generalization of that argument is that we need to keep around
all possible code forever because it might teach us something in the future.

But it is a valid point that we'd need to build up more extension API
tests before we start cutting back significantly on the
maybe-example-maybe-real extensions that we ship in contrib.  We need to
find a good middle ground.

Peter Eisentraut wrote:

> But it is a valid point that we'd need to build up more extension API
> tests before we start cutting back significantly on the
> maybe-example-maybe-real extensions that we ship in contrib.  We need to
> find a good middle ground.

Nowadays we can test concurrent behavior with isolationtester, and we
can test APIs with src/test/modules.  We can now easily add tests for
lots of stuff that we don't currently test, without having modules that
actually produce for-install programs or libraries.

In any case, +1 on moving useful extensions to src/extensions/.  I just
read somewhere (not this thread?) that Debian for their 9.5 packaging is
going to get rid of the postgresql-contrib package, and instead they are
going to ship all that stuff in the main package.  Seems they were
prompted to do this because of our inaction in this area ...

-- 
Álvaro Herrera                http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On Mon, Jun 01, 2015 at 05:48:16PM -0300, Alvaro Herrera wrote:
> Peter Eisentraut wrote:
> 
> > But it is a valid point that we'd need to build up more extension
> > API tests before we start cutting back significantly on the
> > maybe-example-maybe-real extensions that we ship in contrib.  We
> > need to find a good middle ground.
> 
> Nowadays we can test concurrent behavior with isolationtester, and
> we can test APIs with src/test/modules.  We can now easily add tests
> for lots of stuff that we don't currently test, without having
> modules that actually produce for-install programs or libraries.
> 
> In any case, +1 on moving useful extensions to src/extensions/.  I
> just read somewhere (not this thread?) that Debian for their 9.5
> packaging is going to get rid of the postgresql-contrib package, and
> instead they are going to ship all that stuff in the main package.
> Seems they were prompted to do this because of our inaction in this
> area ...

I suspect they may also have realized that anything not shipped in the
main distribution can cause enough bureaucratic pain, as in "write a
justification for the legal department for why we should install this
software," to cause things simply never to get installed.

Cheers,
David.
-- 
David Fetter <david@fetter.org> http://fetter.org/
Phone: +1 415 235 3778  AIM: dfetter666  Yahoo!: dfetter
Skype: davidfetter      XMPP: david.fetter@gmail.com

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate

On Fri, May 29, 2015 at 10:30 AM, Stephen Frost <sfrost@snowman.net> wrote:
> That work will be much less if we simply split what's in contrib now
> into extension and contrib directories, as it's still all one source
> repo to the packagers.  If we punt things out (unless they're being
> formally deprecated/removed) to another repo somewhere, then the
> packagers are going to have to deal with new source repos and related
> complexity.  I'm not saying that's a horrible thing and it might make
> sense in some cases, but generally it's a lot easier to go from one
> soruce package to a bunch of binary ones than from lots of tiny source
> packages to lots of tiny packages.
>
> The versioning aspect of this does come into play though, as having
> everything with one relatively slow versioning cycle (on the order of
> months) is actually keeping the load on the packagers down.  Lots of
> little releases, all at different times, from lots of different source
> packages, would increase the workload.

These are all good points.

> I'm not sure where I ultimately come down on the question about in-repo
> vs. out-of-repo.  My gut feeling is that if the community is willing to
> continue maintaining contrib modules, then that's ultimately a good
> thing and many of them are relatively low-maintenance anyway.  Having a
> high barrier to entry for new modules looks a bit odd, given the
> definition of contrib, but would be more understandable with a proper
> 'extensions' directory.  Of course, we'd have to collectivly agree that
> we feel comfortable with a lower barrier for contrib that what is being
> done now, if the distinction is going to have any meaning.

I don't agree.  It may make sense to keep some stuff in contrib that
is not great code or not really that useful just for historical
reasons.  But if we accept every extension with the quality of, say,
contrib/isn into the core distribution, or core distribution is going
to be huge and bloated with crap.

I'd like to point out that we've already done some significant cleanup
of contrib - some things got moved to src/test/modules, and other
stuff to src/bin.  So what's in that directory is already a lot more
homogenous than it was a few years ago.  Pretty much everything in
there is a loadable module, and if it does something SQL-visible, it's
packaged as an extension.  The only exceptions are oid2name,
pg_standby, start-scripts, and vacuumlo.  For a directory with >60
things in it, that's pretty good.

I don't think it's very practical to talk about getting rid of contrib
when we reliably add to it in every release:

9.1 added auth_delay, dummy_seclabel, file_fdw, pg_test_fsync, and sepgsql
9.2 added pg_test_timing and tcn
9.3 added pg_xlogdump, postgres_fdw, and worker_spi
9.4 added pg_prewarm, test_decoding, and test_shm_mq
9.5 added hstore_plperl, hstore_plpython, ltree_plpython,
tsm_system_rows, and tsm_system_time

I haven't really gotten familiar with the 9.5 stuff, but most of that
earlier stuff was, uh, pretty good stuff.  We wouldn't have been
better off rejecting it, and we wouldn't have been better off putting
it into the main tree.

Any individual person who looks at contrib will no doubt see a fairly
high percentage of stuff they don't care about.  Given that part of
the charter of contrib is to hold things that we don't want to put in
the core product for one reason or another, that is to be expected.
But every single one of those 18 contrib modules we've added in the
last 5 years was something that someone cared deeply about, many of
them get real use, and we're just sticking our head in the sand if we
think that's not going to keep happening.

For what it's worth, I also don't particularly support renaming
contrib.  I don't really see that it buys us enough to justify the
hassle it will cause.  One thing that may be worth doing yet is
separating the code that is just intended as a POC (like worker_spi,
auth_delay and test_decoding) from the stuff that you are really
intended to run in production (like tcn and hstore).  But that
distinction is fuzzier than you might think, because while auth_delay
was intended as a POC, I've subsequently heard rumors of it being used
in production with satisfactory results.  It's very easy to get the
idea that you know "what PostgreSQL users use" but usually that tends
to mean "what I use" and the community is broad enough that those
things are Not The Same.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On 06/04/2015 10:34 AM, Robert Haas wrote:


>
> For what it's worth, I also don't particularly support renaming
> contrib.  I don't really see that it buys us enough to justify the
> hassle it will cause.  One thing that may be worth doing yet is
> separating the code that is just intended as a POC (like worker_spi,
> auth_delay and test_decoding) from the stuff that you are really
> intended to run in production (like tcn and hstore).  But that
> distinction is fuzzier than you might think, because while auth_delay
> was intended as a POC, I've subsequently heard rumors of it being used
> in production with satisfactory results.  It's very easy to get the
> idea that you know "what PostgreSQL users use" but usually that tends
> to mean "what I use" and the community is broad enough that those
> things are Not The Same.
>
>


The biggest problem is that packagers tend just to bundle contrib 
together in one lump. If we could divide it into two, something like 
"standard modules" and "misc", with the former being included with the 
server package, I think that would be an advance, although packagers 
might reasonably want to treat pgcrypto as a special case.

I'm also not in favor of booting packages wholesale out of core, for all 
the good reasons you and others have advanced.

cheers

andrew

On Thu, Jun 4, 2015 at 11:22 AM, Andrew Dunstan <andrew@dunslane.net> wrote:
> The biggest problem is that packagers tend just to bundle contrib together
> in one lump. If we could divide it into two, something like "standard
> modules" and "misc", with the former being included with the server package,
> I think that would be an advance, although packagers might reasonably want
> to treat pgcrypto as a special case.

The problem is that it's very hard to agree on which stuff ought to be
standard and which stuff ought to be misc.  There's probably some
stuff that almost everyone would agree is pretty useful (like hstore
and postgres_fdw) but figuring out which stuff isn't useful is a lot
harder.  Almost anything you say - that's junk - someone else will say
- no, that thing is great, I use it all the time.  For example, I just
offered contrib/isn as a sort of archetypal example of stuff that's
pretty marginal crap and Josh immediately came back and said, hey, I
use that!  I don't see any principled way of getting past that
difficulty.  Just because a module isn't regularly used by someone who
reads -hackers daily doesn't mean it's not worth keeping.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On 06/04/2015 07:34 AM, Robert Haas wrote:

> I don't think it's very practical to talk about getting rid of contrib
> when we reliably add to it in every release:

Except, that is kind of the point. Why are we adding to it? Contrib 
(AFAICS) is all things that don't need to be in -core. That is the whole 
point of having extensions, is it not?

Sincerely,

JD




-- 
Command Prompt, Inc. - http://www.commandprompt.com/  503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.

On 6/4/15 10:30 AM, Robert Haas wrote:
> On Thu, Jun 4, 2015 at 11:22 AM, Andrew Dunstan<andrew@dunslane.net>  wrote:
>> >The biggest problem is that packagers tend just to bundle contrib together
>> >in one lump. If we could divide it into two, something like "standard
>> >modules" and "misc", with the former being included with the server package,
>> >I think that would be an advance, although packagers might reasonably want
>> >to treat pgcrypto as a special case.
> The problem is that it's very hard to agree on which stuff ought to be
> standard and which stuff ought to be misc.

What I took away upthread was the idea here was to distinguish things 
that were "intended as a POC (like worker_spi,
auth_delay and test_decoding)" from everything else.

I think the real problem here that we're skirting around is this idea of 
'blessed extensions', because that's really the only user benefit 
contrib brings: the idea that this stuff is formally blessed by the 
community. If that's really what we're after then we should just be 
explicit about that. Then we can decide if the best way to approach that 
is keeping it in the main repo (as opposed to say, publishing a list of 
explict PGXN package versions and their checksums).

Personally, I'd rather we publish a list of formally vetted and approved 
versions of PGXN modules. There are many benefits to that, and the 
downside of not having that stuff as part of make check would be 
overcome by the explicit testing we would need to have for approved modules.
-- 
Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX
Data in Trouble? Get it in Treble! http://BlueTreble.com

Robert Haas <robertmhaas@gmail.com> writes:
> On Thu, Jun 4, 2015 at 11:22 AM, Andrew Dunstan <andrew@dunslane.net> wrote:
>> The biggest problem is that packagers tend just to bundle contrib together
>> in one lump. If we could divide it into two, something like "standard
>> modules" and "misc", with the former being included with the server package,
>> I think that would be an advance, although packagers might reasonably want
>> to treat pgcrypto as a special case.

As an ex-packager, I agree that would be a good thing.  In particular
we should try to avoid people packaging stuff that's meant either for
server testing or as sample-source-code-not-useful-in-itself.  We've
made some progress on the former via src/test/modules but I wonder
if we're all the way there; test_decoding surely shouldn't be in
contrib on this measure should it?

BTW, pg_upgrade is also a special case from a packager's viewpoint,
since it needs to be bundled with back-version executables.

> The problem is that it's very hard to agree on which stuff ought to be
> standard and which stuff ought to be misc.  There's probably some
> stuff that almost everyone would agree is pretty useful (like hstore
> and postgres_fdw) but figuring out which stuff isn't useful is a lot
> harder.  Almost anything you say - that's junk - someone else will say
> - no, that thing is great, I use it all the time.  For example, I just
> offered contrib/isn as a sort of archetypal example of stuff that's
> pretty marginal crap and Josh immediately came back and said, hey, I
> use that!  I don't see any principled way of getting past that
> difficulty.  Just because a module isn't regularly used by someone who
> reads -hackers daily doesn't mean it's not worth keeping.

Yeah.  One possible way of measuring this would be to go through the
commit logs and count commits in contrib/ that either added a new
feature or fixed a field-reported bug (ie, did not arise simply from
core-code-driven housekeeping).  Either would be solid evidence that
somebody out there is using it.  Gathering the evidence would be
pretty tedious though :-(
        regards, tom lane

On 06/04/2015 11:49 AM, Joshua D. Drake wrote:
>
> On 06/04/2015 07:34 AM, Robert Haas wrote:
>
>> I don't think it's very practical to talk about getting rid of contrib
>> when we reliably add to it in every release:
>
> Except, that is kind of the point. Why are we adding to it? Contrib 
> (AFAICS) is all things that don't need to be in -core. That is the 
> whole point of having extensions, is it not?
>
>

No. You keep getting this wrong. The fact that we have extensions 
doesn't mean that we want to throw out everything that is an extension. 
It's perfectly reasonable for us to maintain some ourselves, not least 
as part of eating out own dog food.

And I say that as someone who has created and maintains quite a few 
third party extensions.

cheers

andrew

On 06/04/2015 09:00 AM, Andrew Dunstan wrote:

>
> No. You keep getting this wrong. The fact that we have extensions
> doesn't mean that we want to throw out everything that is an extension.
> It's perfectly reasonable for us to maintain some ourselves, not least
> as part of eating out own dog food.

Yes. I agree with that. Which is why I suggested .Org -contrib project.

JD


-- 
Command Prompt, Inc. - http://www.commandprompt.com/  503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.

On 06/04/2015 08:55 AM, Jim Nasby wrote:

> Personally, I'd rather we publish a list of formally vetted and approved
> versions of PGXN modules. There are many benefits to that, and the
> downside of not having that stuff as part of make check would be
> overcome by the explicit testing we would need to have for approved
> modules.

I tried to come up with more words but: Exactly.

The benefit to this idea so far exceeds the benefit to having contrib.

Sincerely,

JD


-- 
The most kicking donkey PostgreSQL Infrastructure company in existence.
The oldest, the most experienced, the consulting company to the stars.
Command Prompt, Inc. http://www.commandprompt.com/ +1 -503-667-4564 -
24x7 - 365 - Proactive and Managed Professional Services!

On Thu, Jun 4, 2015 at 11:49 AM, Joshua D. Drake <jd@commandprompt.com> wrote:
> Except, that is kind of the point. Why are we adding to it?

If you don't know the answer to that question already, then you
probably shouldn't be proposing to get rid of the thing.

I think it's because there are some things we want to include in the
core distribution without baking them irrevocably into the server.

Which, IMV, is 100% reasonable.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On 06/04/2015 10:34 AM, Robert Haas wrote:
>
> On Thu, Jun 4, 2015 at 11:49 AM, Joshua D. Drake <jd@commandprompt.com> wrote:
>> Except, that is kind of the point. Why are we adding to it?
>
> If you don't know the answer to that question already, then you
> probably shouldn't be proposing to get rid of the thing.
>

I know the answer some people are saying. That doesn't mean it is the 
correct answer, that I agree with it or that it is a good answer.

> I think it's because there are some things we want to include in the
> core distribution without baking them irrevocably into the server.
>

I have mentioned before isn't really what this discussion is about. 
Stephen Frost and I even went through the entire list of modules of what 
should and should not be included.

> Which, IMV, is 100% reasonable.
>

Nobody is arguing with that.

Sincerely,

Joshua D. Drake


-- 
Command Prompt, Inc. - http://www.commandprompt.com/  503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.

On Thu, Jun 4, 2015 at 1:57 PM, Joshua D. Drake <jd@commandprompt.com> wrote:
>> I think it's because there are some things we want to include in the
>> core distribution without baking them irrevocably into the server.
>
> I have mentioned before isn't really what this discussion is about. Stephen
> Frost and I even went through the entire list of modules of what should and
> should not be included.
>
>> Which, IMV, is 100% reasonable.
>
> Nobody is arguing with that.

Well then I am confused.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On 06/04/2015 11:01 AM, Robert Haas wrote:
>
> On Thu, Jun 4, 2015 at 1:57 PM, Joshua D. Drake <jd@commandprompt.com> wrote:
>>> I think it's because there are some things we want to include in the
>>> core distribution without baking them irrevocably into the server.
>>
>> I have mentioned before isn't really what this discussion is about. Stephen
>> Frost and I even went through the entire list of modules of what should and
>> should not be included.
>>
>>> Which, IMV, is 100% reasonable.
>>
>> Nobody is arguing with that.
>
> Well then I am confused.

My argument was (after some preliminary discussion):

1. Review contrib
2. All modules that are "core worthy" install by default
3. Push all other contrib out into the wild

Possibly:

1. Have a contrib project that sat outside of core
2. Push all contrib or some contrib to pgxn (or some other place)

Because:

1. Decrease in code maintenance for core
2. Removal of the idea that contrib is a holding queue for not quite up 
to snuff code
3. Most extensions don't need to follow the same development pattern 
that core does
4. Eliminate the EGO of saying "I have a contrib module in core"

Derived from:

1. 15 years of the same argument (current source: pg_audit)
2. Helping reduce overall resource need to manage contrib
3. And now (Nasby's excellent argument)

I am not trying to throw a wrench in things. I am trying to help 
streamline them. A lot of the arguments presented just don't hold water 
(eating your own dogfood) because we aren't stopping anyone from doing 
that testing. Whether or not the source (whatever it may be) is in 
-contrib doesn't prevent that testing.

Sincerely,

Joshua D. Drake






-- 
Command Prompt, Inc. - http://www.commandprompt.com/  503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.

On 2015-06-04 11:33:47 -0700, Joshua D. Drake wrote:
> My argument was (after some preliminary discussion):
> 
> 1. Review contrib
> 2. All modules that are "core worthy" install by default
> 3. Push all other contrib out into the wild
> 
> Possibly:
> 
> 1. Have a contrib project that sat outside of core
> 2. Push all contrib or some contrib to pgxn (or some other place)
> 
> Because:
> 
> 1. Decrease in code maintenance for core

The modules that aren't going to be in core, hardly cost time, do they?

> 2. Removal of the idea that contrib is a holding queue for not quite up to
> snuff code

I don't think that idea exists widely.

> 3. Most extensions don't need to follow the same development pattern that
> core does

The point is that users want them to follow that.

> 4. Eliminate the EGO of saying "I have a contrib module in core"

Seriously?

> 1. 15 years of the same argument (current source: pg_audit)

I don't see getting rid of contrib helping with that. The only change
will then be whether something should be in core.

And there's stuff that's just very hard to develop out of core; but
doesn't necessarily immediately belong into
core. E.g. pg_stat_statements is relatively closely tied to things in
core.

> 2. Helping reduce overall resource need to manage contrib

This discussion a significant share of that.

On 2015-06-04 20:41:46 +0200, Andres Freund wrote:
> > 1. 15 years of the same argument (current source: pg_audit)
> 
> I don't see getting rid of contrib helping with that. The only change
> will then be whether something should be in core.
> 
> And there's stuff that's just very hard to develop out of core; but
> doesn't necessarily immediately belong into
> core. E.g. pg_stat_statements is relatively closely tied to things in
> core.

An even better example is pg_upgrade. It'd not have been possible to
bring it somewhere close to maturity without having it in contrib/. It
requires core cooperation, and we weren't immediately to declare it as
part of core.

> On Jun 4, 2015, at 10:55 AM, Jim Nasby <Jim.Nasby@BlueTreble.com> wrote:
>
> Personally, I'd rather we publish a list of formally vetted and approved versions of PGXN modules. There are many
benefitsto that, and the downside of not having that stuff as part of make check would be overcome by the explicit
testingwe would need to have for approved modules. 

I have looked at PGXN and would never install anything from it.  Why?  Because it is impossible to tell, without inside
knowledgeor a lot of work, what is actively maintained and tested, and what is an abandoned proof-of-concept or idea.
Thereis no indication of what versions of pg any of PGXN modules are tested on, or even if there are tests that can be
runto prove the module works correctly with a particular version of pg.  There are many modules that have not been
updatedfor several years.  What is their status?  If they break is there still someone around to fix them or even cares
aboutthem?  If not, then why waste my time. 

So adding to Jim’s comment above, anything that vets or approves PGXN modules is, in my opinion, essentially required
tomake PGXN useful for anything other than a scratchpad.  A big help would be to pull in the date of the last git
commitin the module overview and ask the authors to edit the readme to add what major version of pg the author last
testedor ran on. 

When I install from contrib, at least I have some minimal assurance that the code is meant to work with the
correspondingversion of pg. 

Neil

On Jun 4, 2015, at 11:53 AM, Neil Tiffin <neilt@neiltiffin.com> wrote:

> I have looked at PGXN and would never install anything from it.  Why?  Because it is impossible to tell, without
insideknowledge or a lot of work, what is actively maintained and tested, and what is an abandoned proof-of-concept or
idea.

Well, you can see the last release dates for a basic idea of that sort of thing. Also the release status (stable,
unstable,testing). 

> There is no indication of what versions of pg any of PGXN modules are tested on, or even if there are tests that can
berun to prove the module works correctly with a particular version of pg. 

Yeah, I’ve been meaning to integrate http://pgxn-tester.org/ results for all modules, which would help with that. In
themeantime you can hit that site itself. Awesome work by Tomas Vondra. 

> There are many modules that have not been updated for several years.  What is their status?  If they break is there
stillsomeone around to fix them or even cares about them?  If not, then why waste my time. 

These are challenges to open-source software in general, and not specific to PGXN.

> So adding to Jim’s comment above, anything that vets or approves PGXN modules is, in my opinion, essentially required
tomake PGXN useful for anything other than a scratchpad. 

Most of the distributions on PGXN feature links to their source code repositories.

> A big help would be to pull in the date of the last git commit in the module overview and ask the authors to edit the
readmeto add what major version of pg the author last tested or ran on. 

That’s difficult to maintain; I used to do it for pgTAP, was too much work. pgxn-tester.org is a much better idea.

Best,

David

On 6/4/15 3:11 PM, David E. Wheeler wrote:
>> There is no indication of what versions of pg any of PGXN modules are tested on, or even if there are tests that can
berun to prove the module works correctly with a particular version of pg.
 
> Yeah, I’ve been meaning to integratehttp://pgxn-tester.org/  results for all modules, which would help with that. In
themeantime you can hit that site itself. Awesome work by Tomas Vondra.
 

A number of modules also run Travis-CI. Might be worth having a way for 
a module to provide it's status .png.
-- 
Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX
Data in Trouble? Get it in Treble! http://BlueTreble.com

On Thu, Jun 4, 2015 at 2:33 PM, Joshua D. Drake <jd@commandprompt.com> wrote:
> My argument was (after some preliminary discussion):
>
> 1. Review contrib
> 2. All modules that are "core worthy" install by default
> 3. Push all other contrib out into the wild

So above, I said that we keep adding to contrib because "there are
some things we want to include in the core distribution without baking
them irrevocably into the server" and you said that you weren't
arguing with that, but here you're saying you don't want any such
things to exist.  That doesn't really make any sense.

postgres_fdw is a good example.  It's "core-worthy" in the sense that
it is useful enough to be installed in core, but not everybody may
want their database server to have the ability to make network
connections at the behest of unprivileged users.  Today, they can
achieve that by not installing the extension if they don't want users
to have access to it.  Putting it into core would require us to come
up with some new way to control whether that functionality is
available or not.  That seems like making a lot of work for ourselves
for no real benefit.

This same argument applies to (at least) dblink and adminpack.

> 1. Decrease in code maintenance for core

contrib requires very little maintenance, and is often very helpful
for judging whether other core changes - e.g. changes to hooks - are
working properly.  I see no maintenance benefit to removing it; it
would probably just make it harder to figure out whether other stuff
is broken.  And the removal itself would be a ton of work.

> 2. Removal of the idea that contrib is a holding queue for not quite up to
> snuff code

I don't think it's really being used that away any more.

> 3. Most extensions don't need to follow the same development pattern that
> core does

That's not a reason to move things that are already in contrib to
somewhere else.  At least not that I can see.

> 4. Eliminate the EGO of saying "I have a contrib module in core"

I've got multiple major features in core.  Any ego I may have about my
PostgreSQL contributions is not based on pg_prewarm.

> 1. 15 years of the same argument (current source: pg_audit)

The argument about pg_audit has little to do with contrib.  It is
primarily about code quality, and secondarily about whether one
committer can go do something unliterally when a long list of other
committers and contributors have expressed doubts about it.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Robert,

* Robert Haas (robertmhaas@gmail.com) wrote:
> On Thu, Jun 4, 2015 at 2:33 PM, Joshua D. Drake <jd@commandprompt.com> wrote:
> > 1. 15 years of the same argument (current source: pg_audit)
>
> The argument about pg_audit has little to do with contrib.  It is
> primarily about code quality, and secondarily about whether one
> committer can go do something unliterally when a long list of other
> committers and contributors have expressed doubts about it.

I would certainly welcome any further review or comments regarding the
code quality of pg_audit from anyone interested in the capability.  I do
not agree that the code quality is significantly below that of other
modules or core.  There were design restrictions due to it being an
extension which quite a few people had questions and concerns about,
which I addressed through the discussions on the list.

Further, pg_audit was originally presented by 2ndQ, worked on by
multiple major contributors and committers, and had multiple committers
expressing interest in committing it during the push to close out the
final CF for 9.5.  I understand that I've been the one who has primairly
been spending time discussing it on the lists, but I was also one of the
biggest nay-sayers of it over the summer last year.  What changed my
opinion of it?  The point made by other committers that the upgrade
concerns could be addressed and the strong interest from users in the
capability.

Painting it as the unilateral actions of one committer is uncharitable,
at best.
Thanks!
    Stephen

On Fri, Jun 5, 2015 at 8:32 AM, Stephen Frost <sfrost@snowman.net> wrote:
> Painting it as the unilateral actions of one committer is uncharitable,
> at best.

As I see it, it is just stating the facts.  There were several emails
from other committers on the pg_audit thread not that long before it
was committed, and zero of them were supportive.  None of them were
dead-set against it, but it is not our typical way of doing business
to press forward with committing something when that many doubts have
been expressed.  I believe you will struggle to find comparable
examples involving other patches.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On 06/05/2015 04:56 AM, Robert Haas wrote:
somewhere else.  At least not that I can see.
>
>> 4. Eliminate the EGO of saying "I have a contrib module in core"
>
> I've got multiple major features in core.  Any ego I may have about my
> PostgreSQL contributions is not based on pg_prewarm.

This was worded badly by me. This isn't about your ego, it is about the 
mysticism surrounding the idea that "they have a feature in core". It is 
really last on the list and not really important to this discussion.

>
>> 1. 15 years of the same argument (current source: pg_audit)
>
> The argument about pg_audit has little to do with contrib.  It is
> primarily about code quality, and secondarily about whether one
> committer can go do something unliterally when a long list of other
> committers and contributors have expressed doubts about it.
>

The argument was about whether it should be in contrib, code quality or 
not. If contrib didn't exist and we accepted that extensions are an 
outside core thing, the argument never would have happened.

JD



-- 
Command Prompt, Inc. - http://www.commandprompt.com/  503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Announcing "I'm offended" is basically telling the world you can't
control your own emotions, so everyone else should do it for you.

On 06/05/2015 03:34 AM, Jim Nasby wrote:
> On 6/4/15 3:11 PM, David E. Wheeler wrote:
>>> There is no indication of what versions of pg any of PGXN modules
>>> are tested on, or even if there are tests that can be run to prove
>>> the module works correctly with a particular version of pg.
>> Yeah, I’ve been meaning to integratehttp://pgxn-tester.org/ results
>> for all modules, which would help with that. In the meantime you can
>> hit that site itself. Awesome work by Tomas Vondra.
>
> A number of modules also run Travis-CI. Might be worth having a way
> for a module to provide it's status .png.

It's also quite possible to test extensions in the buildfarm using an
addon module, which is mostly boilerplate. Of course, that does require
that you're running a buildfarm member. Here's the code for the module
that tests the FileTextArray extension.
<https://github.com/PGBuildFarm/client-code/blob/master/PGBuild/Modules/FileTextArrayFDW.pm>

This and the RedisFDW are tested daily or more often, on machines I control.

cheers

andrew

On Fri, Jun  5, 2015 at 02:42:45PM +0100, Simon Riggs wrote:
> On 29 May 2015 at 02:50, Peter Eisentraut <peter_e@gmx.net> wrote:
> 
>     On 5/28/15 3:35 PM, Stephen Frost wrote:
>     > What we would need for this is an 'extensions' directory, or similar,
>     > and a clear definition of what the requirements are around getting into
>     > it are.  With that, we could decide for each module currently in contrib
>     > if it should go into the 'extensions' directory.  I'm not sure that we
>     > would necessairly have to remove the contrib module or any modules which
>     > are deemed to not be appropriate for the 'extensions' directory.
> 
>     This seems reasonable to me.  It's in line with the recent move from
>     contrib to bin.  It'll just be quite a bit bigger of an undertaking.
>     (50 threads to discuss the merits of each module separately?)  Maybe
>     start by picking the top 5 and sort those out.
> 
> 
> +1 for Extensions directory for 9.6
> 
> This doesn't seem worth delaying the release for.

I didn't think any of this was for 9.5 consideration.

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + Everyone has their own god. +

On Thu, Jun 04, 2015 at 01:11:21PM -0700, David E. Wheeler wrote:
> On Jun 4, 2015, at 11:53 AM, Neil Tiffin <neilt@neiltiffin.com> wrote:
> 
> > I have looked at PGXN and would never install anything from it.
> > Why?  Because it is impossible to tell, without inside knowledge
> > or a lot of work, what is actively maintained and tested, and what
> > is an abandoned proof-of-concept or idea.
> 
> Well, you can see the last release dates for a basic idea of that
> sort of thing. Also the release status (stable, unstable, testing).
> 
> > There is no indication of what versions of pg any of PGXN modules
> > are tested on, or even if there are tests that can be run to prove
> > the module works correctly with a particular version of pg.
> 
> Yeah, I’ve been meaning to integrate http://pgxn-tester.org/ results
> for all modules, which would help with that. In the meantime you can
> hit that site itself. Awesome work by Tomas Vondra.

Giant +1 for this.  I had no idea it existed until this morning.  As a
PGXN contributor, this kind of feedback is invaluable to me.

> > There are many modules that have not been updated for several
> > years.  What is their status?  If they break is there still
> > someone around to fix them or even cares about them?  If not, then
> > why waste my time.
> 
> These are challenges to open-source software in general, and not
> specific to PGXN.

The pgxn-tester system answers more of that question than a lot of
other projects do.  At some point, each organization using software,
free or proprietary, open source or closed, has to do some of their
own vetting.

> > So adding to Jim’s comment above, anything that vets or approves
> > PGXN modules is, in my opinion, essentially required to make PGXN
> > useful for anything other than a scratchpad.
> 
> Most of the distributions on PGXN feature links to their source code
> repositories.

Should this just be made a hard requirement for PGXN?  Lack of a
source code repo is a pretty good sign of abandonment.

Cheers,
David.
-- 
David Fetter <david@fetter.org> http://fetter.org/
Phone: +1 415 235 3778  AIM: dfetter666  Yahoo!: dfetter
Skype: davidfetter      XMPP: david.fetter@gmail.com

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate

> On Jun 4, 2015, at 3:11 PM, David E. Wheeler <david@justatheory.com> wrote:
>
> On Jun 4, 2015, at 11:53 AM, Neil Tiffin <neilt@neiltiffin.com> wrote:
>
>> I have looked at PGXN and would never install anything from it.  Why?  Because it is impossible to tell, without
insideknowledge or a lot of work, what is actively maintained and tested, and what is an abandoned proof-of-concept or
idea.
>
> Well, you can see the last release dates for a basic idea of that sort of thing. Also the release status (stable,
unstable,testing). 
>
>> There is no indication of what versions of pg any of PGXN modules are tested on, or even if there are tests that can
berun to prove the module works correctly with a particular version of pg. 
>
> Yeah, I’ve been meaning to integrate http://pgxn-tester.org/ results for all modules, which would help with that. In
themeantime you can hit that site itself. Awesome work by Tomas Vondra. 
>
>> There are many modules that have not been updated for several years.  What is their status?  If they break is there
stillsomeone around to fix them or even cares about them?  If not, then why waste my time. 
>
> These are challenges to open-source software in general, and not specific to PGXN.

Of course, but the solution is having tools to easily determine the risk.  The fact that the modules pass or fail the
testson pgxn-tester is a significant step.  Knowing how long the module has been failing would be even better. 

>
>> So adding to Jim’s comment above, anything that vets or approves PGXN modules is, in my opinion, essentially
requiredto make PGXN useful for anything other than a scratchpad. 
>
> Most of the distributions on PGXN feature links to their source code repositories.
>
>> A big help would be to pull in the date of the last git commit in the module overview and ask the authors to edit
thereadme to add what major version of pg the author last tested or ran on. 
>
> That’s difficult to maintain; I used to do it for pgTAP, was too much work. pgxn-tester.org is a much better idea.

Yes it is.

Wow, that is awesome work (pgxn-tester.org).  Thanks Tomas Vondra, and David for pointing it out.  This improved my
opinionof PGXN significantly.  It might be helpful to at least put a link on the PGXN home page, beta or not, its
awesomeand even in beta it shows the future direction. 

It would be nice to see the development branch in the tests.  One project I am working on now targets 9.5.

It is important to know how long a stable module has been failing for a specific version of pg.  This is IMO a critical
measureof the level of support a module is receiving.  

Neil

On Jun 5, 2015, at 12:34 AM, Jim Nasby <Jim.Nasby@BlueTreble.com> wrote:

> A number of modules also run Travis-CI. Might be worth having a way for a module to provide it's status .png.

Right. Just stick it in your README.
 http://blog.pgxn.org/post/116087351668/badges

David

Andrew Dunstan wrote:

> It's also quite possible to test extensions in the buildfarm using an addon
> module, which is mostly boilerplate. Of course, that does require that
> you're running a buildfarm member. Here's the code for the module that tests
> the FileTextArray extension.
<https://github.com/PGBuildFarm/client-code/blob/master/PGBuild/Modules/FileTextArrayFDW.pm>

I think important modules should all be tested and reported in the
buildfarm in this way.  For instance, I think PostGIS should report
there as well, so that we know when we break it the moment we do.

> This and the RedisFDW are tested daily or more often, on machines I control.

Yeah, that's pretty neat.

-- 
Álvaro Herrera                http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On 6/5/15 10:49 AM, David E. Wheeler wrote:
> On Jun 5, 2015, at 12:34 AM, Jim Nasby <Jim.Nasby@BlueTreble.com> wrote:
>
>> A number of modules also run Travis-CI. Might be worth having a way for a module to provide it's status .png.
>
> Right. Just stick it in your README.
>
>    http://blog.pgxn.org/post/116087351668/badges

I meant something more integrated with PGXN itself, such as what you're 
proposing for pgxn-tester. That would allow for things like PGXN search 
results to show Travis results.
-- 
Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX
Data in Trouble? Get it in Treble! http://BlueTreble.com

On Jun 5, 2015, at 10:47 AM, Jim Nasby <Jim.Nasby@BlueTreble.com> wrote:

>> Right. Just stick it in your README.
>>
>>   http://blog.pgxn.org/post/116087351668/badges
>
> I meant something more integrated with PGXN itself, such as what you're proposing for pgxn-tester. That would allow
forthings like PGXN search results to show Travis results. 

Well, it’s not really possible to tell if there’s a Travis job for a project. I could add something to the META.json,
though.

David