Thread: PGXN Hosting

PGXN Hosting

From
"David E. Wheeler"
Date:
Howdy PG Infrastructure Folks,

To get PGXN off the ground, I've been running it on a box owned by my consulting company, Kineticode. Command Prompt
hasgraciously given me free hosting for this box for a while, but alas, that's coming to an end and I need to move the
box-- or the services it runs, at least -- elsewhere. As part of that, I'd like to move PGXN to a community server, if
atall possible. 

What I'm *not* asking is for you all to manage it. All I need to keep it going is a vhost. I'd move the three services
themake up PGXN to that box and continue managing it as I have up to now.  Of course, if /me the WWW team *wants* to
takeover the host, that would be welcome. 

Is this do-able?

Thanks,

David




Re: PGXN Hosting

From
Dave Page
Date:
On Wednesday, May 11, 2011, David E. Wheeler <david@kineticode.com> wrote:
> Howdy PG Infrastructure Folks,
>
> To get PGXN off the ground, I've been running it on a box owned by my consulting company, Kineticode. Command Prompt
hasgraciously given me free hosting for this box for a while, but alas, that's coming to an end and I need to move the
box-- or the services it runs, at least -- elsewhere. As part of that, I'd like to move PGXN to a community server, if
atall possible. 
>
> What I'm *not* asking is for you all to manage it. All I need to keep it going is a vhost. I'd move the three
servicesthe make up PGXN to that box and continue managing it as I have up to now.  Of course, if /me the WWW team
*wants*to take over the host, that would be welcome. 
>
> Is this do-able?

Yes. Assuming you can use Debian. You'll get monitoring and updates
for free though, as our hosting platform manages that for us.

What sort of size machine were you looking for?

--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


Re: PGXN Hosting

From
"David E. Wheeler"
Date:
On May 11, 2011, at 10:52 AM, Dave Page wrote:

> Yes. Assuming you can use Debian. You'll get monitoring and updates
> for free though, as our hosting platform manages that for us.

I'm using Ubuntu now, so that's fine, as long as I can compile stuff from source (Perl 5.12 for example).

> What sort of size machine were you looking for?

Right now the usage of PGXN is quite light. And the main site itself relies almost entirely on static JSON files to
generatecontent (only the search engine is dynamic). So its needs are pretty modest. 

There are four parts:

1. PGXN Manager. This is a full PostgreSQL-9.0-backed web app, with authentication and all dynamic requests. This is
theapp one uses to register for the network and to upload distributions. Its usage is therefore pretty light. 

2. The master mirror server. PGXN Manager manages it, and otherwise all it needs are an rsyncd instance for it and a
Webserver to serve its files. I've been using Apache. 

3. PGXN API. This is a glorified mirror. It syncs from the master mirror every hour and processes things, creating HTML
documentationpages and updating the full-text index (which is not tsearch, BTW). 

4. PGXN Site. Reads directly from the PGXN API document root to serve the site.

So there are quite a few things it does, but almost all the Web serving is static or lightly processed files. There's
onPostgreSQL 9.0 database and otherwise a mix of Apache, Plack, and rsync processes and cron jobs. The current box is
real(not virtual), but modest: 2 cores, RAID 0, 2GB RAM. The current database is only 8MB in size, so memory
requirementsare not huge. I'd be surprised if it hit 100MB in the next six months. 

Long term, I think it will make sense to split these tasks up into two boxes, but for now, just one modest box ought to
dothe trick. 

Thanks,

David




Re: PGXN Hosting

From
Magnus Hagander
Date:
On Wed, May 11, 2011 at 20:05, David E. Wheeler <david@kineticode.com> wrote:
> On May 11, 2011, at 10:52 AM, Dave Page wrote:
>
>> Yes. Assuming you can use Debian. You'll get monitoring and updates
>> for free though, as our hosting platform manages that for us.
>
> I'm using Ubuntu now, so that's fine, as long as I can compile stuff from source (Perl 5.12 for example).

Yuck.

That is a big no-no on any reasonable hosting platform. That will kill
all monitoring and management of patches, etc.

It really won't work with a standard perl? How much work to make it so?


>> What sort of size machine were you looking for?
>
> Right now the usage of PGXN is quite light. And the main site itself relies almost entirely on static JSON files to
generatecontent (only the search engine is dynamic). So its needs are pretty modest. 
>
> There are four parts:
>
> 1. PGXN Manager. This is a full PostgreSQL-9.0-backed web app, with authentication and all dynamic requests. This is
theapp one uses to register for the network and to upload distributions. Its usage is therefore pretty light. 

We do have a general policy that all services should use our community
login system. Is there a particular reason this one shouldn't? If not,
I'd much like to see that happen at the time something is moved into
the infrastructure - for consistencys sake.


> 2. The master mirror server. PGXN Manager manages it, and otherwise all it needs are an rsyncd instance for it and a
Webserver to serve its files. I've been using Apache. 
>
> 3. PGXN API. This is a glorified mirror. It syncs from the master mirror every hour and processes things, creating
HTMLdocumentation pages and updating the full-text index (which is not tsearch, BTW). 
>
> 4. PGXN Site. Reads directly from the PGXN API document root to serve the site.
>
> So there are quite a few things it does, but almost all the Web serving is static or lightly processed files. There's
onPostgreSQL 9.0 database and otherwise a mix of Apache, Plack, and rsync processes and cron jobs. The current box is
real(not virtual), but modest: 2 cores, RAID 0, 2GB RAM. The current database is only 8MB in size, so memory
requirementsare not huge. I'd be surprised if it hit 100MB in the next six months. 

Ok, should be easy enough to host on one of the machines we have available.

Is there any of this software that is *not* available as packages in
standard debian? (which I assume would be the same as ubuntu for
these). Any such things makes it a big PITA from the management
systems side, since you'll have to write monitors and stuff
manually...

Other than that; I don't see anything on your requirements that shouldn't work.


> Long term, I think it will make sense to split these tasks up into two boxes, but for now, just one modest box ought
todo the trick. 

Yeah, splitting the tasks clearly apart as you have done is a good way
to be able to do that later - but by what you'r esaying here it
certainly doesn't sound like it needs to be done from the start.

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/


Re: PGXN Hosting

From
"David E. Wheeler"
Date:
On May 11, 2011, at 11:10 AM, Magnus Hagander wrote:

> Yuck.
>
> That is a big no-no on any reasonable hosting platform. That will kill
> all monitoring and management of patches, etc.
>
> It really won't work with a standard perl? How much work to make it so?

If there is a Perl 5.12 package, I'm sure it will work. I will probably need some CPAN modules not yet packaged,
though.

> We do have a general policy that all services should use our community
> login system. Is there a particular reason this one shouldn't? If not,
> I'd much like to see that happen at the time something is moved into
> the infrastructure - for consistencys sake.

I'm happy to integrate with the community login system. It's just a simple matter of programming. :-) Is there now an
APIfor it? 

> Ok, should be easy enough to host on one of the machines we have available.
>
> Is there any of this software that is *not* available as packages in
> standard debian? (which I assume would be the same as ubuntu for
> these). Any such things makes it a big PITA from the management
> systems side, since you'll have to write monitors and stuff
> manually...

I'm sure there are CPAN modules that are not packages. Also at least one PostgreSQL 9.0 extension (semver). As long as
there'sa Perl 5.12 package, I think we'll manage fine. 

> Other than that; I don't see anything on your requirements that shouldn't work.

Great!

>> Long term, I think it will make sense to split these tasks up into two boxes, but for now, just one modest box ought
todo the trick. 
>
> Yeah, splitting the tasks clearly apart as you have done is a good way
> to be able to do that later - but by what you'r esaying here it
> certainly doesn't sound like it needs to be done from the start.

Yep, exactly.

Best,

David




Re: PGXN Hosting

From
Magnus Hagander
Date:
On Wed, May 11, 2011 at 20:15, David E. Wheeler <david@kineticode.com> wrote:
> On May 11, 2011, at 11:10 AM, Magnus Hagander wrote:
>
>> Yuck.
>>
>> That is a big no-no on any reasonable hosting platform. That will kill
>> all monitoring and management of patches, etc.
>>
>> It really won't work with a standard perl? How much work to make it so?
>
> If there is a Perl 5.12 package, I'm sure it will work. I will probably need some CPAN modules not yet packaged,
though.

There isn't. The question wasn't clear enough - how much work would it
take to make it run on the standard version of perl on debian squeeze,
which is 5.10?

And I assume you have a list of the CPAN modules somewhere :-) I'm
sure we can work that out.


>> We do have a general policy that all services should use our community
>> login system. Is there a particular reason this one shouldn't? If not,
>> I'd much like to see that happen at the time something is moved into
>> the infrastructure - for consistencys sake.
>
> I'm happy to integrate with the community login system. It's just a simple matter of programming. :-) Is there now an
APIfor it? 

Yeah, it's trivial - connect to a PostgreSQL server with specified
credentials, and run SELECT community_login().

Since you're a perl guy,
http://git.postgresql.org/gitweb?p=pgcommitfest.git;a=summary is
probably a good reference implementation.


>> Ok, should be easy enough to host on one of the machines we have available.
>>
>> Is there any of this software that is *not* available as packages in
>> standard debian? (which I assume would be the same as ubuntu for
>> these). Any such things makes it a big PITA from the management
>> systems side, since you'll have to write monitors and stuff
>> manually...
>
> I'm sure there are CPAN modules that are not packages. Also at least one PostgreSQL 9.0 extension (semver). As long
asthere's a Perl 5.12 package, I think we'll manage fine. 


--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/


Re: PGXN Hosting

From
Magnus Hagander
Date:
On Wed, May 11, 2011 at 20:30, David E. Wheeler <david@kineticode.com> wrote:
> On May 11, 2011, at 11:22 AM, Magnus Hagander wrote:
>
>> There isn't. The question wasn't clear enough - how much work would it
>> take to make it run on the standard version of perl on debian squeeze,
>> which is 5.10?
>
> Not a lot, but annoying. You're not going to make me use PostgreSQL 8.x too, are you?

Actually, the default packages are 8.4 ;)

But IIRC we have a backport around for 9.0 already.

If it's too annoying, we could go with a backport of perl as well,
assuming one exists.


> Which 5.10?

Version: 5.10.1-17

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/


Re: PGXN Hosting

From
"David E. Wheeler"
Date:
On May 11, 2011, at 11:32 AM, Magnus Hagander wrote:

> Actually, the default packages are 8.4 ;)
>
> But IIRC we have a backport around for 9.0 already.

Great.

> If it's too annoying, we could go with a backport of perl as well,
> assuming one exists.

Looks like 5.12.3 has been built for sid:
 http://packages.debian.org/search?keywords=perl

Is that do-able? Would save me some effort to use that (effort better spent on community auth integration).

Thanks,

David



Re: PGXN Hosting

From
Stefan Kaltenbrunner
Date:
On 05/11/2011 08:39 PM, David E. Wheeler wrote:
> On May 11, 2011, at 11:32 AM, Magnus Hagander wrote:
>
>> Actually, the default packages are 8.4 ;)
>>
>> But IIRC we have a backport around for 9.0 already.
>
> Great.
>
>> If it's too annoying, we could go with a backport of perl as well,
>> assuming one exists.
>
> Looks like 5.12.3 has been built for sid:
>
>    http://packages.debian.org/search?keywords=perl
>
> Is that do-able? Would save me some effort to use that (effort better spent on community auth integration).

I would stringly prefer to stay on 5.10 from squeeze if that is doable, 
manual backporting of such a huge package like perl with its millions of 
forward and reverse dependencies will cause no end of pain :(
Running the (available) postgresql 9.0 backport is a breeze compared to 
that however.



Stefan


Re: PGXN Hosting

From
"David E. Wheeler"
Date:
On May 11, 2011, at 12:18 PM, Stefan Kaltenbrunner wrote:

>> Looks like 5.12.3 has been built for sid:
>>
>>   http://packages.debian.org/search?keywords=perl
>>
>> Is that do-able? Would save me some effort to use that (effort better spent on community auth integration).
>
> I would stringly prefer to stay on 5.10 from squeeze if that is doable, manual backporting of such a huge package
likeperl with its millions of forward and reverse dependencies will cause no end of pain :( 
> Running the (available) postgresql 9.0 backport is a breeze compared to that however.

Well, if I could compile 5.12, I'd install it in /usr/local/. No need to replace the system Perl at all. That's how I
generallywork with this stuff: Leave the system Perl for system tasks; build my own Perl for the apps I build. 

Best,

David



Re: PGXN Hosting

From
Devrim GÜNDÜZ
Date:
On Wed, 2011-05-11 at 12:25 -0700, David E. Wheeler wrote:
>
> > I would stringly prefer to stay on 5.10 from squeeze if that is
> doable, manual backporting of such a huge package like perl with its
> millions of forward and reverse dependencies will cause no end of
> pain :(
> > Running the (available) postgresql 9.0 backport is a breeze compared
> to that however.
>
> Well, if I could compile 5.12, I'd install it in /usr/local/. No need
> to replace the system Perl at all. That's how I generally work with
> this stuff: Leave the system Perl for system tasks; build my own Perl
> for the apps I build.

We immortalks are reading a thread between a developer and sysadmins :)

--
Devrim GÜNDÜZ
Principal Systems Engineer @ EnterpriseDB: http://www.enterprisedb.com
PostgreSQL Danışmanı/Consultant, Red Hat Certified Engineer
Community: devrim~PostgreSQL.org, devrim.gunduz~linux.org.tr
http://www.gunduz.org  Twitter: http://twitter.com/devrimgunduz

Re: PGXN Hosting

From
Stefan Kaltenbrunner
Date:
On 05/11/2011 09:25 PM, David E. Wheeler wrote:
> On May 11, 2011, at 12:18 PM, Stefan Kaltenbrunner wrote:
>
>>> Looks like 5.12.3 has been built for sid:
>>>
>>>    http://packages.debian.org/search?keywords=perl
>>>
>>> Is that do-able? Would save me some effort to use that (effort better spent on community auth integration).
>>
>> I would stringly prefer to stay on 5.10 from squeeze if that is doable, manual backporting of such a huge package
likeperl with its millions of forward and reverse dependencies will cause no end of pain :(
 
>> Running the (available) postgresql 9.0 backport is a breeze compared to that however.
>
> Well, if I could compile 5.12, I'd install it in /usr/local/. No need to replace the system Perl at all. That's how I
generallywork with this stuff: Leave the system Perl for system tasks; build my own Perl for the apps I build.
 

yeah and now the fun starts... "ok we use the packaged postgresql - in 
need plperl in ther to be 5.12", "we cannot use the packaged DBD::Pg 
because that one needs to be compiled agains 5.12 as well", "an oh 
because it is simpled we cannot use any packaged perl lib at all because 
it is so much easier if I just install my own copy that will never be 
security tracked".
We(mostly magnus) basically spent man years to get the new 
infrastructure up and long term maintainable and having to start 
supporting random hand compiled complex packages (again) on them does 
not sound like the right thing to do - it is where we came from and we 
defintely dont want to get back there.
If it really needs perl 5.12 (and maybe other stuff) I would rather 
think it is better to keep pgxn as a seperate entity for the time being.
Infrastructure is all about reliability, sustainability and 
manageability - that often does not mix too well with developer needs 
but that is how it is...


Stefan



Re: PGXN Hosting

From
Stefan Kaltenbrunner
Date:
On 05/11/2011 09:28 PM, Devrim GÜNDÜZ wrote:
> On Wed, 2011-05-11 at 12:25 -0700, David E. Wheeler wrote:
>>
>>> I would stringly prefer to stay on 5.10 from squeeze if that is
>> doable, manual backporting of such a huge package like perl with its
>> millions of forward and reverse dependencies will cause no end of
>> pain :(
>>> Running the (available) postgresql 9.0 backport is a breeze compared
>> to that however.
>>
>> Well, if I could compile 5.12, I'd install it in /usr/local/. No need
>> to replace the system Perl at all. That's how I generally work with
>> this stuff: Leave the system Perl for system tasks; build my own Perl
>> for the apps I build.
>
> We immortalks are reading a thread between a developer and sysadmins :)

spot on :P - but the fact is that the postgresql infrastructure is 
maintained by the postgresql.org SYSADMIN team and NOT the 
postgresql.org developers (as in -hackers) :)


Stefan


Re: PGXN Hosting

From
Dave Page
Date:
On Wed, May 11, 2011 at 8:45 PM, Stefan Kaltenbrunner
<stefan@kaltenbrunner.cc> wrote:
> On 05/11/2011 09:25 PM, David E. Wheeler wrote:
>>
>> On May 11, 2011, at 12:18 PM, Stefan Kaltenbrunner wrote:
>>
>>>> Looks like 5.12.3 has been built for sid:
>>>>
>>>>   http://packages.debian.org/search?keywords=perl
>>>>
>>>> Is that do-able? Would save me some effort to use that (effort better
>>>> spent on community auth integration).
>>>
>>> I would stringly prefer to stay on 5.10 from squeeze if that is doable,
>>> manual backporting of such a huge package like perl with its millions of
>>> forward and reverse dependencies will cause no end of pain :(
>>> Running the (available) postgresql 9.0 backport is a breeze compared to
>>> that however.
>>
>> Well, if I could compile 5.12, I'd install it in /usr/local/. No need to
>> replace the system Perl at all. That's how I generally work with this stuff:
>> Leave the system Perl for system tasks; build my own Perl for the apps I
>> build.
>
> yeah and now the fun starts... "ok we use the packaged postgresql - in need
> plperl in ther to be 5.12", "we cannot use the packaged DBD::Pg because that
> one needs to be compiled agains 5.12 as well", "an oh because it is simpled
> we cannot use any packaged perl lib at all because it is so much easier if I
> just install my own copy that will never be security tracked".
> We(mostly magnus) basically spent man years to get the new infrastructure up
> and long term maintainable and having to start supporting random hand
> compiled complex packages (again) on them does not sound like the right
> thing to do - it is where we came from and we defintely dont want to get
> back there.
> If it really needs perl 5.12 (and maybe other stuff) I would rather think it
> is better to keep pgxn as a seperate entity for the time being.
> Infrastructure is all about reliability, sustainability and manageability -
> that often does not mix too well with developer needs but that is how it
> is...

Well said. We can't go back to the dark times - that way madness lies.


--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


Re: PGXN Hosting

From
Cédric Villemain
Date:
2011/5/11 Stefan Kaltenbrunner <stefan@kaltenbrunner.cc>:
> On 05/11/2011 09:25 PM, David E. Wheeler wrote:
>>
>> On May 11, 2011, at 12:18 PM, Stefan Kaltenbrunner wrote:
>>
>>>> Looks like 5.12.3 has been built for sid:
>>>>
>>>>   http://packages.debian.org/search?keywords=perl
>>>>
>>>> Is that do-able? Would save me some effort to use that (effort better
>>>> spent on community auth integration).
>>>
>>> I would stringly prefer to stay on 5.10 from squeeze if that is doable,
>>> manual backporting of such a huge package like perl with its millions of
>>> forward and reverse dependencies will cause no end of pain :(
>>> Running the (available) postgresql 9.0 backport is a breeze compared to
>>> that however.
>>
>> Well, if I could compile 5.12, I'd install it in /usr/local/. No need to
>> replace the system Perl at all. That's how I generally work with this stuff:
>> Leave the system Perl for system tasks; build my own Perl for the apps I
>> build.
>
> yeah and now the fun starts... "ok we use the packaged postgresql - in need
> plperl in ther to be 5.12", "we cannot use the packaged DBD::Pg because that
> one needs to be compiled agains 5.12 as well", "an oh because it is simpled
> we cannot use any packaged perl lib at all because it is so much easier if I
> just install my own copy that will never be security tracked".
> We(mostly magnus) basically spent man years to get the new infrastructure up
> and long term maintainable and having to start supporting random hand
> compiled complex packages (again) on them does not sound like the right
> thing to do - it is where we came from and we defintely dont want to get
> back there.
> If it really needs perl 5.12 (and maybe other stuff) I would rather think it
> is better to keep pgxn as a seperate entity for the time being.
> Infrastructure is all about reliability, sustainability and manageability -
> that often does not mix too well with developer needs but that is how it
> is...

hey ! you scared me : will sysadmin use pgxn to install non-security
tracked extension stuff for their favorites developers and
co-workers....

just a poll for sysadmins:

* will you allow cpan usage ?
* will you allow pgxn usage ?

I though cpan has something to track the modules built locally and
update them all. As probably have pgxn (if cpan has) or will have at
some point. Am I wrong ?

>
>
> Stefan
>
>
> --
> Sent via pgsql-www mailing list (pgsql-www@postgresql.org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-www
>



--
Cédric Villemain               2ndQuadrant
http://2ndQuadrant.fr/     PostgreSQL : Expertise, Formation et Support


Re: PGXN Hosting

From
"David E. Wheeler"
Date:
On May 11, 2011, at 12:28 PM, Devrim GÜNDÜZ wrote:

> We immortalks are reading a thread between a developer and sysadmins :)

No question. :-)

D



Re: PGXN Hosting

From
Devrim GÜNDÜZ
Date:
On Wed, 2011-05-11 at 21:55 +0200, Cédric Villemain wrote:
> just a poll for sysadmins:
>
> * will you allow cpan usage ?
> * will you allow pgxn usage ?

No (seriously)
--
Devrim GÜNDÜZ
Principal Systems Engineer @ EnterpriseDB: http://www.enterprisedb.com
PostgreSQL Danışmanı/Consultant, Red Hat Certified Engineer
Community: devrim~PostgreSQL.org, devrim.gunduz~linux.org.tr
http://www.gunduz.org  Twitter: http://twitter.com/devrimgunduz

Re: PGXN Hosting

From
"David E. Wheeler"
Date:
On May 11, 2011, at 12:45 PM, Stefan Kaltenbrunner wrote:

> yeah and now the fun starts... "ok we use the packaged postgresql - in need plperl in ther to be 5.12", "we cannot
usethe packaged DBD::Pg because that one needs to be compiled agains 5.12 as well", "an oh because it is simpled we
cannotuse any packaged perl lib at all because it is so much easier if I just install my own copy that will never be
securitytracked". 
> We(mostly magnus) basically spent man years to get the new infrastructure up and long term maintainable and having to
startsupporting random hand compiled complex packages (again) on them does not sound like the right thing to do - it is
wherewe came from and we defintely dont want to get back there. 
> If it really needs perl 5.12 (and maybe other stuff) I would rather think it is better to keep pgxn as a seperate
entityfor the time being. 
> Infrastructure is all about reliability, sustainability and manageability - that often does not mix too well with
developerneeds but that is how it is... 

Yeah, I don't want to create maintenance nightmares for you guys. That's why I said I'm happy to manage  it myself.

I don't think it *has* to use Perl 5.12; I could do the work to backport it, I don't think it'd be a ton of work,
mostlyjust an annoyance. I'd strongly prefer 5.12, obviously, but if it's just not do-able, and you don't want me
buildingstuff separate from the Debian-packaged stuff, I could make it work. 

There almost certainly will be some CPAN modules I use not packaged, though.

Best,

David





Re: PGXN Hosting

From
"David E. Wheeler"
Date:
On May 11, 2011, at 12:55 PM, Cédric Villemain wrote:

> I though cpan has something to track the modules built locally and
> update them all. As probably have pgxn (if cpan has) or will have at
> some point. Am I wrong ?

Not sure what you mean by this. Daniele Varrazzo has been writing a PGXN client, though. Are you thinking of something
like
   pgxn update all

That would install the latest version of all installed extensions?

As for packaging what's on PGXN, I've discussed a bit with Daniele, Magnus, and Devrim (via Twitter) the idea of having
somekind of automated way for PGXN distributions to automatically be repackaged for RPM and/or .deb and/or StackBuilder
distribution.

Best,

David




Re: PGXN Hosting

From
Magnus Hagander
Date:
On Wed, May 11, 2011 at 22:05, David E. Wheeler <david@kineticode.com> wrote:
> On May 11, 2011, at 12:55 PM, Cédric Villemain wrote:
>
>> I though cpan has something to track the modules built locally and
>> update them all. As probably have pgxn (if cpan has) or will have at
>> some point. Am I wrong ?
>
> Not sure what you mean by this. Daniele Varrazzo has been writing a PGXN client, though. Are you thinking of
somethinglike 
>
>    pgxn update all
>
> That would install the latest version of all installed extensions?

Well, what's needed for the infrastructure is:

* A munin/nagios compatible monitor that will alert when *security or
other critical updates* are avaiable. It should *not* fire off alerts
if it's just a new feature release, we're only interested in patches.
* A "pgxn update all" command like the one you mentioned, but one that
again only deals with patches and security updates, not new versions.

We want to be able to basically automate monitoring and updating
*without* risking compatibility problems.

IIRC you can't do that with CPAN today for example, which is why
people tend to prefer to rely on the DEB packaged CPAN modules,
because then the packager takes on the responsibility to backpatch
fixes.


> As for packaging what's on PGXN, I've discussed a bit with Daniele, Magnus, and Devrim (via Twitter) the idea of
havingsome kind of automated way for PGXN distributions to automatically be repackaged for RPM and/or .deb and/or
StackBuilderdistribution. 

Yeah, that's definitely something that would be *very* useful.

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/


Re: PGXN Hosting

From
Cédric Villemain
Date:
2011/5/11 David E. Wheeler <david@kineticode.com>:
> On May 11, 2011, at 12:55 PM, Cédric Villemain wrote:
>
>> I though cpan has something to track the modules built locally and
>> update them all. As probably have pgxn (if cpan has) or will have at
>> some point. Am I wrong ?
>
> Not sure what you mean by this. Daniele Varrazzo has been writing a PGXN client, though. Are you thinking of
somethinglike 
>
>    pgxn update all

yes, just wondering if it exists in cpan and pgxn.

>
> That would install the latest version of all installed extensions?

yes, like aptitude safe-upgrade does.

>
> As for packaging what's on PGXN, I've discussed a bit with Daniele, Magnus, and Devrim (via Twitter) the idea of
havingsome kind of automated way for PGXN distributions to automatically be repackaged for RPM and/or .deb and/or
StackBuilderdistribution. 

cool ! that is a good idea too.

>
> Best,
>
> David
>
>
>



--
Cédric Villemain               2ndQuadrant
http://2ndQuadrant.fr/     PostgreSQL : Expertise, Formation et Support


Re: PGXN Hosting

From
"David E. Wheeler"
Date:
On May 11, 2011, at 1:09 PM, Magnus Hagander wrote:

> IIRC you can't do that with CPAN today for example, which is why
> people tend to prefer to rely on the DEB packaged CPAN modules,
> because then the packager takes on the responsibility to backpatch
> fixes.

That's correct. It's the same for PGXN.

>> As for packaging what's on PGXN, I've discussed a bit with Daniele, Magnus, and Devrim (via Twitter) the idea of
havingsome kind of automated way for PGXN distributions to automatically be repackaged for RPM and/or .deb and/or
StackBuilderdistribution. 
>
> Yeah, that's definitely something that would be *very* useful.

I think it'd be pretty do-able.

Best,

David




Re: PGXN Hosting

From
David E. Wheeler
Date:
On May 11, 2011, at 1:02 PM, David E. Wheeler wrote:

> On May 11, 2011, at 12:45 PM, Stefan Kaltenbrunner wrote:
>
>> yeah and now the fun starts... "ok we use the packaged postgresql - in need plperl in ther to be 5.12",

Oh, and this reminds me, Manager does make extensive use of PL/Perl. I assume the 9.0 backport includes it? Also
contribmodule (like citext) and, as I said, the "semver" extension, distributed on PGXN. 

As for Perl modules, PGXN::Manager requires:
   build_requires     => {       'Archive::Tar'                      => '1.68',       'File::Copy'
 => '2.18',       'HTTP::Request::Common'             => '5.824',       'MIME::Base64'                      => '3.08',
    'Module::Build::DB'                 => '0.10',       'Software::License'                 => '0.102340',
'TAP::Parser::SourceHandler::pgTAP'=> '3.23',       'Test::File'                        => '1.29',
'Test::File::Contents'             => '0.05',       'Test::Harness'                     => '3.18',
'Test::MockModule'                 => '0.05',       'Test::More'                        => '0.70',
'Test::NoWarnings'                 => '1.02',       'Test::XML'                         => '0.08',       'Test::XPath'
                    => '0.13',   },   requires => {       'aliased'                           => '0.30',
'Archive::Extract'                 => '0.42',       'Archive::Zip'                      => '1.30',       'Class::ISA'
                    => '0.36',       'Cwd'                               => '3.33',       'Data::Dump'
     => '1.19',       'Data::Validate::URI'               => '0.05',       'DBD::Pg'                           =>
'2.15.1',      'DBI'                               => '1.608',       'DBIx::Connector'                   => '0.42',
 'Digest::SHA1'                      => '2.13',       'Email::Valid'                      => '0.184',
'Email::MIME::Creator'             => '1.905',       'Email::Sender::Simple'             => '0.102370',
'Email::Address'                   => '1.892',       'Encode'                            => '2.40',
'Exception::Class::DBI'            => '1.00',       'File::Basename'                    => '2.78',       'File::Copy'
                    => '2.18',       'File::Path'                        => '2.08',       'File::Spec'
     => '3.33',       'File::Temp'                        => '0.22',       'Getopt::Long'                      =>
'2.34',      'HTTP::Body'                        => '1.08',       'HTML::Entities'                    => '3.68',
'HTTP::Negotiate'                  => '5.835',       'I18N::LangTags::Detect'            => '1.04',       'JSON::XS'
                     => '2.3',       'Locale::Maketext'                  => '1.13',       'Moose'
     => '1.15',       'MooseX::Singleton'                 => '0.25',       'Moose::Util::TypeConstraints'      =>
'1.15',      'namespace::autoclean'              => '0.11',       'parent'                            => '0.223',
'Net::Twitter::Lite'               => '0.10003',       'perl'                              => 5.012,       'Plack'
                      => '0.9956',       'Plack::Builder'                    => 0,       'Plack::App::File'
    => 0,       'Plack::Response'                   => '0.9956',       'Plack::Session::Store::File'       => '0.12',
   'Plack::Middleware::Session'        => '0.12',       'Plack::Middleware::MethodOverride' => '0.10',
'Pod::Usage'                       => '1.33',       'namespace::autoclean'              => '0.11',
'Router::Resource'                 => '0.11',       'SemVer'                            => '0.1.0',
'Template::Declare'                => '0.43',       'Template::Declare::Tags'           => '0.43',       'Try::Tiny'
                    => '0.06',       'URI::Template'                     => '0.15',   },   recommends => {
'Plack::Middleware::ReverseProxy'=> '0.07',       'Test::Pod'                       => '1.41',
'Test::Pod::Coverage'            => '1.06',   }, 

PGXN::API requires:
   build_requires     => {       'Module::Build'        => '0.30',       'Software::License'    => '0.102340',
'Test::Exception'     => '0.31',       'Test::File'           => '1.29',       'Test::File::Contents' => '0.20',
'Test::MockModule'    => '0.05',       'Test::More'           => '0.70',       'Test::Output'         => '0.16',   },
requires=> {       'Archive::Zip'                 => '1.30',       'Cwd'                          => '3.33',
'Data::Dump'                  => '1.17',       'Digest::SHA1'                 => '2.13',       'Email::MIME::Creator'
     => '1.905',       'Email::Sender::Simple'        => '0.102370',       'File::Path'                   => '2.08',
  'File::Copy::Recursive'        => '0.38',       'File::Spec'                   => '3.33',       'JSON'
        => '2.27',       'JSON::XS'                     => '2.3',       'KinoSearch'                   => '0.311',
'List::Util'                   => '1.23',       'List::MoreUtils'              => '0.30',       'Moose'
      => '1.15',       'Moose::Util::TypeConstraints' => '1.15',       'MooseX::Singleton'            => '0.25',
'namespace::autoclean'        => '0.11',       'perl'                         => 5.012,       'PGXN::API::Searcher'
    => '0.9.0',       'Plack'                        => '0.9977',       'Plack::App::Directory'        => 0,
'Plack::App::File'            => 0,       'Plack::Middleware::JSONP'     => 0,       'Plack::Builder'               =>
0,      'Text::Markup'                 => '0.12',       'URI::Template'                => '0.15',       'XML::LibXML'
              => '1.70',   },   recommends => {       'Test::Pod'           => '1.41',       'Test::Pod::Coverage' =>
'1.06',  }, 

And PGXN::Site requires:
   build_requires     => {       'Module::Build'        => '0.30',       'Software::License'    => '0.102340',
'Test::File::Contents'=> '0.20',       'Test::More'           => '0.70',       'Test::MockModule'     => '0.05',
'Text::MultiMarkdown' => '1.000033',   },   requires => {       'Data::Dump'                   => '1.17',
'Encode'                      => '2.40',       'Gravatar::URL'                => '1.02',       'File::Basename'
     => 0,       'HTML::TagCloud'               => '0.34',       'I18N::LangTags::Detect'       => '1.04',
'Locale::Maketext'            => '1.13',       'List::Util'                   => '1.20',       'namespace::autoclean'
     => '0.11',       'perl'                         => 5.012,       'Plack'                        => '0.9956',
'Plack::Builder'              => 0,       'Plack::Request'               => '0.9956',       'Plack::Response'
  => '0.9956',       'SemVer'                       => '0.2.0',       'Template::Declare'            => '0.43',
'Template::Declare::Tags'     => '0.43',       'WWW::PGXN'                    => '0.12.0',   },   test_recommends => {
    'HTTP::Request::Common' => '5.824',       'Test::Pod'             => '1.41',       'Test::Pod::Coverage'   =>
'1.06',      'Test::XML'             => '0.08',       'Test::XPath'           => '0.13',   }, 

BTW, PGXN::API and PGXN::Site are distributed on CPAN.

Thanks,

David



Re: PGXN Hosting

From
Devrim GÜNDÜZ
Date:
On Wed, 2011-05-11 at 13:37 -0700, David E.Wheeler wrote:
> As for Perl modules, PGXN::Manager requires:

<snip>

/me smells Fedora rawhide.
--
Devrim GÜNDÜZ
Principal Systems Engineer @ EnterpriseDB: http://www.enterprisedb.com
PostgreSQL Danışmanı/Consultant, Red Hat Certified Engineer
Community: devrim~PostgreSQL.org, devrim.gunduz~linux.org.tr
http://www.gunduz.org  Twitter: http://twitter.com/devrimgunduz

Re: PGXN Hosting

From
"David E. Wheeler"
Date:
On May 11, 2011, at 1:40 PM, Devrim GÜNDÜZ wrote:

> /me smells Fedora rawhide.

You lost me. :-( I really don't care what distribution we use, as long as it's Unix-ish.

Best,

David



Re: PGXN Hosting

From
Magnus Hagander
Date:
2011/5/11 David E. Wheeler <david@kineticode.com>:
> On May 11, 2011, at 1:40 PM, Devrim GÜNDÜZ wrote:
>
>> /me smells Fedora rawhide.
>
> You lost me. :-( I really don't care what distribution we use, as long as it's Unix-ish.

Fedora Rawhide is the bleeding edge, nightly snapshot, build of
Fedora. That nobody even remotely sane runs on a server - they even
recommend you stay away from it on workstations.

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/


Re: PGXN Hosting

From
"David E. Wheeler"
Date:
On May 11, 2011, at 1:45 PM, Magnus Hagander wrote:

>>> /me smells Fedora rawhide.
>>
>> You lost me. :-( I really don't care what distribution we use, as long as it's Unix-ish.
>
> Fedora Rawhide is the bleeding edge, nightly snapshot, build of
> Fedora. That nobody even remotely sane runs on a server - they even
> recommend you stay away from it on workstations.

Ah. Let's not do that, then.

David



Re: PGXN Hosting

From
"David E. Wheeler"
Date:
On May 11, 2011, at 1:02 PM, David E. Wheeler wrote:

> I don't think it *has* to use Perl 5.12; I could do the work to backport it, I don't think it'd be a ton of work,
mostlyjust an annoyance. I'd strongly prefer 5.12, obviously, but if it's just not do-able, and you don't want me
buildingstuff separate from the Debian-packaged stuff, I could make it work. 

Okay, I've backported everything to support Perl 5.10, so that's not a barrier anymore.

Best,

David

Re: PGXN Hosting

From
"David E. Wheeler"
Date:
On May 11, 2011, at 11:22 AM, Magnus Hagander wrote:

> There isn't. The question wasn't clear enough - how much work would it
> take to make it run on the standard version of perl on debian squeeze,
> which is 5.10?

Not a lot, but annoying. You're not going to make me use PostgreSQL 8.x too, are you?

Which 5.10?

> And I assume you have a list of the CPAN modules somewhere :-) I'm
> sure we can work that out.

Yes.

>> I'm happy to integrate with the community login system. It's just a simple matter of programming. :-) Is there now
anAPI for it? 
>
> Yeah, it's trivial - connect to a PostgreSQL server with specified
> credentials, and run SELECT community_login().

Oh, really? That'd work I guess.

> Since you're a perl guy,
> http://git.postgresql.org/gitweb?p=pgcommitfest.git;a=summary is
> probably a good reference implementation.

Thanks.

Best,

David



Re: PGXN Hosting

From
David E. Wheeler
Date:
On May 11, 2011, at 5:06 PM, David E. Wheeler wrote:

> On May 11, 2011, at 1:02 PM, David E. Wheeler wrote:
>
>> I don't think it *has* to use Perl 5.12; I could do the work to backport it, I don't think it'd be a ton of work,
mostlyjust an annoyance. I'd strongly prefer 5.12, obviously, but if it's just not do-able, and you don't want me
buildingstuff separate from the Debian-packaged stuff, I could make it work. 
>
> Okay, I've backported everything to support Perl 5.10, so that's not a barrier anymore.

So, what's the next step here? Do you guys need to configure the host for me? Or do you need something more from me
rightnow? 

Thanks,

David



Re: PGXN Hosting

From
Devrim GÜNDÜZ
Date:
On Mon, 2011-05-16 at 11:03 -0700, David E.Wheeler wrote:
>
> So, what's the next step here?

(Almost) everyone in the web and infrastructure teams will be at PGCon.
You can pester whoever you like :)
--
Devrim GÜNDÜZ
Principal Systems Engineer @ EnterpriseDB: http://www.enterprisedb.com
PostgreSQL Danışmanı/Consultant, Red Hat Certified Engineer
Community: devrim~PostgreSQL.org, devrim.gunduz~linux.org.tr
http://www.gunduz.org  Twitter: http://twitter.com/devrimgunduz