Thread: Increased company involvement
I am very excited to see companies involved in PostgreSQL development. It gives us funding for developers and features that is new for us. We had Fujitsu funding some features for 8.0 and that really helped us. However, there was a lot of coordination that happened with Fujitsu that I don't see happening with the current companies involved. Companies are already duplicating work that is also done by community members or by other companies. The big issue is communication. Because the PostgreSQL code base is common for most of the companies involved, there has to be coordination in what they are working on and their approaches. If that doesn't happen, two companies will work on the same feature, and only one can be added, or a complex process of merging the two patches into one patch has to happen --- again duplicated effort. I am willing to do the coordination, or even better, have the companies involved publicly post their efforts so all the other companies can know what is happening. I realize this is hard for companies because their efforts are in some ways part of their profitability. Does profitability require duplication of effort and code collisions? I am not sure, but if it does, we are in trouble. I am not sure the community has the resources to resolve that many collisions. Second, some developers are being hired from the community to work on closed-source additions to PostgreSQL. That is fine and great, but one way to kill PostgreSQL is to hire away its developers. If a commercial company wanted to hurt us, that is certainly one way they might do it. Anyway, it is a concern I have. I am hoping community members hired to do closed-source additions can at least spend some of their time on community work. And finally, we have a few companies working on features that they eventually want merged back into the PostgreSQL codebase. That is a very tricky process and usually goes badly unless the company seeks community involvement from the start, including user interface, implementation, and coding standards. I hate to be discouraging here, but I am trying to communicate what we have learned over the past few years to help companies be effective in working with open source communities. I am available to talk to any company that wants further details. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania 19073
> However, there was a lot of coordination that happened with Fujitsu that > I don't see happening with the current companies involved. Companies > are already duplicating work that is also done by community members or > by other companies. That is bound to happen no matter what. Look at plJava and plJ. Some people just feel that their way is better. Some people just don't get along etc... That is why we have 80 Linux distributions and a dozen FreeBSD distributions (can I include MacOSX?). > The big issue is communication. Because the > PostgreSQL code base is common for most of the companies involved, there > has to be coordination in what they are working on and their approaches. I can see this as an issue but sometimes that community is a hampering course as well. I recognize the community goals and respect them but in some things the community can move really slow. From what I can tell some of this is caused by the no new features rules etc... In business moving slow can mean death to a project. Which is why (hate to beat a dead horse) many OSS projects have moved to 6 month release cycles. > is happening. I realize this is hard for companies because their > efforts are in some ways part of their profitability. That is true, there are sometimes strategic reasons to not annouce a project. > profitability require duplication of effort and code collisions? I am > not sure, but if it does, we are in trouble. I am not sure the > community has the resources to resolve that many collisions. Which is why you are starting to see forks such as Bizgres but it is also why you are seeing forks go away (Mammoth PostgreSQL). > Second, some developers are being hired from the community to work on > closed-source additions to PostgreSQL. That is fine and great, but one > way to kill PostgreSQL is to hire away its developers. If a commercial > company wanted to hurt us, that is certainly one way they might do it. > Anyway, it is a concern I have. I am hoping community members hired to > do closed-source additions can at least spend some of their time on > community work. I would think that most of the developers would stipulate that in order to take the position??? I know Command Prompt would always make sure that the developer could work on the community stuff. > And finally, we have a few companies working on features that they > eventually want merged back into the PostgreSQL codebase. That is a > very tricky process and usually goes badly unless the company seeks > community involvement from the start, including user interface, > implementation, and coding standards. I concur with this. We ran into this with plPerl. Sincerely, Joshua D. Drake Command Prompt, Inc. -- Your PostgreSQL solutions provider, Command Prompt, Inc. 24x7 support - 1.800.492.2240, programming, and consulting Home of PostgreSQL Replicator, plPHP, plPerlNG and pgPHPToolkit http://www.commandprompt.com / http://www.postgresql.org
> And finally, we have a few companies working on features that they > eventually want merged back into the PostgreSQL codebase. That is a > very tricky process and usually goes badly unless the company seeks > community involvement from the start, including user interface, > implementation, and coding standards. You might be interested to know that someone is evening wanting to sponsor some phpPgAdmin development. Chris
Joshua D. Drake wrote: > > However, there was a lot of coordination that happened with Fujitsu that > > I don't see happening with the current companies involved. Companies > > are already duplicating work that is also done by community members or > > by other companies. > > That is bound to happen no matter what. Look at plJava and plJ. Some > people just feel that their way is better. Some people just don't get > along etc... > > That is why we have 80 Linux distributions and a dozen FreeBSD > distributions (can I include MacOSX?). Do companies want to write for Blue Hat PostgreSQL and Suza PostgreSQL because that might be what happens if we don't stay organized? In fact, it might have be happening already. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania 19073
> > Do companies want to write for Blue Hat PostgreSQL and Suza PostgreSQL > because that might be what happens if we don't stay organized? In fact, > it might have be happening already. Well that depends... If the companies are writing for Pervasive PostgreSQL I don't think they would have a problem with that ;). And I do agree with you Bruce, it is happening already -- I don't think there is any question in that. Sincerely, Joshua D. Drake > -- Your PostgreSQL solutions provider, Command Prompt, Inc. 24x7 support - 1.800.492.2240, programming, and consulting Home of PostgreSQL Replicator, plPHP, plPerlNG and pgPHPToolkit http://www.commandprompt.com / http://www.postgresql.org
Joshua D. Drake wrote: > > > > > Do companies want to write for Blue Hat PostgreSQL and Suza PostgreSQL > > because that might be what happens if we don't stay organized? In fact, > > it might have be happening already. > > Well that depends... If the companies are writing for Pervasive > PostgreSQL I don't think they would have a problem with that ;). > > And I do agree with you Bruce, it is happening already -- I don't think > there is any question in that. Oh, well good I am not seeing phantoms. I have had time to focus on where we are for the past few days and I am worried about what I see. It can be great, or it can be a mess, for both us and the companies involved, and I want to work to make it the former. The fact is, we now have more and more companies relying on us to be a success, and that is an added responsibility for us. We can make it work, but just hoping for it isn't enough --- we have to focus on the issues early and plan for that success. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania 19073
Joshua D. Drake wrote: >> However, there was a lot of coordination that happened with Fujitsu that >> I don't see happening with the current companies involved. Companies >> are already duplicating work that is also done by community members or >> by other companies. > > > That is bound to happen no matter what. Look at plJava and plJ. Some > people just feel that their way is better. Some people just don't get > along etc... > Actually, I think that PL/Java versus PL/J is a good example of where some coordination would have helped a lot. The short story: I was between jobs in December 2003 through February the following year. A lot of work on PL/Java was made during that time. I had no clue that there was another active project with similar objectives until after my first fully functional submission to gborg. Had I known, the outcome would have been different. Today there are ongoing and very active efforts to collaborate. The longer story (if anyone is interested): Before I started PL/Java I informed the community of my intentions (see hackers thread "pljava revisited" http://archives.postgresql.org/pgsql-hackers/2003-12/msg00310.php ). I got a lot of feedback and good advice such as using C instead of C++, hosting the project at gborg, etc. but nobody told me back then that there was an active PL/J project. I found traces of that project on sourceforge but it seemed to have been dead for over a year. At that time there was no redirect from sourceforge. Jan Wieck started the thread "PL/Java issues" http://archives.postgresql.org/pgsql-hackers/2003-12/msg00819.php in which I made 2 major posts. Nothing in that thread indicated that there was an ongoing project and I got no reply to my posts. On January the 7th, I made my first submission to gborg. When I, in mid February, realized that the PL/J project was indeed alive and active, I wrote the "PL/Java - next step?" http://archives.postgresql.org/pgsql-hackers/2004-02/msg00713.php where I outlined possible futures for PL/Java and PL/J. The outcome of that was that PL/J (Dave Cramer and Laszlo Hornyak) and I had an IRC meeting where we agreed on some limited collaboration (see "Minutes from Pl/Java -next step IRC" http://archives.postgresql.org/pgsql-hackers/2004-03/msg00171.php ). From that point and until a month or so ago, the active collaboration between the projects could have been better. Some things did happen though. Laszlo asked me to publish some PL/Java interfaces in a public maven repository which I did and we had some discussions. I made an attempt to have a major sponsor step in and take the lead in a project aiming to provide a flexible solution where a choice of approach could be made but the sponsor understandably wanted to wait and see. Today, we (the PL/Java and PL/J project members) make common efforts to factor out client tools that indeed can be common to a separate project. We are also discussing how to make the PostgreSQL user experience as similar as possible and thus allowing use of PL/Java or PL/J without changing anything but configuration. Kind regards, Thomas Hallgren
On K, 2005-04-27 at 22:21 -0700, Joshua D. Drake wrote: > > However, there was a lot of coordination that happened with Fujitsu that > > I don't see happening with the current companies involved. Companies > > are already duplicating work that is also done by community members or > > by other companies. > > That is why we have 80 Linux distributions and a dozen FreeBSD > distributions (can I include MacOSX?). I guess more aprropriate comparison would be to distibution specific linux kernels, i.e. RedHat linux kernel vs. suse linux kernel v.s. "vanilla" or "real" :) linux kernel. > > The big issue is communication. Because the > > PostgreSQL code base is common for most of the companies involved, there > > has to be coordination in what they are working on and their approaches. > > I can see this as an issue but sometimes that community is a hampering > course as well. I recognize the community goals and respect them but in > some things the community can move really slow. From what I can tell > some of this is caused by the no new features rules etc... > > In business moving slow can mean death to a project. > > Which is why (hate to beat a dead horse) many OSS projects have moved > to 6 month release cycles. Well, it is a two-sided thing. On one hand, businesses usually need new features "yesterday", but on the other hand, business would loose most of the benefit of getting the feature fast, if it is not included in the main branch along the road, preferrably in the next official release, because said business would be dependent of the implementor of his specific feature for integrating _all_ other new and desirable fetures of next releas in their specific version of postgres. > > is happening. I realize this is hard for companies because their > > efforts are in some ways part of their profitability. > > That is true, there are sometimes strategic reasons to not annouce a > project. I can see no strategic advantages for sponsors in not announcing the project. There may be some small tactical wins for the actual implementors, but not even tactical wins the sponsors. There may be some strategic/tactical reasons for sponsors to announce who they (the sponsors) are, but I can't see any reason not to announce the project or not to encourage/ask/demand the implementor to do so. > > profitability require duplication of effort and code collisions? I am > > not sure, but if it does, we are in trouble. I am not sure the > > community has the resources to resolve that many collisions. > > Which is why you are starting to see forks such as Bizgres but it is > also why you are seeing forks go away (Mammoth PostgreSQL). At first glance at least, BizGres looks like a community oriented project and I hope that BizGres will be a testbed/early implementation of some DataWarehouse features, which will be integrated back to postgres at first possibility. But I too expected the discussion to take place on pgsql-hackers, not some half-hidden mailinglist on pgfoundry. Or at least an announcement of that mailinglist to be made on pgsql-hachers. OTOH, there are some things (2PC, recursive queries, ...) which are discussed on psql-hacker and still are lingering somewhere and are for various reasons (one of them may be lack of time/money/sponsor-pressure for the developer) not in the postgres proper yet. OTOOH, I see that a growing number of companies who might think of sponsoring PG development are interested in BI/DW and OLAP features in addition to plain OLTP enchancements. -- Hannu Krosing <hannu@skype.net>
Hannu Krosing wrote: > > Which is why (hate to beat a dead horse) many OSS projects have moved > > to 6 month release cycles. > > Well, it is a two-sided thing. On one hand, businesses usually need new > features "yesterday", but on the other hand, business would loose most > of the benefit of getting the feature fast, if it is not included in the > main branch along the road, preferrably in the next official release, > because said business would be dependent of the implementor of his > specific feature for integrating _all_ other new and desirable fetures > of next releas in their specific version of postgres. Yes, you can bet that will happen. You can also bet that the community release will have 2X more feature additions than any commercial release that is a version behind unless _huge_ sums of money are thrown at the commercial release. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania 19073
On Thursday 28 April 2005 01:48, Joshua D. Drake wrote:
> > Do companies want to write for Blue Hat PostgreSQL and Suza PostgreSQL
> > because that might be what happens if we don't stay organized? In fact,
> > it might have be happening already.
>
> Well that depends... If the companies are writing for Pervasive
> PostgreSQL I don't think they would have a problem with that ;).
>
> And I do agree with you Bruce, it is happening already -- I don't think
> there is any question in that.
>
ISTM the allure of differentiation and branding is going to be too strong for
us to prevent such things. An easy way to differentiate is to add some
proprietary/unique extension to the main code and then package that up. If
you have to have all your extensions be put into the community version then
lose this advantage over your comptetitors. (Mammoth PostgreSQL/Replicator is
an example of this) The same holds true for branding.... if your Pervasive
you want to sell Pervasive Postgres rather than PostgreSQL because you get to
push your name out there, and get people thinking about your company whenever
they talk about the database.
I think our goal is to encorage companies to push these changes into the core
as much as possible, pointing out things like the advantages community
support brings like ongoing maintainance, support in add-on tools like
pgadmin or phppgadmin, and eliminating the chance that someone else will
submit a similar solution that gets accepted to the community code there by
deprecating the work they have already done.
--
Robert Treat
Build A Brighter Lamp :: Linux Apache {middleware} PostgreSQL
Hannu, > But I too expected the discussion to take place on pgsql-hackers, not > some half-hidden mailinglist on pgfoundry. Or at least an announcement > of that mailinglist to be made on pgsql-hachers. Yeah, we should announce the mailing list. Actually, I did direct e-mail a bunch of people (including you) about it and invite them to the mailing list. So: http://pgfoundry.org/mail/?group_id=1000107 For discussing potential features, though, I'm personally reluctant at this point to discuss major features until I and my collaborators (example, newsysviews) have a concrete proposal together. It's far too easy to get sidetracked into a discussion of minutia and politics on -hackers; by generating a complete draft spec (at least) on a small mailing list, it's a lot easier to focus on specific goals and schedules, and discussions on hackers around detailed proposals tend to be a lot more focused. -- Josh Berkus Aglio Database Solutions San Francisco
Robert Treat wrote: > ISTM the allure of differentiation and branding is going to be too strong for > us to prevent such things. An easy way to differentiate is to add some > proprietary/unique extension to the main code and then package that up. If > you have to have all your extensions be put into the community version then > lose this advantage over your comptetitors. (Mammoth PostgreSQL/Replicator is > an example of this) The same holds true for branding.... if your Pervasive > you want to sell Pervasive Postgres rather than PostgreSQL because you get to > push your name out there, and get people thinking about your company whenever > they talk about the database. > > I think our goal is to encorage companies to push these changes into the core > as much as possible, pointing out things like the advantages community > support brings like ongoing maintainance, support in add-on tools like > pgadmin or phppgadmin, and eliminating the chance that someone else will > submit a similar solution that gets accepted to the community code there by > deprecating the work they have already done. I remember something the president of Great Bridge told me, he said, "Great Bridge needs PostgreSQL. If Great Bridge dies, PostgreSQL goes on (as it did), but if PostgreSQL dies, Great Bridge is dead too". -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania 19073
I've deliberately let the dust settle slightly on this. One thing that might help is a more open sponsorship "clearing house". Example (not meant as a bid, but just to illustrate): the JDBC driver needs a scanner overhaul - it breaks on dollar quoting and a bunch of other stuff. I could do that work (as could others, of course) but I don't have time, unless someone buys some of my professional time. Someone might want to do just that, but how would they find me? Regarding the secret code stuff - I predict that it will quickly bite whoever does it, unless they are extremely lucky. cheers andrew Bruce Momjian wrote: >I am very excited to see companies involved in PostgreSQL development. >It gives us funding for developers and features that is new for us. We >had Fujitsu funding some features for 8.0 and that really helped us. > >However, there was a lot of coordination that happened with Fujitsu that >I don't see happening with the current companies involved. Companies >are already duplicating work that is also done by community members or >by other companies. The big issue is communication. Because the >PostgreSQL code base is common for most of the companies involved, there >has to be coordination in what they are working on and their approaches. > >If that doesn't happen, two companies will work on the same feature, and >only one can be added, or a complex process of merging the two patches >into one patch has to happen --- again duplicated effort. I am willing >to do the coordination, or even better, have the companies involved >publicly post their efforts so all the other companies can know what >is happening. I realize this is hard for companies because their >efforts are in some ways part of their profitability. Does >profitability require duplication of effort and code collisions? I am >not sure, but if it does, we are in trouble. I am not sure the >community has the resources to resolve that many collisions. > >Second, some developers are being hired from the community to work on >closed-source additions to PostgreSQL. That is fine and great, but one >way to kill PostgreSQL is to hire away its developers. If a commercial >company wanted to hurt us, that is certainly one way they might do it. >Anyway, it is a concern I have. I am hoping community members hired to >do closed-source additions can at least spend some of their time on >community work. > >And finally, we have a few companies working on features that they >eventually want merged back into the PostgreSQL codebase. That is a >very tricky process and usually goes badly unless the company seeks >community involvement from the start, including user interface, >implementation, and coding standards. > >I hate to be discouraging here, but I am trying to communicate what we >have learned over the past few years to help companies be effective in >working with open source communities. I am available to talk to any >company that wants further details. > > >
Andrew Dunstan wrote: > > I've deliberately let the dust settle slightly on this. > > One thing that might help is a more open sponsorship "clearing house". > Example (not meant as a bid, but just to illustrate): the JDBC driver > needs a scanner overhaul - it breaks on dollar quoting and a bunch of > other stuff. I could do that work (as could others, of course) but I > don't have time, unless someone buys some of my professional time. > Someone might want to do just that, but how would they find me? > > Regarding the secret code stuff - I predict that it will quickly bite > whoever does it, unless they are extremely lucky. Let me add that we really do want these companies to succeed. For me, it is all a matter of meeting the company's expectations. If a company wants to add a closed-source feature to PostgreSQL, that's fine, but if they work on a features and eventually want it integrated into the main code, I don't want that to fail and have them be disappointed. The same goes for duplicated effort --- if they don't wnat it to happen, I want to help them prevent it. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania 19073
Andrew Dunstan <andrew@dunslane.net> writes:
> Regarding the secret code stuff - I predict that it will quickly bite
> whoever does it, unless they are extremely lucky.
Yeah. Bruce and I were worrying about this on the phone today.
If a company is doing some work with the intent that it's a proprietary
extension they can sell, no problem --- the BSD license is specifically
intended to let them do that. What's bothering us is the thought that
people are off in corners developing code that they think they are going
to contribute back into the community code base "after it's done". Past
history shows that the odds of getting such things accepted into the PG
community code base are *very* bad if you didn't communicate with the
community from the start of your development process.
We'd like to avoid such unpleasant surprises, but how to get the word
out?
regards, tom lane
On Fri, 29 Apr 2005, Andrew Dunstan wrote: > One thing that might help is a more open sponsorship "clearing house". > Example (not meant as a bid, but just to illustrate): the JDBC driver > needs a scanner overhaul - it breaks on dollar quoting and a bunch of > other stuff. I could do that work (as could others, of course) but I > don't have time, unless someone buys some of my professional time. > Someone might want to do just that, but how would they find me? I don't think this is a big issue. I don't know of any companies who were desperate for a feature and willing to throw money at the problem who couldn't find a developer to take them up on it. Right now this seems to be a kind of behind the scenes operation that relies heavily on knowing the right people, but I think most of our sponsor contact points are able to point sponsors to the right people. Could this process be more open? Depends on how the sponsor wants to handle it, they probably don't just want to throw the task out there and see who comes calling, they want an assurance from someone they trust that the chosen developer is capable. One thing that definitely would be nice would be to be able to combine funds from various sponsors for various features. Alone a company can't spring for it, but by pooling resources it could get done. This is a lot tougher to coordinate and unless there is a complete spec in place different sponsors will pull in different directions. Other bounty type schemes don't seem to produce results, largely from a lack of cash. (Here's $500 for two weeks of work). Anyone care to shed some light on how it works now? Kris Jurka
> We'd like to avoid such unpleasant surprises, but how to get the word
> out?
More prominent placement of how to contribute would probably help. The
PGF could help with this as well once it is done. Right now it is ether
on how to contribute unless you know where to look.
Right now on the front page when we ask for support we are asking for
people to donate money. We don't need money. We need people. The support
link goes to bandwidth but a great deal of the project is hosted over
many, many servers with many providers. That really isn't as much of an
issue anymore. At least IMHO.
If it were I there would be a big link in bright text (not literally)
that says, "How to contribute" on the front page of the website. This
would go to a page that talks about the different ways to contribute
with contacts for each topic.
Who is head of Documentation?
Who can I talk to about submitting patches?
To libpq
To ECPG
To JDBC
etc....
What is the CVS policy?
Anyway... just some thoughts.
Sincerely,
Joshua D. Drake
Command Prompt, Inc.
On Fri, Apr 29, 2005 at 11:02:51PM -0700, Joshua D. Drake wrote: > Right now on the front page when we ask for support we are asking for > people to donate money. We don't need money. We need people. The support > link goes to bandwidth but a great deal of the project is hosted over > many, many servers with many providers. That really isn't as much of an > issue anymore. At least IMHO. If money's not an issue anymore, can we get a bigger box to host pgfoundry on then? :) -- Jim C. Nasby, Database Consultant decibel@decibel.org Give your computer some brain candy! www.distributed.net Team #1828 Windows: "Where do you want to go today?" Linux: "Where do you want to go tomorrow?" FreeBSD: "Are you guys coming, or what?"
> > > If money's not an issue anymore, can we get a bigger box to host > pgfoundry on then? :) It's been done and is in the process of being brought up at a new colo facility. There is also a backup box being built for failover purposes ;) Sincerely, Joshua D. Drake Command Prompt, Inc.
Anyone interested in pooling funds for features should take a look at http://people.freebsd.org/~phk/funding.html, which is about a FreeBSD developer who offered to work full-time on developing some specific features should enough people donate. Also worthy of mention is http://www.freebsdfoundation.org/. I think that for certain key features there's probably a lot of people who would fork over between $100 and $1000 towards getting a feature completed. Improved replication might be a good example. Table partitioning would absolutely be an example. If there was a means for these people to donate money towards work being done on some feature, it's very likely that large chunks of development time could be paid for just from smaller shops, let alone places that can afford to toss $20k towards the development of something. On Sat, Apr 30, 2005 at 12:47:45AM -0500, Kris Jurka wrote: > > > On Fri, 29 Apr 2005, Andrew Dunstan wrote: > > > One thing that might help is a more open sponsorship "clearing house". > > Example (not meant as a bid, but just to illustrate): the JDBC driver > > needs a scanner overhaul - it breaks on dollar quoting and a bunch of > > other stuff. I could do that work (as could others, of course) but I > > don't have time, unless someone buys some of my professional time. > > Someone might want to do just that, but how would they find me? > > I don't think this is a big issue. I don't know of any companies who were > desperate for a feature and willing to throw money at the problem who > couldn't find a developer to take them up on it. Right now this seems to > be a kind of behind the scenes operation that relies heavily on knowing > the right people, but I think most of our sponsor contact points are able > to point sponsors to the right people. Could this process be more open? > Depends on how the sponsor wants to handle it, they probably don't just > want to throw the task out there and see who comes calling, they want an > assurance from someone they trust that the chosen developer is capable. > > One thing that definitely would be nice would be to be able to combine > funds from various sponsors for various features. Alone a company can't > spring for it, but by pooling resources it could get done. This is a lot > tougher to coordinate and unless there is a complete spec in place > different sponsors will pull in different directions. Other bounty type > schemes don't seem to produce results, largely from a lack of cash. > (Here's $500 for two weeks of work). > > Anyone care to shed some light on how it works now? > > Kris Jurka > > ---------------------------(end of broadcast)--------------------------- > TIP 3: if posting/reading through Usenet, please send an appropriate > subscribe-nomail command to majordomo@postgresql.org so that your > message can get through to the mailing list cleanly > -- Jim C. Nasby, Database Consultant decibel@decibel.org Give your computer some brain candy! www.distributed.net Team #1828 Windows: "Where do you want to go today?" Linux: "Where do you want to go tomorrow?" FreeBSD: "Are you guys coming, or what?"
On Fri, Apr 29, 2005 at 11:21:35PM -0700, Joshua D. Drake wrote: > > > > > >If money's not an issue anymore, can we get a bigger box to host > >pgfoundry on then? :) > > It's been done and is in the process of being brought up at a new colo > facility. There is also a backup box being built for failover purposes ;) Any ETA? I don't mean to harp, but it looks really bad when someone new to postgresql comes to investigate something and the site is just crawling. -- Jim C. Nasby, Database Consultant decibel@decibel.org Give your computer some brain candy! www.distributed.net Team #1828 Windows: "Where do you want to go today?" Linux: "Where do you want to go tomorrow?" FreeBSD: "Are you guys coming, or what?"
> > Any ETA? I don't mean to harp, but it looks really bad when someone new > to postgresql comes to investigate something and the site is just > crawling. Well the backup should come up in a couple of weeks. I know that the new pgFoundry is being worked on right now. Josh would have a better idea. Sincerely, Joshua D. Drake
> Anyone interested in pooling funds for features should take a look at > http://people.freebsd.org/~phk/funding.html, which is about a FreeBSD > developer who offered to work full-time on developing some specific > features should enough people donate. Also worthy of mention is > http://www.freebsdfoundation.org/. This was really a great idea and we (my company) also supported it because we use freebsd as our primary os. We also use PostgreSQL as our primary db so it would be more than likely that we would donate money for something similar with postgresql if either : a) we can direct the money at one or more specific tasks or b) the tasks founded will be related to core postgresql features e.g. generel performance or other benefits that fits all. > I think that for certain key features there's probably a lot of people > who would fork over between $100 and $1000 towards getting a feature > completed. Yes - without any promise I would probably be able to raise between $1000 and $3000 in a period of the next 3 months. I would definately try it and I have multiple customers that have giving their intent on something like this. > Improved replication might be a good example. Table > partitioning would absolutely be an example. If there was a means for > these people to donate money towards work being done on some feature, > it's very likely that large chunks of development time could be paid for > just from smaller shops, let alone places that can afford to toss $20k > towards the development of something. I totally agree. In our preference list I would have the following tasks : 1) IOT (Index Ordered Tables) 2) Table partitioning 3) Better multimaster replication framework 4) Extending PostgreSQL's plugin support with additional hooks in the backend e.g. : - for adding new tablestore engines (like mysql can) - for adding callbacks that get's called on transaction success/failure using SPI. (e.g. for housekeeping and cleanup) 5) Adding parameter support for NOTIFY / LISTEN > Jim C. Nasby, Database Consultant decibel@decibel.org Nicolai Petri COD, catpipe systems - denmark Ps. sorry for the x-post - should this be moved to advocacy ?
On Sat, 30 Apr 2005, Nicolai Petri (lists) wrote: > We also use PostgreSQL as our primary db so it would be more than likely > that we would donate money for something similar with postgresql if > either : > a) we can direct the money at one or more specific tasks > or > b) the tasks founded will be related to core postgresql features e.g. > generel performance or other benefits that fits all. > The problem is organization. Who decides who gets what money? What about features that are paid for and worked on and not accepted into the community codebase? This was something I hoped the PostgreSQL Foundation http://thepostgresqlfoundation.org/ would step in and do, but we seem much more focused on advocacy efforts rather than developemnt ones. Kris Jurka
Tom Lane wrote: > Andrew Dunstan <andrew@dunslane.net> writes: > > Regarding the secret code stuff - I predict that it will quickly bite > > whoever does it, unless they are extremely lucky. > > Yeah. Bruce and I were worrying about this on the phone today. > If a company is doing some work with the intent that it's a proprietary > extension they can sell, no problem --- the BSD license is specifically > intended to let them do that. What's bothering us is the thought that > people are off in corners developing code that they think they are going > to contribute back into the community code base "after it's done". Past > history shows that the odds of getting such things accepted into the PG > community code base are *very* bad if you didn't communicate with the > community from the start of your development process. > > We'd like to avoid such unpleasant surprises, but how to get the word > out? Well, I received a private email with most of the companies involved as CC's and many of them have made comments on the issue, so I think we can say they saw the postings on this topic. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania 19073
Kris Jurka wrote: > One thing that definitely would be nice would be to be able to combine > funds from various sponsors for various features. Alone a company can't > spring for it, but by pooling resources it could get done. This is a lot > tougher to coordinate and unless there is a complete spec in place > different sponsors will pull in different directions. Other bounty type > schemes don't seem to produce results, largely from a lack of cash. > (Here's $500 for two weeks of work). > > Anyone care to shed some light on how it works now? I can tell you how I have done it. If a company contacts me to add a feature, I find the person I think is most qualified, check their availability, and then get them in touch with the donor. In some cases, I coordinate the work, and actually funnel the money and guarantee that the developer will be paid. I can do that because I often have an existing relationship with the funding company and the developer. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania 19073
On Saturday 30 April 2005 02:02, Joshua D. Drake wrote:
> > We'd like to avoid such unpleasant surprises, but how to get the word
> > out?
>
> More prominent placement of how to contribute would probably help. The
> PGF could help with this as well once it is done. Right now it is ether
> on how to contribute unless you know where to look.
>
> Right now on the front page when we ask for support we are asking for
> people to donate money. We don't need money. We need people. The support
> link goes to bandwidth but a great deal of the project is hosted over
> many, many servers with many providers. That really isn't as much of an
> issue anymore. At least IMHO.
At the least we could add a link in there with some info on sponsoring
development.
>
> If it were I there would be a big link in bright text (not literally)
> that says, "How to contribute" on the front page of the website. This
> would go to a page that talks about the different ways to contribute
> with contacts for each topic.
>
> Who is head of Documentation?
>
> Who can I talk to about submitting patches?
> To libpq
> To ECPG
> To JDBC
> etc....
>
> What is the CVS policy?
>
> Anyway... just some thoughts.
>
Some of this sounds like things that should be put together in the devlopers
faq.
--
Robert Treat
Build A Brighter Lamp :: Linux Apache {middleware} PostgreSQL
On Sat, Apr 30, 2005 at 10:37:06AM +0200, Nicolai Petri (lists) wrote: > >Anyone interested in pooling funds for features should take a look at > >http://people.freebsd.org/~phk/funding.html, which is about a FreeBSD > >developer who offered to work full-time on developing some specific > >features should enough people donate. Also worthy of mention is > >http://www.freebsdfoundation.org/. > This was really a great idea and we (my company) also supported it because > we use freebsd as our primary os. We also use PostgreSQL as our primary db > so it would be more than likely that we would donate money for something > similar with postgresql if either : > a) we can direct the money at one or more specific tasks > or > b) the tasks founded will be related to core postgresql features e.g. > generel performance or other benefits that fits all. > > >I think that for certain key features there's probably a lot of > >people who would fork over between $100 and $1000 towards getting a > >feature completed. > Yes - without any promise I would probably be able to raise between > $1000 and $3000 in a period of the next 3 months. I would definately > try it and I have multiple customers that have giving their intent > on something like this. Great! > >Improved replication might be a good example. Table > >partitioning would absolutely be an example. If there was a means for > >these people to donate money towards work being done on some feature, > >it's very likely that large chunks of development time could be paid for > >just from smaller shops, let alone places that can afford to toss $20k > >towards the development of something. > I totally agree. In our preference list I would have the following tasks : > 1) IOT (Index Ordered Tables) Is this different from CLUSTER? > 2) Table partitioning That'd be nice. It may be part of the bizgres effort, which underscores the point others have made about this being not exactly easy to find. > 3) Better multimaster replication framework Slony-2 will be one of these. It depends (I think) on 2PC, which appears slated for 8.1. :) > 4) Extending PostgreSQL's plugin support with additional hooks in the > backend e.g. : > - for adding new tablestore engines (like mysql can) To some approximation, DBI-Link http://pgfoundry.org/projects/dbi-link/ does this. Any help with this would be greatly appreciated, as it is quickly approaching the edge of my skillset. > - for adding callbacks that get's called on transaction > success/failure using SPI. (e.g. for housekeeping and cleanup) Would this be like an ON COMMIT TRIGGER? > 5) Adding parameter support for NOTIFY / LISTEN What's this for, and what cool stuff could one do with it? Cheers, D -- David Fetter david@fetter.org http://fetter.org/ phone: +1 510 893 6100 mobile: +1 415 235 3778 Remember to vote!
On Saturday 30 April 2005 17:40, David Fetter wrote:
> On Sat, Apr 30, 2005 at 10:37:06AM +0200, Nicolai Petri (lists) wrote:
> > I totally agree. In our preference list I would have the following tasks
> > : 1) IOT (Index Ordered Tables)
>
> Is this different from CLUSTER?
>
I think what he is looking for is a table that is maintained in CLUSTERed
order... ie. new inserts will be put in the appropriate place mimicing what
happens with indexes. This has been discussed in the past and has been shot
down before...see archives on pgsql-hackers for past discussions.
> > 4) Extending PostgreSQL's plugin support with additional hooks in the
> > backend e.g. :
> > - for adding new tablestore engines (like mysql can)
>
my$ql's multiple table handlers is about the ugliest hack I have ever seen
inside a database. I mean I used to not like it, but I've been studying it a
little bit lately and now find it absolutly dreadfull. I can't imagine that
we would ever implement a scheme like the one they have used; whatever
advantage people think they can gain from such a setup can probably be
implemented in a cleaner fasion in postgresql assuming it would be an
advantage at all (and some of those features have been dismissed in the past
as well so there's no garauntee)
> > - for adding callbacks that get's called on transaction
> > success/failure using SPI. (e.g. for housekeeping and cleanup)
>
> Would this be like an ON COMMIT TRIGGER?
>
Another idea I have seen shot down on hackers a number of times...
The point of this email being that, I think the general idea probably needs to
be that any coordinated community sponsored development needs to be centered
on picking your favorite items from the TODO list to be worked on rather than
just specifying a list of things you think should be the top priorities. Of
course there is nothing preventing people from getting new items added to the
todo list, but as a general guidline picking items from the todo gets you
halfway home right out of the gate.
--
Robert Treat
Build A Brighter Lamp :: Linux Apache {middleware} PostgreSQL
On Saturday 30 April 2005 06:05, Kris Jurka wrote: > On Sat, 30 Apr 2005, Nicolai Petri (lists) wrote: > > We also use PostgreSQL as our primary db so it would be more than likely > > that we would donate money for something similar with postgresql if > > either : > > a) we can direct the money at one or more specific tasks > > or > > b) the tasks founded will be related to core postgresql features e.g. > > generel performance or other benefits that fits all. > > The problem is organization. Who decides who gets what money? What about > features that are paid for and worked on and not accepted into the > community codebase? This was something I hoped the PostgreSQL Foundation > http://thepostgresqlfoundation.org/ would step in and do, but we seem much > more focused on advocacy efforts rather than developemnt ones. > We need to get our 501c status out of the way before we can really get involved in these issues. Once that happens there will certainly be opportunity to discuss what role we should play in pushing forward development. -- Robert Treat Build A Brighter Lamp :: Linux Apache {middleware} PostgreSQL
On Sat, 30 Apr 2005, Tom Lane wrote: > Andrew Dunstan <andrew@dunslane.net> writes: >> Regarding the secret code stuff - I predict that it will quickly bite >> whoever does it, unless they are extremely lucky. > > Yeah. Bruce and I were worrying about this on the phone today. > If a company is doing some work with the intent that it's a proprietary > extension they can sell, no problem --- the BSD license is specifically > intended to let them do that. What's bothering us is the thought that > people are off in corners developing code that they think they are going > to contribute back into the community code base "after it's done". Past > history shows that the odds of getting such things accepted into the PG > community code base are *very* bad if you didn't communicate with the > community from the start of your development process. > > We'd like to avoid such unpleasant surprises, but how to get the word > out? Well, any company that is going to do something like this is going to have some sort of representative on the lists ... why not write up a short FAQ, posted once a month to -hackers, explain how we contributions work, and that any company lookin gdown this direction should contact the list, and make sure that there is some sort of liason ... that is by no means the wording I'd suggestion, just the concept ... this thread will act as a good 'heads up' to those already here, but its something that needs to be brought up regularly for those 'new to the lists' ... ---- Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: scrappy@hub.org Yahoo!: yscrappy ICQ: 7615664
On Sat, 30 Apr 2005, Jim C. Nasby wrote: > On Fri, Apr 29, 2005 at 11:21:35PM -0700, Joshua D. Drake wrote: >>> >>> >>> If money's not an issue anymore, can we get a bigger box to host >>> pgfoundry on then? :) >> >> It's been done and is in the process of being brought up at a new colo >> facility. There is also a backup box being built for failover purposes ;) > > Any ETA? I don't mean to harp, but it looks really bad when someone new > to postgresql comes to investigate something and the site is just > crawling. When is the last time you checked it? While waiting for the new server to be ready, I've moved the site to the new server that just went online this past week ... its still not a rocket, but its not 'just crawling' either ... ---- Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: scrappy@hub.org Yahoo!: yscrappy ICQ: 7615664
On N, 2005-04-28 at 20:13 -0700, Josh Berkus wrote: > Hannu, > > > But I too expected the discussion to take place on pgsql-hackers, not > > some half-hidden mailinglist on pgfoundry. Or at least an announcement > > of that mailinglist to be made on pgsql-hachers. > > Yeah, we should announce the mailing list. Actually, I did direct e-mail a > bunch of people (including you) about it and invite them to the mailing list. Probably this got lost somewhere on the way. > For discussing potential features, though, I'm personally reluctant at this > point to discuss major features until I and my collaborators (example, > newsysviews) have a concrete proposal together. It's far too easy to get > sidetracked into a discussion of minutia and politics on -hackers; by > generating a complete draft spec (at least) on a small mailing list, it's a > lot easier to focus on specific goals and schedules, and discussions on > hackers around detailed proposals tend to be a lot more focused. Maybe :) , still not convinced though. There may be some quite fundamental things we may miss but which could be spotted by wider audience early on. -- Hannu Krosing <hannu@skype.net>
Martha Stewart called it a Good Thing when decibel@decibel.org ("Jim C. Nasby") wrote:
> Anyone interested in pooling funds for features should take a look at
> http://people.freebsd.org/~phk/funding.html, which is about a FreeBSD
> developer who offered to work full-time on developing some specific
> features should enough people donate. Also worthy of mention is
> http://www.freebsdfoundation.org/.
>
> I think that for certain key features there's probably a lot of people
> who would fork over between $100 and $1000 towards getting a feature
> completed. Improved replication might be a good example. Table
> partitioning would absolutely be an example. If there was a means for
> these people to donate money towards work being done on some feature,
> it's very likely that large chunks of development time could be paid for
> just from smaller shops, let alone places that can afford to toss $20k
> towards the development of something.
Note that money isn't necessarily the most useful thing.
For Slony-I, I can think of three places where specific contributions
of specific efforts could be really valuable, and potentially free up
other peoples' time to do "heavier lifting."
1. Fully scripted test cases.
There are about a dozen scripts that exist now that test for a number of known conditions, either generally
checkingto see if replication is functioning, or trying to exercise particular bits of functionaly, or verifying
thatcertain bugs are either present or absent.
There are some tests I'd like to set up but never get time to script up. Improving the tests that can be
_trivially_run would be a big help, and would have a general positive effect on reliability.
2. Documentation
I tried to get someone to write up "how to do PG upgrades using Slony-I"; wound up essentially writing it
myself.
There is plenty of room for "How I Did Foo With ..." with Things Other Haven't Tried yet. For instance, I
haven'ttried Slony-I on cases involving inheritance; a test script and a document on this would be super.
3. Actually requesting features
There is a small queue of outside-requested features for Slony-I, but the queue is pretty small. The vast
majorityof things queued have come from discussions between about 4 people, all of whom are writing code for the
project.
I daresay I am being totally myopic here, thinking only of "my
project." There lie my priorities, tough luck :-)!
I'd hazard the guess that would-be contributors might be better off
contributing relatively small things like improving documentation or
assisting by providing usefully detailed test cases than they would be
in contributing small sums of money.
It is _really_ not obvious how specks of money can be usefully put
together to get bigger features to happen.
I think a REALLY valuable thing would be if we could get another
person that was pretty expert with the query optimizer. The only way
to do that is to get someone to spend a year fighting with it.
Throwing a thousand dollars at someone here and there isn't likely to
direct them towards that.
--
output = ("cbbrowne" "@" "gmail.com")
http://linuxfinances.info/info/rdbms.html
programmer, n: A red eyed, mumbling mammal capable of conversing with inanimate monsters.
Jurka, > One thing that definitely would be nice would be to be able to combine > funds from various sponsors for various features. Alone a company can't > spring for it, but by pooling resources it could get done. This is a lot > tougher to coordinate and unless there is a complete spec in place > different sponsors will pull in different directions. Other bounty type > schemes don't seem to produce results, largely from a lack of cash. > (Here's $500 for two weeks of work). Actually, I talked to Opensoucexperts.com about this ages ago and they set up an online bounty system for OSS projects in general. I know that other players in the OSS space have talked about similar things; also companies like SRA and CMD are willing to act as development $$$ funnels for multi-party projects. > The problem is organization. Who decides who gets what money? What about > features that are paid for and worked on and not accepted into the > community codebase? This was something I hoped the PostgreSQL Foundation > http://thepostgresqlfoundation.org/ would step in and do, but we seem much > more focused on advocacy efforts rather than developemnt ones. Unfortunately, pooling funds for development is not something a non-profit can realistically do in the US without a whole lot of legal/tax help to navigate US law. Here NPOs are strictly defined as non-commercial. It might make sense to set up an NPO in another country, such as Australia, where the regulations on such things are much more liberal. More importantly, the Foundation is *still* waiting on its NPO paperwork from the IRS, and I really don't want to do any major fundraising while there's still the possibility we could be denied. > Well the backup should come up in a couple of weeks. I know that the new > pgFoundry is being worked on right now. Josh would have a better idea. We ran into a problem installing GForge. I don't know if Tom had time to work on it over the weekend; if not I'll be tackling it tonight. -- Josh Berkus Aglio Database Solutions San Francisco
Andrew Dunstan wrote: > > I've deliberately let the dust settle slightly on this. > > One thing that might help is a more open sponsorship "clearing house". > Example (not meant as a bid, but just to illustrate): the JDBC driver > needs a scanner overhaul - it breaks on dollar quoting and a bunch of > other stuff. I could do that work (as could others, of course) but I > don't have time, unless someone buys some of my professional time. > Someone might want to do just that, but how would they find me? > > Regarding the secret code stuff - I predict that it will quickly bite > whoever does it, unless they are extremely lucky. > I like this idea. There is another issue too. In general, there is a feeling like one needs to produce something that works and not wait for the slower movement of the community's approval. I don't think several open source forks for the project necessarily produce problems if most of these are used as experimental projects. The example which comes to mind is Samba-TNG. So some of this concern may be overblown. This is also the way things work with the SQL Standard: The various vendors (PostgreSQL included) go out and start with the base, extend that feature set, and eventually come back together to try to build the next standard based on everyone's experience. This embrace and extend process is indeed critical for the further development of the standard. At the same time, I agree with Bruce's main point-- that the lack of communication is a threat to this progress. So at least some note of "Best practices" regarding these extensions or contributions would be a help. Adding a clearing house to this would also add a critical tool and would also have the side effect of increasing the pace of development. Maybe have it divided into two sections: Bids and Bounties. Best Wishes, Chris Travers Metatron Technology Consulting
Mitch Pirtle wrote: > On 4/30/05, Jim C. Nasby <decibel@decibel.org> wrote: > >>If money's not an issue anymore, can we get a bigger box to host >>pgfoundry on then? :) > > > If you guys are planning on running Gforge, then you better make 'box' plural. Well we already run it :) For pgFoundry and you are correct it seems to be a bit of a pig. Our new machines should handle it better though. Sincerely, Joshua D. Drake -- Your PostgreSQL solutions company - Command Prompt, Inc. 1.800.492.2240 PostgreSQL Replication, Consulting, Custom Programming, 24x7 support Managed Services, Shared and Dedication Hosting Co-Authors: plPHP, plPerlNG - http://www.commandprompt.com/
On Tue, 2005-05-03 at 14:26 -0400, Mitch Pirtle wrote: > If you guys are planning on running Gforge, then you better make 'box' plural. > > I'm running MamboForge.net, and the poor thing is getting beat into > the cold hard earth every day. We (Mambo) should really have two > servers, at least to dedicate hardware to the database. Most of the > servers in that class are dual Xeons as well - just as an indicator of > what you are getting yourselves into ;-) Of course, Mitch is running the second largest GForge site on the planet (as far as I know).... second only to https://helixcommunity.org/. Here's a list of them: http://gforge.org/docman/view.php/1/52/gforge-sites.html Yours, Tom Copeland
On Wed, 4 May 2005 04:40 am, Tom Copeland wrote: > On Tue, 2005-05-03 at 14:26 -0400, Mitch Pirtle wrote: > > If you guys are planning on running Gforge, then you better make 'box' plural. > > > > I'm running MamboForge.net, and the poor thing is getting beat into > > the cold hard earth every day. We (Mambo) should really have two > > servers, at least to dedicate hardware to the database. Most of the > > servers in that class are dual Xeons as well - just as an indicator of > > what you are getting yourselves into ;-) > > Of course, Mitch is running the second largest GForge site on the planet > (as far as I know).... second only to https://helixcommunity.org/. > Here's a list of them: > > http://gforge.org/docman/view.php/1/52/gforge-sites.html > Where does all the CPU/disk time go? Do we have any idea what are the strained parts of the system? Is it the database? Regards Russell Smith. > Yours, > > Tom Copeland > > > > ---------------------------(end of broadcast)--------------------------- > TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org > >
On 5/4/05, Russell Smith <mr-russ@pws.com.au> wrote: > On Wed, 4 May 2005 04:40 am, Tom Copeland wrote: > > On Tue, 2005-05-03 at 14:26 -0400, Mitch Pirtle wrote: > > > > Of course, Mitch is running the second largest GForge site on the planet > > (as far as I know).... second only to https://helixcommunity.org/. > > Here's a list of them: > > > > http://gforge.org/docman/view.php/1/52/gforge-sites.html > > > Where does all the CPU/disk time go? Do we have any idea what are the strained parts of the system? > > Is it the database? It is an even mix of apache/mod_php and postgresql. After looking around, I'm not that impressed with the state of the PHP code, and suspect that the data model may also be in great need of some TLC. Thinking about getting more involved in Gforge, to see if some of the glaring issues can be taken care of. -- Mitch