Thread: Feature Freeze date for 8.4
For planning purposes, I think its always a good idea to lay down some dates for the next lot of development milestones. These can be provisional, until declared solid later. So: When is the next Feature Freeze? Is it March 31? If not, when? I know the answer is "too early to tell for certain", but what will it be if we release in early Dec/early Jan, or whenever. If we really are very uncertain, lets at least say it will be "Not Before DateX". PPPPPPP and all that. AFAICS, more than 50% of the patches are written by professional developers, so our various sponsors need to know when the next release is due. Kinda. Ish. Thank you. -- Simon Riggs 2ndQuadrant http://www.2ndQuadrant.com
On Mon, 22 Oct 2007 19:43:28 +0100
Simon Riggs <simon@2ndquadrant.com> wrote:
> For planning purposes, I think its always a good idea to lay down some
> dates for the next lot of development milestones. These can be
> provisional, until declared solid later.
We can't not realistically consider this until we at least come up with
a release date for 8.3.
I seem to recall that we were originally going to release 8.3 in June.
Jsohua D. Drake
>
> So: When is the next Feature Freeze?
>
> Is it March 31? If not, when?
>
> I know the answer is "too early to tell for certain", but what will it
> be if we release in early Dec/early Jan, or whenever.
>
> If we really are very uncertain, lets at least say it will be "Not
> Before DateX". PPPPPPP and all that.
>
> AFAICS, more than 50% of the patches are written by professional
> developers, so our various sponsors need to know when the next release
> is due. Kinda. Ish.
>
> Thank you.
>
--
=== The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 24x7/Emergency: +1.800.492.2240
PostgreSQL solutions since 1997 http://www.commandprompt.com/ UNIQUE NOT NULL
Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate
PostgreSQL Replication: http://www.commandprompt.com/products/
On Mon, 2007-10-22 at 11:53 -0700, Joshua D. Drake wrote: > On Mon, 22 Oct 2007 19:43:28 +0100 > Simon Riggs <simon@2ndquadrant.com> wrote: > > > For planning purposes, I think its always a good idea to lay down some > > dates for the next lot of development milestones. These can be > > provisional, until declared solid later. > > We can't not realistically consider this until we at least come up with > a release date for 8.3. There's always a way of planning through unknowns. We can issue a provisional date. We could also say "at least 6 months after release date of 8.3". I'm sure there's other options too. -- Simon Riggs 2ndQuadrant http://www.2ndQuadrant.com
Simon, > We can issue a provisional date. We could also say "at least 6 months > after release date of 8.3". I'm sure there's other options too. I'm going to suggest 4 months after 8.3. 8.3 was supposed to be a *short* release so that we could move our calendar around. HOT and some of the other unexpected massive patches prevented that. Again, we have enough in the "deferred for 8.4" queue that if we finished up only that it would qualify as a release. So my thought is, shoot for a short release so that we can get away from summer consolidations and December releases, and extend the cycle if someone dumps another 50,000 lines of attractive patches on us. In fact, I could see doing a "no-catalog-changes, no major patches we don't already know about, 6-month release". It would reset our cycle and get PL/proxy, DSM, clustered indexes, etc. out the door. It could mean turning away patches which look attractive, though, so the whole community has to be into this. -- --Josh Josh Berkus PostgreSQL @ Sun San Francisco
Josh Berkus wrote: > So my thought is, shoot for a short release so that we can get away from > summer consolidations and December releases, and extend the cycle if > someone dumps another 50,000 lines of attractive patches on us. > > Before we settle on any dates I think we should have some discussion about how we can shorten the period between feature freeze and beta, which was far too long this time. Perhaps we need to be more aggressive about what what makes the cut and what doesn't. cheers andrew
Josh Berkus wrote: > In fact, I could see doing a "no-catalog-changes, no major patches we don't > already know about, 6-month release". It would reset our cycle and get > PL/proxy, DSM, clustered indexes, etc. out the door. It could mean > turning away patches which look attractive, though, so the whole community > has to be into this. Ah, you mean like we planned for 8.0 and failed, then for 8.1 and failed, then for 8.2 and failed, then for 8.3 and failed? I can definitely support that idea. -- Alvaro Herrera Developer, http://www.PostgreSQL.org/ "We are who we choose to be", sang the goldfinch when the sun is high (Sandman)
On Mon, 22 Oct 2007 16:47:41 -0300
Alvaro Herrera <alvherre@commandprompt.com> wrote:
> Josh Berkus wrote:
>
> > In fact, I could see doing a "no-catalog-changes, no major patches
> > we don't already know about, 6-month release". It would reset our
> > cycle and get PL/proxy, DSM, clustered indexes, etc. out the
> > door. It could mean turning away patches which look attractive,
> > though, so the whole community has to be into this.
>
> Ah, you mean like we planned for 8.0 and failed, then for 8.1 and
> failed, then for 8.2 and failed, then for 8.3 and failed? I can
> definitely support that idea.
>
As I recall 8.0 and 8.1 actually went pretty well.
Joshua D. Drake
--
=== The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 24x7/Emergency: +1.800.492.2240
PostgreSQL solutions since 1997 http://www.commandprompt.com/ UNIQUE NOT NULL
Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate
PostgreSQL Replication: http://www.commandprompt.com/products/
"Joshua D. Drake" <jd@commandprompt.com> writes:
> Alvaro Herrera <alvherre@commandprompt.com> wrote:
>> Ah, you mean like we planned for 8.0 and failed, then for 8.1 and
>> failed, then for 8.2 and failed, then for 8.3 and failed? I can
>> definitely support that idea.
>>
> As I recall 8.0 and 8.1 actually went pretty well.
I don't recall any such plans for those releases, but certainly Josh's
proposal is *EXACTLY* what the plan was for 8.3, and look how well we
adhered to that one.
In point of fact, the big patches that aren't in 8.3 were rejected
because they weren't ready. They won't get into 8.4, either, unless
someone does a lot more work on them. So I don't follow this idea
of how we have a pre-loaded queue of good stuff all ready to go into
8.4. We thought that was true for the 8.3 cycle, which it wasn't,
but there isn't even any basis to think that about 8.4.
regards, tom lane
"Tom Lane" <tgl@sss.pgh.pa.us> writes: > In point of fact, the big patches that aren't in 8.3 were rejected > because they weren't ready. They won't get into 8.4, either, unless > someone does a lot more work on them. So I don't follow this idea > of how we have a pre-loaded queue of good stuff all ready to go into > 8.4. We thought that was true for the 8.3 cycle, which it wasn't, > but there isn't even any basis to think that about 8.4. Incidentally what big features do we have in progress? I see: . GII - there's been discussion about some kind of refactoring the index api to avoid the layer violations here. . Bitmap Indexes - needs a design review and probably changes possibly needs the same api refactoring asGII . DSM - I think Heikki's idea to implement the storage via the buffer manager so it doesn't have fixed size storagelimitations like the FSM is a good one . Recursive Queries - I haven't really started the meat of it but wouldn't mind feedback on the outlineI posted a while back There are some more in the developer.postgresql.org patch status page but I'm not too familiar with what's missing for those. It does seem like most of these are blocked waiting on ideas rather than SMOP issues, so I'm not sure counting on them to be ready on a particular schedule is going to be especially safe. Of course the ideas are more likely to come once we start discussing the issues. I imagine everyone's focused on the beta right now. -- Gregory Stark EnterpriseDB http://www.enterprisedb.com
josh@agliodbs.com (Josh Berkus) writes: > Simon, > >> We can issue a provisional date. We could also say "at least 6 months >> after release date of 8.3". I'm sure there's other options too. > > I'm going to suggest 4 months after 8.3. 8.3 was supposed to be a *short* > release so that we could move our calendar around. HOT and some of the > other unexpected massive patches prevented that. Again, we have enough in > the "deferred for 8.4" queue that if we finished up only that it would > qualify as a release. > > So my thought is, shoot for a short release so that we can get away from > summer consolidations and December releases, and extend the cycle if > someone dumps another 50,000 lines of attractive patches on us. > > In fact, I could see doing a "no-catalog-changes, no major patches we don't > already know about, 6-month release". It would reset our cycle and get > PL/proxy, DSM, clustered indexes, etc. out the door. It could mean > turning away patches which look attractive, though, so the whole community > has to be into this. There are good things about that idea. There would also be good things about picking a somewhat *longer* cycle in that we already just had a cycle where the "feature freeze" period was supposedly a short one, which precluded implementing anything requiring more planning. - It seems at least somewhat unfair to burden the 8.4 cycle with the "sins" of the 8.3 cycle. - There is the risk that even with the restriction, 8.4 might still not be a short cycle, which would make the attempt futile. - And would we then say "hey, we need for 8.5 to have a shortened cycle too"? -- (reverse (concatenate 'string "ofni.secnanifxunil" "@" "enworbbc")) http://linuxfinances.info/info/multiplexor.html Space is big. Really big. You won't believe how vastly mind-bogglingly big it is. I mean, you may think it's a long way down the road to the chemist, but that's just peanuts to space. Listen....
Andrew Dunstan <andrew@dunslane.net> writes:
> Before we settle on any dates I think we should have some discussion
> about how we can shorten the period between feature freeze and beta,
> which was far too long this time. Perhaps we need to be more aggressive
> about what what makes the cut and what doesn't.
I think basically we need to redefine "feature freeze". The definition
we effectively used for the last couple of cycles was "if you've posted
a patch, even a slushy one, by the stated FF date, you make the cut".
This was compounded in the 8.3 cycle by reviewers (and I'm looking at
myself here) figuring that we could postpone reviewing patches until
after FF because the policy didn't require that they be in good shape
*before* that date. That would've worked OK if there were only a few
such patches, but we had a lot of big ones.
If we want a short FF-to-beta period then the criterion will have to be
that patches are either committed or darn near ready to commit on the FF
date. No springing mostly-done patches on the community a few days
before FF. And the reviewers will need to work harder on reviewing
stuff earlier, and committing before FF whenever possible. And we need
to be much more ready to bounce stuff that's not quite done, rather than
drag out the cycle to let it get finished.
No, it doesn't sound like any fun :-(. But this cycle was clearly
mismanaged. It's not productive to have a freeze this long.
[ thinks for a bit... ] A truly hard-nosed approach would be to define
FF as "if your patch isn't committed by the FF date, you lose". The
FF-to-beta delay then is only long enough to make sure we've documented
everything, written release notes, etc. I'm not sure this would be a
more pleasant way to work, as there'd be a heck of a lot of pressure on
the committers as the days tick down to FF. But it'd fix the scheduling
problem.
regards, tom lane
Chris Browne <cbbrowne@acm.org> writes:
> josh@agliodbs.com (Josh Berkus) writes:
>> In fact, I could see doing a "no-catalog-changes, no major patches we don't
>> already know about, 6-month release". It would reset our cycle and get
>> PL/proxy, DSM, clustered indexes, etc. out the door. It could mean
>> turning away patches which look attractive, though, so the whole community
>> has to be into this.
> There are good things about that idea.
> There would also be good things about picking a somewhat *longer*
> cycle in that we already just had a cycle where the "feature freeze"
> period was supposedly a short one, which precluded implementing
> anything requiring more planning.
[ chewing on this a bit... ] The curious thing about that is that
despite this being designed to be a short release cycle, we ended up
landing a bunch of major patches that weren't on the radar screen at
all at the start of the cycle. This suggests to me that there's
something wrong with the concept that no one can get anything major done
without a long release cycle to do it in.
Indeed, the thing that seemed to me to be killing us in this freeze
cycle was that the patches coming in were already at the upper limit of
what we can review and digest. I don't think I want to say "okay guys,
we'll give you a year so that you can write a 200K-line patch and drop
it on us the day before feature freeze".
I'd rather encourage people to work in an incremental, not-so-big-bang
fashion. Obviously one of the requirements for that will be quicker
review turnaround and commit, so that there's time to build on a
previous patch...
regards, tom lane
Tom Lane wrote: > Andrew Dunstan <andrew@dunslane.net> writes: > >> Before we settle on any dates I think we should have some discussion >> about how we can shorten the period between feature freeze and beta, >> which was far too long this time. Perhaps we need to be more aggressive >> about what what makes the cut and what doesn't. >> > > I think basically we need to redefine "feature freeze". The definition > we effectively used for the last couple of cycles was "if you've posted > a patch, even a slushy one, by the stated FF date, you make the cut". > This was compounded in the 8.3 cycle by reviewers (and I'm looking at > myself here) figuring that we could postpone reviewing patches until > after FF because the policy didn't require that they be in good shape > *before* that date. That would've worked OK if there were only a few > such patches, but we had a lot of big ones. > > If we want a short FF-to-beta period then the criterion will have to be > that patches are either committed or darn near ready to commit on the FF > date. No springing mostly-done patches on the community a few days > before FF. And the reviewers will need to work harder on reviewing > stuff earlier, and committing before FF whenever possible. And we need > to be much more ready to bounce stuff that's not quite done, rather than > drag out the cycle to let it get finished. > > No, it doesn't sound like any fun :-(. But this cycle was clearly > mismanaged. It's not productive to have a freeze this long. > > [ thinks for a bit... ] A truly hard-nosed approach would be to define > FF as "if your patch isn't committed by the FF date, you lose". The > FF-to-beta delay then is only long enough to make sure we've documented > everything, written release notes, etc. I'm not sure this would be a > more pleasant way to work, as there'd be a heck of a lot of pressure on > the committers as the days tick down to FF. But it'd fix the scheduling > problem. > I'd settle for your earlier slightly porous formulation, at least for one go round. We don't seem very fond of hard and fast rules as a community (c.f. txid controversy) :-) The target should be to get to beta within about a month from feature freeze, I think. Maybe we need a better triage at the start of feature freeze which gives candidates a preliminary review, at least enough to say "this is/is not very close to being able to be applied". Anything that isn't, even if submitted well before freeze, misses out. If reviewers are more active then the candidate list would be small. cheers andrew
On Mon, 22 Oct 2007, Tom Lane wrote: > If we want a short FF-to-beta period then the criterion will have to be > that patches are either committed or darn near ready to commit on the FF > date. I think you're stuck with a certain amount of schedule delay regardless of how mature code is at submission time when there's a large performance component involved, rather than strictly a feature one. There was a lot of that in 8.3, where it seemed to me the benchmarking and similar quantifying of the true impact of the patch wasn't correlated so much with the code quality at submission time. Good performance testing of any sort takes a long time, there's only so many people who can do it, and having a couple of different perspectives is almost mandatory to avoid optimizing only for a particular application type. When you have a couple of such things in the pool, you're not going to get a lot of work done on multiple patches of that type in parallel, especially when there's any overlap between them. I personally think that shorting the minor release cycle time too far is counterproductive anyway. From the DBA and system administrator perspective, new version releases are a giant QA and maintenance mess. Better to have less of them that each add larger features rather than a more regular stream of small ones from where I'm sitting. -- * Greg Smith gsmith@gregsmith.com http://www.gregsmith.com Baltimore, MD
Tom Lane wrote: > Andrew Dunstan <andrew@dunslane.net> writes: >> Before we settle on any dates I think we should have some discussion >> about how we can shorten the period between feature freeze and beta, >> which was far too long this time. Perhaps we need to be more aggressive >> about what what makes the cut and what doesn't. > [ thinks for a bit... ] A truly hard-nosed approach would be to define > FF as "if your patch isn't committed by the FF date, you lose". The > FF-to-beta delay then is only long enough to make sure we've documented > everything, written release notes, etc. I'm not sure this would be a > more pleasant way to work, as there'd be a heck of a lot of pressure on > the committers as the days tick down to FF. But it'd fix the scheduling > problem. I actually don't disagree with anything you have to say but would like to offer one other thought. Instead of a hard feature freeze, we offer a continual development model. In short, we develop, review, commit etc.. until we feel a need to release. Certainly we don't want to be releasing every 4 years but then again, in reality 8.3 could have been released 6 months ago, should we decided, hey... Tsearch2 and Hot.. well they aren't done so they get 8.4 and perhaps because Tsearch2 and HOT are so cool, once they went in, even though it is only 6 months later, it would be worth while to release 8.4. If we followed this model, we would be pushing 8.4 beta1 right now, not 8.3. Further the people wanting specific features of a specific release, don't have to wait 12-15 months to get them. I recognize this would be a *lot* easier if we didn't have the initdb requirement but still... release early, release often. I have really taken to the Ubuntu style of releasing. Every 6 months (or so) they release. Every 2 years (or so) the LTS. Is there potential for that here? I don't know, but it seems worth exploring. Sincerely, Joshua D. Drake
On Mon, Oct 22, 2007 at 11:56:02PM -0400, Tom Lane wrote: > I'd rather encourage people to work in an incremental, not-so-big-bang > fashion. Obviously one of the requirements for that will be quicker > review turnaround and commit, so that there's time to build on a > previous patch... From my outsider point of view the big difference is that the probablility of $RANDOM_PATCH being reviewed during features freeze is very much greater than at any other time. So the best way to get a patch completed is to post it sometime before FF and after FF use the subsequent feedback to improve it. If we can fix the disparity I think most of the problems will go away... Ofcourse, posting just prior FF doesn't help, but I think that's a chicken/egg problem. People do what works. Have a ncie day, -- Martijn van Oosterhout <kleptog@svana.org> http://svana.org/kleptog/ > From each according to his ability. To each according to his ability to litigate.
> Further the people wanting specific features of a specific release, > don't have to wait 12-15 months to get them. > > I recognize this would be a *lot* easier if we didn't have the initdb > requirement but still... release early, release often. > > I have really taken to the Ubuntu style of releasing. Every 6 months (or > so) they release. Every 2 years (or so) the LTS. > > Is there potential for that here? I don't know, but it seems worth > exploring. > 1x per year full version is well. And it's much better then in MySQL or in Firebird. I thing so problem is in long patch queue. There are lot of sleeping patches (enugh for 8.4 now), and isn't possible transform to core it any short time (mainly with bombs like tsearch or hot). Pavel
> On Mon, 22 Oct 2007, Tom Lane wrote: > > > If we want a short FF-to-beta period then the criterion will have to be > > that patches are either committed or darn near ready to commit on the FF > > date. > > I think you're stuck with a certain amount of schedule delay regardless of > how mature code is at submission time when there's a large performance > component involved, rather than strictly a feature one. There was a lot > of that in 8.3, where it seemed to me the benchmarking and similar > quantifying of the true impact of the patch wasn't correlated so much with > the code quality at submission time. Good performance testing of any sort > takes a long time, there's only so many people who can do it, and having a > couple of different perspectives is almost mandatory to avoid optimizing > only for a particular application type. When you have a couple of such > things in the pool, you're not going to get a lot of work done on multiple > patches of that type in parallel, especially when there's any overlap > between them. > > I personally think that shorting the minor release cycle time too far is > counterproductive anyway. From the DBA and system administrator > perspective, new version releases are a giant QA and maintenance mess. > Better to have less of them that each add larger features rather than a > more regular stream of small ones from where I'm sitting. +1. Shorter release cycles are maybe good for fancy GUI oriented applications, but not so good for DBMS. -- Tatsuo Ishii SRA OSS, Inc. Japan
"Tom Lane" <tgl@sss.pgh.pa.us> writes: > [ thinks for a bit... ] A truly hard-nosed approach would be to define > FF as "if your patch isn't committed by the FF date, you lose". The > FF-to-beta delay then is only long enough to make sure we've documented > everything, written release notes, etc. You're assuming nothing committed could possibly be broken. I think "normal" feature freeze semantics are that you freeze adding new features so you can get the features that are committed working properly. While it's nice that we have such a high standard for the patches that are committed it's also kind of limiting us. For example, I think it worked pretty well when you committed HOT and then based on experience and discussion added the per-page xid. Insisting on every patch being 100% perfected before it gets committed makes it hard to collaborate since it limits the visibility of the work in progress. I do understand the concern that committing lots of broken or partial patches will make it hard for others to work on unrelated work and make for lots of work cleaning up the tree later. Tools like git/monotone/svk might help there. But even with CVS there ought to be a better tradeoff than pushing all the development into patches and requiring the CVS tree to be 100% perfected complete patches. Working with patches over email is painful. -- Gregory Stark EnterpriseDB http://www.enterprisedb.com
"Tom Lane" <tgl@sss.pgh.pa.us> writes: > I'd rather encourage people to work in an incremental, not-so-big-bang > fashion. Obviously one of the requirements for that will be quicker > review turnaround and commit, so that there's time to build on a > previous patch... I'll second that. It's awfully hard to work in isolation building a larger and larger patch for months without being sure that you're even on the right track... Note that quicker turnaround time doesn't have to mean actually doing reviews faster. It just means not waiting until the patch is in release-ready state. It's latency that's hurting us, not bandwidth... -- Gregory Stark EnterpriseDB http://www.enterprisedb.com
Tatsuo Ishii wrote: > > +1. Shorter release cycles are maybe good for fancy GUI oriented > applications, but not so good for DBMS. > -- I agree, sure it will be great to have even more and new features as soon as possible, but not if the quality of the final product decrease. The most important characteristics of postgresql are stability, ,reliability and performance. It just works. We should not forget this. And as others have said, from a DBA and system administrator perspective, maintaining large installations updated in a 6 months release cycle will be a nightmare. We are always 1 year back the main release. We are testing and planing the move to 8.2 now, and it won't happen until desember. In a 6 month cycle we will have to jump over every second release. regards -- Rafael Martinez, <r.m.guerrero@usit.uio.no>Center for Information Technology ServicesUniversity of Oslo, Norway PGP Public Key: http://folk.uio.no/rafael/
On Tue, 2007-10-23 at 11:00 +0200, Rafael Martinez wrote: > We are always 1 year back the main release. We are testing and planing > the move to 8.2 now, and it won't happen until desember. In a 6 month > cycle we will have to jump over every second release. We here are also just in the process of upgrading to 8.2, so in principle I agree... except nobody is forced to go through ALL versions - skipping over every second release is perfectly OK... but: shorter releases would allow people to install only releases which actually bring performance/functionality improvements they really need. All other releases can be skipped... in fact you could possibly go for installing a specific release only on some of the DBs based on what you need there. The main question is if shorter release will impact quality - from a user point of view (and I can only give my opinion in this respect) I don't see any other possible disadvantage... Cheers, Csaba.
On Mon, 2007-10-22 at 23:36 -0400, Tom Lane wrote: > Andrew Dunstan <andrew@dunslane.net> writes: > > Before we settle on any dates I think we should have some discussion > > about how we can shorten the period between feature freeze and beta, > > which was far too long this time. Perhaps we need to be more aggressive > > about what what makes the cut and what doesn't. > > I think basically we need to redefine "feature freeze". Yes, I think so. > But this cycle was clearly > mismanaged. It's not productive to have a freeze this long. That's a good starting point. We need to be objective about what worked and what didn't. I'm very happy about the way we handled performance in this release. We took time to measure things, analyse the problems and then fix them with new code that hadn't even been envisaged before FF. So I'd suggest we have formal phase in the process *after* the code has been fully integrated where we can measure performance and tune anything we need to. > [ thinks for a bit... ] A truly hard-nosed approach would be to define > FF as "if your patch isn't committed by the FF date, you lose". The > FF-to-beta delay then is only long enough to make sure we've documented > everything, written release notes, etc. I'm not sure this would be a > more pleasant way to work, as there'd be a heck of a lot of pressure on > the committers as the days tick down to FF. But it'd fix the scheduling > problem. I'd be happy with that approach. My feeling is there is more than one problem that is being discussed here. Some developers have complained that it sometimes takes a long time to get a patch reviewed and committed. Committers are clearly swamped by the volume of patches and we need to avoid another FF peak. Generally, the project does a good job of committing work as it arrives. Some patches do get missed and so the transit time through the patch queue can be very long in some cases. I'd suggest we have multiple checkpoints during the cycle. Checkpoint is a "patch queue blitz" where we stop developing and reduce the queue to nothing. Perhaps a two-week period where everybody helps reduce the queue, not just Tom and Bruce. Every outstanding patch gets told what they need to do in order to get it committed. FF is then just the last in a series of checkpoints. Suggest we do a checkpoint every 2 months. -- Simon Riggs 2ndQuadrant http://www.2ndQuadrant.com
On Mon, 2007-10-22 at 12:37 -0700, Josh Berkus wrote: > Simon, > > > We can issue a provisional date. We could also say "at least 6 months > > after release date of 8.3". I'm sure there's other options too. > > I'm going to suggest 4 months after 8.3. 8.3 was supposed to be a *short* > release so that we could move our calendar around. HOT and some of the > other unexpected massive patches prevented that. Again, we have enough in > the "deferred for 8.4" queue that if we finished up only that it would > qualify as a release. > > So my thought is, shoot for a short release so that we can get away from > summer consolidations and December releases, and extend the cycle if > someone dumps another 50,000 lines of attractive patches on us. > > In fact, I could see doing a "no-catalog-changes, no major patches we don't > already know about, 6-month release". It would reset our cycle and get > PL/proxy, DSM, clustered indexes, etc. out the door. It could mean > turning away patches which look attractive, though, so the whole community > has to be into this. Those dates shocked me when I first read them, but I feel we need to have an aggressive release schedule, so I'll support that. I hope that the single most important criterion for deciding the release date(s) is How to Get the Most Good New Features into Postgres. That should be more important than other considerations, but I guess in the end, turning patches away may be the only way to ever do a release. -- Simon Riggs 2ndQuadrant http://www.2ndQuadrant.com
Simon Riggs <simon@2ndquadrant.com> writes:
> I'd suggest we have multiple checkpoints during the cycle. Checkpoint is
> a "patch queue blitz" where we stop developing and reduce the queue to
> nothing. Perhaps a two-week period where everybody helps reduce the
> queue, not just Tom and Bruce. Every outstanding patch gets told what
> they need to do in order to get it committed. FF is then just the last
> in a series of checkpoints. Suggest we do a checkpoint every 2 months.
I like this idea ...
regards, tom lane
Tom, all: > > I'd suggest we have multiple checkpoints during the cycle. Checkpoint is > > a "patch queue blitz" where we stop developing and reduce the queue to > > nothing. Perhaps a two-week period where everybody helps reduce the > > queue, not just Tom and Bruce. Every outstanding patch gets told what > > they need to do in order to get it committed. FF is then just the last > > in a series of checkpoints. Suggest we do a checkpoint every 2 months. > > I like this idea ... This would also avoid some bit-rot. However, I'd suggest every 6 weeks; every 2 months would only cycle 2-4 times during a development cycle, and every month would be too frequently. So, say that 8.4 dev officially began on December 1, it would be: Dec 1 - Jan 1: development Jan 1 - Jan 14: queue blitz Jan 15 - Feb 14: development Feb 15 - Feb 28: queue blitz ... June 1: Final patch deadline, 100% stable except for performance/docs, or you wait for 8.5. The other thing we really really need to make this work is *tracking*. For example, I have access to performance and quality testing resources at Sun, but it needs to be crystal clear to my team which patches are ready for testing, where to get them, how to apply them, and what to test. And it can't be dependant on reading 100 e-mail messages a day and filtering for that information the way it currently is. So we should start doing Stefan's patch grid from day 1 of 8.4, and all patch submitters should register with developer.postgresql.org and keep their patch information updated. Do we have any way to upload patch files to the wiki? If not, where can we put them? Or do we want to do a real patch manager? -- Josh Berkus PostgreSQL @ Sun San Francisco
On Tue, 23 Oct 2007 17:28:14 +0900 (JST) Tatsuo Ishii <ishii@postgresql.org> wrote: > > On Mon, 22 Oct 2007, Tom Lane wrote: > > I personally think that shorting the minor release cycle time too > > far is counterproductive anyway. From the DBA and system > > administrator perspective, new version releases are a giant QA and > > maintenance mess. Better to have less of them that each add larger > > features rather than a more regular stream of small ones from where > > I'm sitting. > > +1. Shorter release cycles are maybe good for fancy GUI oriented > applications, but not so good for DBMS. Well my original post was only an example. Consider that it could be 1 month, 8 months, 2 years... The point is there is specific "feature freeze". We develop and commit like normal *until* the community feels there is enough for release. Then we announce a feature freeze. The idea here is that... I could easily argue that 8.3 is enough of a release with *just* HOT and TSearch2. Or that all the other stuff is enough for a release without HOT and Tsearch2. So why did we waste all summer reviewing HOT and TSearch2? We could have released... (That isn't to say that HOT and Tsearch2 are worth the effort, they obviously are). After release of 8.3, we could have picked up immediately on HOT and Tsearch2 and perhaps released 8.4 with other incrementals in about a month. There is nothing wrong with that in the least. People "can" choose not to run a particular version of software, further the community "can" determine which versions of PostgreSQL will be supported for our traditional 3+ years. Joshua D. Drake > -- > Tatsuo Ishii > SRA OSS, Inc. Japan > > ---------------------------(end of > broadcast)--------------------------- TIP 7: You can help support the > PostgreSQL project by donating at > > http://www.postgresql.org/about/donate > -- === The PostgreSQL Company: Command Prompt, Inc. === Sales/Support: +1.503.667.4564 24x7/Emergency: +1.800.492.2240 PostgreSQL solutions since 1997 http://www.commandprompt.com/ UNIQUE NOT NULL Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate PostgreSQL Replication: http://www.commandprompt.com/products/
On Tue, 23 Oct 2007 11:00:59 +0200
Rafael Martinez <r.m.guerrero@usit.uio.no> wrote:
> Tatsuo Ishii wrote:
>
> >
> > +1. Shorter release cycles are maybe good for fancy GUI oriented
> > applications, but not so good for DBMS.
> > --
>
> We are always 1 year back the main release. We are testing and planing
> the move to 8.2 now, and it won't happen until desember. In a 6 month
> cycle we will have to jump over every second release.
Yes you would have to jump over every second release. I fail to see how
this matters? I don't understand the argument.
Sincerely,
Joshua D. Drake
>
> regards
--
=== The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 24x7/Emergency: +1.800.492.2240
PostgreSQL solutions since 1997 http://www.commandprompt.com/ UNIQUE NOT NULL
Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate
PostgreSQL Replication: http://www.commandprompt.com/products/
On Tue, Oct 23, 2007 at 09:39:39AM +0100, Gregory Stark wrote: > > "Tom Lane" <tgl@sss.pgh.pa.us> writes: > > > I'd rather encourage people to work in an incremental, > > not-so-big-bang fashion. Obviously one of the requirements for > > that will be quicker review turnaround and commit, so that there's > > time to build on a previous patch... > > I'll second that. It's awfully hard to work in isolation building a > larger and larger patch for months without being sure that you're > even on the right track... It would be much easier with a distributed SCM system because that way the process of checking patch integration would be branching and merging, these being cheap and simple operations in DSCMs like git, where it's pretty nearly impossibly hard with CVS. You can start using git right away :) http://repo.or.cz/w/PostgreSQL.git > Note that quicker turnaround time doesn't have to mean actually > doing reviews faster. It just means not waiting until the patch is > in release-ready state. It's latency that's hurting us, not > bandwidth... A DSCM can do a *lot* to get this latency down :) Cheers, David (not advocating any particular DSCM at the moment, just the process of using one) -- David Fetter <david@fetter.org> http://fetter.org/ Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter Skype: davidfetter XMPP: david.fetter@gmail.com Remember to vote! Consider donating to Postgres: http://www.postgresql.org/about/donate
On Tue, 23 Oct 2007 09:29:58 -0400 Tom Lane <tgl@sss.pgh.pa.us> wrote: > Simon Riggs <simon@2ndquadrant.com> writes: > > I'd suggest we have multiple checkpoints during the cycle. > > Checkpoint is a "patch queue blitz" where we stop developing and > > reduce the queue to nothing. Perhaps a two-week period where > > everybody helps reduce the queue, not just Tom and Bruce. Every > > outstanding patch gets told what they need to do in order to get it > > committed. FF is then just the last in a series of checkpoints. > > Suggest we do a checkpoint every 2 months. > > I like this idea ... As do I. It will also allow us to decrease the amount of changes that have to be reviewed for regressions during testing. I know I just love it when a customer breaks something and I ask what changed and it is 56 different things ;) My question is.. with a checkpoint every 2 months, would it make it very easy to release every 6 (or 4 or 3 or 9) months? I am not saying we "have" to but it certainly opens up the possibility to the argument I made. With a 2 months checkpoint, we can release when we want. When we feel we have enough and not have a ginormous back log of patches to go through. Sincerely, Joshua D. Drake > > regards, tom lane > > ---------------------------(end of > broadcast)--------------------------- TIP 7: You can help support the > PostgreSQL project by donating at > > http://www.postgresql.org/about/donate > -- === The PostgreSQL Company: Command Prompt, Inc. === Sales/Support: +1.503.667.4564 24x7/Emergency: +1.800.492.2240 PostgreSQL solutions since 1997 http://www.commandprompt.com/ UNIQUE NOT NULL Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate PostgreSQL Replication: http://www.commandprompt.com/products/
On 10/24/07, David Fetter <david@fetter.org> wrote: > On Tue, Oct 23, 2007 at 09:39:39AM +0100, Gregory Stark wrote: > > > > "Tom Lane" <tgl@sss.pgh.pa.us> writes: > > > > > I'd rather encourage people to work in an incremental, > > > not-so-big-bang fashion. Obviously one of the requirements for > > > that will be quicker review turnaround and commit, so that there's > > > time to build on a previous patch... > > > > I'll second that. It's awfully hard to work in isolation building a > > larger and larger patch for months without being sure that you're > > even on the right track... +1. I think use of branches would be a big improvement. > > It would be much easier with a distributed SCM system because that way > the process of checking patch integration would be branching and > merging, these being cheap and simple operations in DSCMs like git, > where it's pretty nearly impossibly hard with CVS. It would be much easier with any modern SCM system at all, distributed or not. I've been using feature branches in Subversion for a couple years now and it's been a pleasure. I'm not trying to derail the discussion into a SCM jihad or anything, just pointing out that any move *away* from CVS, in any direction, is a good move.
Joshua D. Drake wrote: > We develop and commit like normal *until* the community feels there is > enough for release. Then we announce a feature freeze. > > No, I think this is hopeless on several grounds. First, it increases uncertainty. People need to be able to work towards a target. Second, there will always be a temptation to "just squeeze in killer feature X". I think there is some virtue in Simon's suggestion, provided that adequate reviewer resources are available. That might mean you need to put on your fundraising hat and raise some reviewer sponsorship money. cheers andrew
On Tue, 23 Oct 2007 12:45:14 -0400
Andrew Dunstan <andrew@dunslane.net> wrote:
>
>
> Joshua D. Drake wrote:
> > We develop and commit like normal *until* the community feels there
> > is enough for release. Then we announce a feature freeze.
> >
> >
>
> No, I think this is hopeless on several grounds. First, it increases
> uncertainty. People need to be able to work towards a target. Second,
> there will always be a temptation to "just squeeze in killer feature
> X".
Well that flows into that checkpoint idea I think.
>
> I think there is some virtue in Simon's suggestion, provided that
> adequate reviewer resources are available. That might mean you need
> to put on your fundraising hat and raise some reviewer sponsorship
> money.
Did someone say money? ;)
Sincerely,
Joshua D. Drake
>
> cheers
>
> andrew
>
--
=== The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 24x7/Emergency: +1.800.492.2240
PostgreSQL solutions since 1997 http://www.commandprompt.com/ UNIQUE NOT NULL
Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate
PostgreSQL Replication: http://www.commandprompt.com/products/
"Joshua D. Drake" <jd@commandprompt.com> writes: > I know I just love it when a customer breaks something and I ask what > changed and it is 56 different things ;) > > My question is.. with a checkpoint every 2 months, would it make it > very easy to release every 6 (or 4 or 3 or 9) months? I am not saying > we "have" to but it certainly opens up the possibility to the argument > I made. We could release "alpha" releases. But that assumes that these reviews actually result in stuff getting committed even if they're not 100% complete. I think that would be a good thing but I don't think everyone else agrees. Also, not all reviewers are committers. -- Gregory Stark EnterpriseDB http://www.enterprisedb.com
All, > We could release "alpha" releases. But that assumes that these reviews > actually result in stuff getting committed even if they're not 100% > complete. I think that would be a good thing but I don't think everyone > else agrees. Also, not all reviewers are committers. This is what I'm thinking, too. It would be a *lot* easier for the Sun peformance team (and presumably others) to test performance of stuff which was getting committed rather than having to hunt down *this* version of *that* patch and apply it against the snapshot from *this specific date* ... (puts PR hat on) it would also allow me to keep up a stream of info to the community and tech press about the new features and optimizations we're working on, which would mean better coverage for each feature. -- Josh Berkus PostgreSQL @ Sun San Francisco
Gregory Stark wrote: > "Joshua D. Drake" <jd@commandprompt.com> writes: > >> I know I just love it when a customer breaks something and I ask what >> changed and it is 56 different things ;) >> >> My question is.. with a checkpoint every 2 months, would it make it >> very easy to release every 6 (or 4 or 3 or 9) months? I am not saying >> we "have" to but it certainly opens up the possibility to the argument >> I made. > > We could release "alpha" releases. But that assumes that these reviews > actually result in stuff getting committed even if they're not 100% complete. That's sounding a lot like mysql.. Next you'll start recommending people to run that in production.. :-P //Magnus
"Josh Berkus" <josh@agliodbs.com> writes: > All, > >> We could release "alpha" releases. But that assumes that these reviews >> actually result in stuff getting committed even if they're not 100% >> complete. I think that would be a good thing but I don't think everyone >> else agrees. Also, not all reviewers are committers. > > This is what I'm thinking, too. It would be a *lot* easier for the Sun > peformance team (and presumably others) to test performance of stuff which > was getting committed rather than having to hunt down *this* version of > *that* patch and apply it against the snapshot from *this specific date* ... Fwiw I had put together a jumbo patch for three of the patches which you were interested in, HOT, LDC, and GII. This was specifically for the benefit of users like you to get an early experience of these patches. I think it was actually shortly after you had reported a problem running them together which was blocking you from running benchmarks on them. The problem I ran into was that by the time I had them all wrapped up major new commits to the CVS tree made it uninteresting to benchmark the snapshot I had. Also I think a new version of HOT had been posted. -- Gregory Stark EnterpriseDB http://www.enterprisedb.com
Greg, > The problem I ran into was that by the time I had them all wrapped up major > new commits to the CVS tree made it uninteresting to benchmark the snapshot > I had. Also I think a new version of HOT had been posted. Yeah, that's exactly what I was thinking of. The Sun benchmarking folks would have done more testing of HOT, much earlier, if not for those issues. Anyway, is there anyone who thinks the "cycle the queue every 6 weeks or 2 months or suitable short period" is a *bad* idea? It might be hard to pull off, but we won't know until we try. It'll also help keep vacation/summer schedules from being an issue for release timelines, since review will go on year-round and not just in the n.h. summer. And, as a bonus for the reviewers, this means that rather than having to devote yourselves full-time to review for 4 months out of the year, you can space it out in short bursts so that you don't get worn out looking at other people's code instead of working on your own for months! I'm excited about this, and think it's the best suggestion I've heard in years for how to improve our review process. Can we do it? -- Josh Berkus PostgreSQL @ Sun San Francisco
Gregory Stark wrote: > > The problem I ran into was that by the time I had them all wrapped up major > new commits to the CVS tree made it uninteresting to benchmark the snapshot I > had. > That's going to be a problem with any snapshot approach. For the most part, interesting patches are going to hit a number of common hotspots ISTM. I'm fairly resistant to putting less-than-ready code in the tree, I must say. +1 to the idea of patches in smaller pieces that has been mentioned upthread. I tried to do that in one or two places recently, but it could go much further. cheers andrew
Gregory Stark wrote: > "Josh Berkus" <josh@agliodbs.com> writes: > > > All, > > > >> We could release "alpha" releases. But that assumes that these reviews > >> actually result in stuff getting committed even if they're not 100% > >> complete. I think that would be a good thing but I don't think everyone > >> else agrees. Also, not all reviewers are committers. > > > > This is what I'm thinking, too. It would be a *lot* easier for the Sun > > peformance team (and presumably others) to test performance of stuff which > > was getting committed rather than having to hunt down *this* version of > > *that* patch and apply it against the snapshot from *this specific date* ... > > Fwiw I had put together a jumbo patch for three of the patches which you were > interested in, HOT, LDC, and GII. This was specifically for the benefit of > users like you to get an early experience of these patches. I think it was > actually shortly after you had reported a problem running them together which > was blocking you from running benchmarks on them. It would make a lot of sense to keep them in a branch of a distributed SCMS where you can propagate stuff from mainline into the branches. Maybe what we, developers, should do is start using a DSCM privately to keep track of our patches and of mainline. I know some derived projects already use one (Postgres-R?) -- Alvaro Herrera Valdivia, Chile ICBM: S 39º 49' 18.1", W 73º 13' 56.4" "Limítate a mirar... y algun día veras"
tgl@sss.pgh.pa.us (Tom Lane) writes: > Simon Riggs <simon@2ndquadrant.com> writes: >> I'd suggest we have multiple checkpoints during the cycle. Checkpoint is >> a "patch queue blitz" where we stop developing and reduce the queue to >> nothing. Perhaps a two-week period where everybody helps reduce the >> queue, not just Tom and Bruce. Every outstanding patch gets told what >> they need to do in order to get it committed. FF is then just the last >> in a series of checkpoints. Suggest we do a checkpoint every 2 months. > > I like this idea ... Don't we need to have a background writer process to gradually flush this data so that checkpoints aren't so massive? ;-) -- let name="cbbrowne" and tld="acm.org" in String.concat "@" [name;tld];; http://linuxfinances.info/info/oses.html "If there are aliens, they use Lisp." --Edward Lasker, mutatis mutandi
On Tue, Oct 23, 2007 at 04:59:51PM -0400, Chris Browne wrote: > tgl@sss.pgh.pa.us (Tom Lane) writes: > > Simon Riggs <simon@2ndquadrant.com> writes: > >> I'd suggest we have multiple checkpoints during the cycle. > >> Checkpoint is a "patch queue blitz" where we stop developing and > >> reduce the queue to nothing. Perhaps a two-week period where > >> everybody helps reduce the queue, not just Tom and Bruce. Every > >> outstanding patch gets told what they need to do in order to get > >> it committed. FF is then just the last in a series of > >> checkpoints. Suggest we do a checkpoint every 2 months. > > > > I like this idea ... > > Don't we need to have a background writer process to gradually flush > this data so that checkpoints aren't so massive? ;-) I'm proposing a DSCM with easy branching and merging to implement this background writer process :) The one below is already available, so we don't have to do a "flag day" with it. http://repo.or.cz/w/PostgreSQL.git There are git-cvsimport and git-cvsexport utilities which make communication between the legacy SCM and the DSCM straight-forward. Cheers, David. -- David Fetter <david@fetter.org> http://fetter.org/ Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter Skype: davidfetter XMPP: david.fetter@gmail.com Remember to vote! Consider donating to Postgres: http://www.postgresql.org/about/donate
Andrew Dunstan <andrew@dunslane.net> writes:
> I'm fairly resistant to putting less-than-ready code in the tree, I must
> say.
Me too, at least if "less than ready" means "unstable". The committed
code has to always be solid enough to let everybody continue working on
their own bits. However, in the past we've tended to refuse stuff if
it "didn't do anything interesting", and I think that attitude will need
adjustment. We'll have to be prepared to accept patches that are only
interesting as waystations to some long-term goal.
regards, tom lane
Josh Berkus <josh@agliodbs.com> writes:
> Anyway, is there anyone who thinks the "cycle the queue every 6 weeks or 2
> months or suitable short period" is a *bad* idea? It might be hard to pull
> off, but we won't know until we try.
It seems worth a try --- we can certainly abandon it easily if it
doesn't work.
Personally I feel every six weeks would be too short: we'd be talking
only a month of work between commit-fests. I like a two-month cycle
partly because it wouldn't rotate relative to the calendar: we'd always
know that the first half of every odd-numbered month, or something like
that, is commit-fest time.
regards, tom lane
Tom Lane wrote: > Andrew Dunstan <andrew@dunslane.net> writes: > >> I'm fairly resistant to putting less-than-ready code in the tree, I must >> say. >> > > Me too, at least if "less than ready" means "unstable". The committed > code has to always be solid enough to let everybody continue working on > their own bits. However, in the past we've tended to refuse stuff if > it "didn't do anything interesting", and I think that attitude will need > adjustment. We'll have to be prepared to accept patches that are only > interesting as waystations to some long-term goal. > > > Yes, this is a sensible adjustment, even across releases, IMNSHO. cheers andrew
Tom Lane wrote: > Josh Berkus <josh@agliodbs.com> writes: > >> Anyway, is there anyone who thinks the "cycle the queue every 6 weeks or 2 >> months or suitable short period" is a *bad* idea? It might be hard to pull >> off, but we won't know until we try. >> > > It seems worth a try --- we can certainly abandon it easily if it > doesn't work. > > Personally I feel every six weeks would be too short: we'd be talking > only a month of work between commit-fests. I like a two-month cycle > partly because it wouldn't rotate relative to the calendar: we'd always > know that the first half of every odd-numbered month, or something like > that, is commit-fest time. > > > +1 on both counts. cheers andrew
David Fetter wrote: > On Tue, Oct 23, 2007 at 04:59:51PM -0400, Chris Browne wrote: > >> tgl@sss.pgh.pa.us (Tom Lane) writes: >> >>> Simon Riggs <simon@2ndquadrant.com> writes: >>> >>>> I'd suggest we have multiple checkpoints during the cycle. >>>> Checkpoint is a "patch queue blitz" where we stop developing and >>>> reduce the queue to nothing. Perhaps a two-week period where >>>> everybody helps reduce the queue, not just Tom and Bruce. Every >>>> outstanding patch gets told what they need to do in order to get >>>> it committed. FF is then just the last in a series of >>>> checkpoints. Suggest we do a checkpoint every 2 months. >>>> >>> I like this idea ... >>> >> Don't we need to have a background writer process to gradually flush >> this data so that checkpoints aren't so massive? ;-) >> > > I'm proposing a DSCM with easy branching and merging to implement this > background writer process :) > > The one below is already available, so we don't have to do a "flag > day" with it. > > http://repo.or.cz/w/PostgreSQL.git > > There are git-cvsimport and git-cvsexport utilities which make > communication between the legacy SCM and the DSCM straight-forward. > > > You are way ahead of us here. And my vote *still* goes to Mercurial, if we're picking SCMs. cheers andrew
Tom, > Personally I feel every six weeks would be too short: we'd be talking > only a month of work between commit-fests. I like a two-month cycle > partly because it wouldn't rotate relative to the calendar: we'd always > know that the first half of every odd-numbered month, or something like > that, is commit-fest time. Well, you'd know better than me, certainly. -- --Josh Josh Berkus PostgreSQL @ Sun San Francisco
On Tue, Oct 23, 2007 at 06:19:42PM -0400, Andrew Dunstan wrote: > David Fetter wrote: > >On Tue, Oct 23, 2007 at 04:59:51PM -0400, Chris Browne wrote: > >>tgl@sss.pgh.pa.us (Tom Lane) writes: > >>>Simon Riggs <simon@2ndquadrant.com> writes: > >>> > >>>>I'd suggest we have multiple checkpoints during the cycle. > >>>>Checkpoint is a "patch queue blitz" where we stop developing and > >>>>reduce the queue to nothing. Perhaps a two-week period where > >>>>everybody helps reduce the queue, not just Tom and Bruce. Every > >>>>outstanding patch gets told what they need to do in order to get > >>>>it committed. FF is then just the last in a series of > >>>>checkpoints. Suggest we do a checkpoint every 2 months. > >>>> > >>>I like this idea ... > >>> > >>Don't we need to have a background writer process to gradually > >>flush this data so that checkpoints aren't so massive? ;-) > >> > > > >I'm proposing a DSCM with easy branching and merging to implement > >this background writer process :) > > > >The one below is already available, so we don't have to do a "flag > >day" with it. > > > >http://repo.or.cz/w/PostgreSQL.git > > > >There are git-cvsimport and git-cvsexport utilities which make > >communication between the legacy SCM and the DSCM straight-forward. > > You are way ahead of us here. And my vote *still* goes to Mercurial, > if we're picking SCMs. I'm not picking a DSCM. I'm saying we already have tools in place for a DSCM *without* having a "flag day." If Mercurial has a similar migration/legacy support path, then by all means, let's try that out, too. :) Cheers, David. -- David Fetter <david@fetter.org> http://fetter.org/ Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter Skype: davidfetter XMPP: david.fetter@gmail.com Remember to vote! Consider donating to Postgres: http://www.postgresql.org/about/donate
Folks, > You are way ahead of us here. And my vote *still* goes to Mercurial, if > we're picking SCMs. Will a new SCM actually make this easier, or are people just using it as an excuse? Mind you, I'm in favor of one. A new SCM would make some other development tasks easier. However, I'm reluctant to open the can-of-worms which is the "what SCM should we use" discussion again, and complicate something which we seem to have consensus on. -- --Josh Josh Berkus PostgreSQL @ Sun San Francisco
On Tue, 2007-10-23 at 09:28 -0700, Joshua D. Drake wrote: > On Tue, 23 Oct 2007 09:29:58 -0400 > Tom Lane <tgl@sss.pgh.pa.us> wrote: > > > Simon Riggs <simon@2ndquadrant.com> writes: > > > I'd suggest we have multiple checkpoints during the cycle. > > > Checkpoint is a "patch queue blitz" where we stop developing and > > > reduce the queue to nothing. Perhaps a two-week period where > > > everybody helps reduce the queue, not just Tom and Bruce. Every > > > outstanding patch gets told what they need to do in order to get it > > > committed. FF is then just the last in a series of checkpoints. > > > Suggest we do a checkpoint every 2 months. > > > > I like this idea ... > > As do I. It will also allow us to decrease the amount of changes that > have to be reviewed for regressions during testing. Cool. Nobody seems to have disagreed... > I know I just love it when a customer breaks something and I ask what > changed and it is 56 different things ;) > > My question is.. with a checkpoint every 2 months, would it make it > very easy to release every 6 (or 4 or 3 or 9) months? I am not saying > we "have" to but it certainly opens up the possibility to the argument > I made. > > With a 2 months checkpoint, we can release when we want. When we feel > we have enough and not have a ginormous back log of patches to go > through. Maybe. I'm looking for ways to increase the amount of development time we have compared with time releasing. If we release twice as often, we won't get twice the beta test contribution from everybody, so our code will be less robust, which will hurt us in the long run. -- Simon Riggs 2ndQuadrant http://www.2ndQuadrant.com
Joshua D. Drake wrote: > > We develop and commit like normal *until* the community feels there is > enough for release. Then we announce a feature freeze. I think you just described what will happen in reality regardless of whatever is decided to be an official "plan". :) I don't think that's necessarily a bad thing. IMHO 8.3 is looking like a better product because it's "late", and DBAs concerned about stability will likely stay on 8.2 for a while anyway.
Simon, > Maybe. I'm looking for ways to increase the amount of development time > we have compared with time releasing. If we release twice as often, we > won't get twice the beta test contribution from everybody, so our code > will be less robust, which will hurt us in the long run. I don't think we actually want to release more than once a year, except maybe moving the annual release date a couple months one way or the other (December is not a great time to be releasing SW). 1 year is a good compromise between the developers (who would like a release every 6 months) and the corporate users/packagers (who would like a release every 3 years). Plus, for the developers and other people who really need to be bleeding-edge, this new plan would result in less-unstable snapshots every 2 months with defined feature sets which someone who wanted to run them at their own risk could. Which would result in more bug reports, earlier, for us (and lots of forwarding the canned "milestone-releases-are-not-stable" canned e-mail). -- --Josh Josh Berkus PostgreSQL @ Sun San Francisco
Josh Berkus <josh@agliodbs.com> writes:
> Mind you, I'm in favor of one. A new SCM would make some other development
> tasks easier. However, I'm reluctant to open the can-of-worms which is the
> "what SCM should we use" discussion again, and complicate something which
> we seem to have consensus on.
As near as I can tell, the arguments for a new SCM mostly apply to work
which individual developers are doing outside the main tree. So, given
the existence of stuff like git-cvsimport, I don't see a strong reason
why anyone who wants to work that way can't already sync the core CVS
with a local SCM-of-their-choice and get on with it.
You're right that this is utterly unrelated to the scheduling question,
anyway.
regards, tom lane
Josh Berkus <josh@agliodbs.com> writes:
> Plus, for the developers and other people who really need to be
> bleeding-edge, this new plan would result in less-unstable snapshots every
> 2 months with defined feature sets which someone who wanted to run them at
> their own risk could. Which would result in more bug reports, earlier,
> for us (and lots of forwarding the canned
> "milestone-releases-are-not-stable" canned e-mail).
Hmm, I was not envisioning that we'd produce any sort of "release"
corresponding to these checkpoints. I see it only as a a way to
(a) discipline ourselves to not let patches go unreviewed/uncommitted
for long periods, and (b) encourage developers to submit relatively
small patches rather than enormous six-months-of-work ones.
Since there are always bugs, and we're certainly not going to schedule a
round of formal beta testing right after each commit-fest, I should
think that tarballs made right after a commit-fest would be particularly
unlikely to be good candidates for non-developer use.
(Actually, it might be the case that a CVS snap from just *before*
a commit-fest would be the most stable development-cycle code, since
there'd have been time to shake out bugs committed in the previous
fest... but we're even less likely to do beta testing on that.)
regards, tom lane
Tom, > Since there are always bugs, and we're certainly not going to schedule a > round of formal beta testing right after each commit-fest, I should > think that tarballs made right after a commit-fest would be particularly > unlikely to be good candidates for non-developer use. > > (Actually, it might be the case that a CVS snap from just *before* > a commit-fest would be the most stable development-cycle code, since > there'd have been time to shake out bugs committed in the previous > fest... but we're even less likely to do beta testing on that.) Hmmm. Well, what I'm looking for is a milestone build we can grab for testing purposes. Right now we grab the daily snapshots, but there's no archive of those, so we're often fighting the fact that a patch developer did his patch on a different snapshot than you have and there's no going back, so you have to share not just the patch but the entire PG tree it's against. If we had milestone snapshots of each 2 months ... probably just before each commit-fest ... and kept them available on ftp.postgresql.org until official beta, then it would make it easier for testers to have a common point of reference to work with. -- --Josh Josh Berkus PostgreSQL @ Sun San Francisco
Josh Berkus <josh@agliodbs.com> writes:
> If we had milestone snapshots of each 2 months ... probably just before
> each commit-fest ... and kept them available on ftp.postgresql.org until
> official beta, then it would make it easier for testers to have a common
> point of reference to work with.
This seems pretty entirely orthogonal to the commit-fest proposal.
I see no reason to think that snapshots taken at those times would
be any better than any other nightly snapshot, nor any reason
to memorialize them in an archive.
Now to the extent that regular commit-fests keep patch development
more closely aligned with the mainline CVS, the fest proposal might
indirectly alleviate your pain. But I'd think that snaps taken
*after* the fests would be the best for that, as they'd be closer
to what any subsequent development would be based on. The stability
argument I made doesn't seem like a problem for testing purposes,
only if someone were to try to use such a snap for production purposes
(which putting them up for archiving might encourage).
regards, tom lane
Tom Lane wrote: > Now to the extent that regular commit-fests keep patch development > more closely aligned with the mainline CVS, the fest proposal might > indirectly alleviate your pain. But I'd think that snaps taken > *after* the fests would be the best for that, as they'd be closer > to what any subsequent development would be based on. The stability > argument I made doesn't seem like a problem for testing purposes, > only if someone were to try to use such a snap for production purposes > (which putting them up for archiving might encourage). > > > It might be worth applying a simple tag (but not a branch) at the end (and maybe also at the start) of each checkpoint/fest/whatever cheers andrew
Andrew Dunstan <andrew@dunslane.net> writes:
> It might be worth applying a simple tag (but not a branch) at the end
> (and maybe also at the start) of each checkpoint/fest/whatever
Perhaps, though of course one could easily enough pull a CVS snapshot by
date instead (especially if we stick to a pretty predictable schedule
for the fests).
Thinking about that a bit more, it seems like a rigid "two weeks" plan
is pointless, since the amount of work to be done will vary depending
on what's in the queue. It seems what we ought to do is something like
this:
* Commit-fest starts on the first of each alternate month. It ends
whenever all the patches that were in the queue on the first are dealt
with; either committed, rejected permanently, or sent back for specific
rework. It might take a week, or two, or three, but in any case we all
try to focus on patch review rather than new work until it's done.
If we do it that way, then an end-of-fest tag might be worthwhile,
so you'd not have to dig through the mailing list archives to figure out
when a fest ended.
regards, tom lane
On Tue, 23 Oct 2007, David Fetter wrote: > If Mercurial has a similar migration/legacy support path, then by all > means, let's try that out, too. There's an import tool at http://hg.beekhof.net/hg/cvs-import but the experience of the Mozilla project suggests it's on the buggy and slow side for large repositories with lots of history: http://weblogs.mozillazine.org/preed/2007/04/version_control_system_shootou_1.html The main problem with using Git on a much larger scale is that there's still limited ability to use it on Win32. Google is working on that: http://code.google.com/p/msysgit/ but it's not quite there yet; there's also a partial MinGW port. The way I suspect any plan for incorporating a new [D]VCS on a larger scale in this project will need to progress is like this: 1) Make a converted copy of the existing CVS repository 2) Keep the mirrored repo up to date with new commits 3) Provide working guidelines so that developers can use the new VCS to build local patches and improve their productivity 4) Get enough developers using the new system that it becomes a popular way to increase visibility on work in progress patches 5) Reach a critical mass of developers showing improved productivity on the new system to sway the core toward that particular VCS 6) Convert the main repository to the new VCS Git has reached (2) already. I started documenting a process for using Subversion, it would only take a little more work on http://developer.postgresql.org/index.php/Working_with_CVS and that would hit (3). I personally don't think there's enough potential for gain converting to SVN to make it worth the trouble, which is why I haven't bothered doing more there. Mercurial/Monotone/Bazaar are all interesting possibilities as well, but it will take an advocate willing to start down this trail for a particular tool to kick off a serious investigation of any of them. -- * Greg Smith gsmith@gregsmith.com http://www.gregsmith.com Baltimore, MD
On 10/24/07, Greg Smith <gsmith@gregsmith.com> wrote: > 1) Make a converted copy of the existing CVS repository > 2) Keep the mirrored repo up to date with new commits > 3) Provide working guidelines so that developers can use the new VCS to > build local patches and improve their productivity > 4) Get enough developers using the new system that it becomes a popular > way to increase visibility on work in progress patches > 5) Reach a critical mass of developers showing improved productivity on > the new system to sway the core toward that particular VCS > 6) Convert the main repository to the new VCS > > Git has reached (2) already. I started documenting a process for using > Subversion, it would only take a little more work on > http://developer.postgresql.org/index.php/Working_with_CVS and that would > hit (3). I personally don't think there's enough potential for gain > converting to SVN to make it worth the trouble, which is why I haven't > bothered doing more there. How up to date is the Git repos? Does it pull individual commits from CVS, or does it resync the whole history periodically? If so, what's the lag? An important part of (2) is that the mirrored repos be sufficiently up to date that using it for your dev work and producing patches doesn't put you significantly behind HEAD. I think the Subversion repos resyncs every six hours, but I'm not sure (strangely there's not a whole lot of information about it on the wiki). To me, six hours seems a little slow. If you hit (3) on Git I for one will gladly start using the Git repository for my dev work. I work on minor upgrades, not the major stuff, so I think the benefits of a distributed system might be slim for me (because I'm not going to be switching across large branches). But I'd be happy to give it a shot.
On Tue, Oct 23, 2007 at 07:24:21PM -0400, Tom Lane wrote: > Josh Berkus <josh@agliodbs.com> writes: > > Mind you, I'm in favor of one. A new SCM would make some other > > development tasks easier. However, I'm reluctant to open the > > can-of-worms which is the "what SCM should we use" discussion > > again, and complicate something which we seem to have consensus > > on. We don't need to, as the tool is already in place. > As near as I can tell, the arguments for a new SCM mostly apply to > work which individual developers are doing outside the main tree. > So, given the existence of stuff like git-cvsimport, I don't see a > strong reason why anyone who wants to work that way can't already > sync the core CVS with a local SCM-of-their-choice and get on with > it. > > You're right that this is utterly unrelated to the scheduling > question, anyway. It's not even slightly unrelated. It's a way of preventing bit-rot on large patches and keeping them in sync :) Cheers, David. -- David Fetter <david@fetter.org> http://fetter.org/ Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter Skype: davidfetter XMPP: david.fetter@gmail.com Remember to vote! Consider donating to Postgres: http://www.postgresql.org/about/donate
On Tue, 2007-10-23 at 16:19 -0700, Josh Berkus wrote:
> > Maybe. I'm looking for ways to increase the amount of development time
> > we have compared with time releasing. If we release twice as often, we
> > won't get twice the beta test contribution from everybody, so our code
> > will be less robust, which will hurt us in the long run.
>
> I don't think we actually want to release more than once a year
I don't want to either!
Not sure how you've taken my words backwards, but apologies for not
being clear enough. I was trying to say that its possible ("maybe"),
*but* that there were arguments against it, so not desirable since it
will hurt us in the long run. I agree with all of your additional
arguments as to why not also.
-- Simon Riggs 2ndQuadrant http://www.2ndQuadrant.com
On Tue, 2007-10-23 at 19:42 -0400, Tom Lane wrote: > Josh Berkus <josh@agliodbs.com> writes: > > Plus, for the developers and other people who really need to be > > bleeding-edge, this new plan would result in less-unstable snapshots every > > 2 months with defined feature sets which someone who wanted to run them at > > their own risk could. Which would result in more bug reports, earlier, > > for us (and lots of forwarding the canned > > "milestone-releases-are-not-stable" canned e-mail). > > Hmm, I was not envisioning that we'd produce any sort of "release" > corresponding to these checkpoints. I see it only as a a way to > (a) discipline ourselves to not let patches go unreviewed/uncommitted > for long periods, and (b) encourage developers to submit relatively > small patches rather than enormous six-months-of-work ones. > > Since there are always bugs, and we're certainly not going to schedule a > round of formal beta testing right after each commit-fest, I should > think that tarballs made right after a commit-fest would be particularly > unlikely to be good candidates for non-developer use. > > (Actually, it might be the case that a CVS snap from just *before* > a commit-fest would be the most stable development-cycle code, since > there'd have been time to shake out bugs committed in the previous > fest... but we're even less likely to do beta testing on that.) +1 -- Simon Riggs 2ndQuadrant http://www.2ndQuadrant.com
> The main problem with using Git on a much larger scale is that there's > still limited ability to use it on Win32. Google is working on that: > http://code.google.com/p/msysgit/ but it's not quite there yet; there's > also a partial MinGW port. IIRC the official word from the git people is that they don't care about windows. As long as they say that, I consider thata dealbreaker. And from a quick look that google project is still just for msys, which is a dependency that we've rid ourselves of. If we're going to switch scm, we should switch to one that's supported on our platforms... /Magnus
On 10/24/07, Tom Lane <tgl@sss.pgh.pa.us> wrote: > Josh Berkus <josh@agliodbs.com> writes: > > Mind you, I'm in favor of one. A new SCM would make some other development > > tasks easier. However, I'm reluctant to open the can-of-worms which is the > > "what SCM should we use" discussion again, and complicate something which > > we seem to have consensus on. > > As near as I can tell, the arguments for a new SCM mostly apply to work > which individual developers are doing outside the main tree. So, given > the existence of stuff like git-cvsimport, I don't see a strong reason > why anyone who wants to work that way can't already sync the core CVS > with a local SCM-of-their-choice and get on with it. Yes, external devs can already work fine with DSCM. And after using the repo.or.cz git tree for some time, mostly tracking and also for some small patches I really have no interest touching CVS anymore. > You're right that this is utterly unrelated to the scheduling question, > anyway. As we seem discussing developement in general, there is one obstacle in the way of individual use of DSCMs - context diff format as only one accepted. Both leading DSCMs - GIT and Mercurial do not support it. Currently converting patch from DSCM to context diff is pain, especially if it contains new files. It would make lives of DSCM users much easier if both formats would be accepted in -hackers. -- marko
On 10/24/07, Magnus Hagander <magnus@hagander.net> wrote: > > The main problem with using Git on a much larger scale is that there's > > still limited ability to use it on Win32. Google is working on that: > > http://code.google.com/p/msysgit/ but it's not quite there yet; there's > > also a partial MinGW port. > > IIRC the official word from the git people is that they don't care about windows. As long as they say that, I considerthat a dealbreaker. > > And from a quick look that google project is still just for msys, which is a dependency that we've rid ourselves of. > > If we're going to switch scm, we should switch to one that's supported on our platforms... Well, that _was_ the official word from core git devs, in the sense that they wonr put any effort to windows support and any support should not obfuscate core code. But there were other devs who decided to work on it and the windows support is humming along well: http://marc.info/?l=git&m=119283639121273&w=2 Note that msys is needed for _building_ git not using it. -- marko
On Wed, Oct 24, 2007 at 11:04:34AM +0300, Marko Kreen wrote: > On 10/24/07, Magnus Hagander <magnus@hagander.net> wrote: > > > The main problem with using Git on a much larger scale is that there's > > > still limited ability to use it on Win32. Google is working on that: > > > http://code.google.com/p/msysgit/ but it's not quite there yet; there's > > > also a partial MinGW port. > > > > IIRC the official word from the git people is that they don't care about windows. As long as they say that, I considerthat a dealbreaker. > > > > And from a quick look that google project is still just for msys, which is a dependency that we've rid ourselves of. > > > > If we're going to switch scm, we should switch to one that's supported on our platforms... > > Well, that _was_ the official word from core git devs, in the sense > that they wonr put any effort to windows support and any support > should not obfuscate core code. But there were other devs who > decided to work on it and the windows support is humming along well: > > http://marc.info/?l=git&m=119283639121273&w=2 > > Note that msys is needed for _building_ git not using it. Ah. Like I said, only gave it a quick look :-) As long as it works without msys, I'm fine with that. Once they reach non-alpha state, of course. (I'm not going to comment on the relative merits of the different scms - as long as it works, and it not unsupported in a way that it can suddenly break and not be quickly fixed...) //Magnus
On 10/24/07, Magnus Hagander <magnus@hagander.net> wrote: > On Wed, Oct 24, 2007 at 11:04:34AM +0300, Marko Kreen wrote: > > On 10/24/07, Magnus Hagander <magnus@hagander.net> wrote: > > > > The main problem with using Git on a much larger scale is that there's > > > > still limited ability to use it on Win32. Google is working on that: > > > > http://code.google.com/p/msysgit/ but it's not quite there yet; there's > > > > also a partial MinGW port. > > > > > > IIRC the official word from the git people is that they don't care about windows. As long as they say that, I considerthat a dealbreaker. > > > > > > And from a quick look that google project is still just for msys, which is a dependency that we've rid ourselves of. > > > > > > If we're going to switch scm, we should switch to one that's supported on our platforms... > > > > Well, that _was_ the official word from core git devs, in the sense > > that they wonr put any effort to windows support and any support > > should not obfuscate core code. But there were other devs who > > decided to work on it and the windows support is humming along well: > > > > http://marc.info/?l=git&m=119283639121273&w=2 > > > > Note that msys is needed for _building_ git not using it. > > Ah. Like I said, only gave it a quick look :-) As long as it works without > msys, I'm fine with that. Once they reach non-alpha state, of course. (I'm > not going to comment on the relative merits of the different scms - as long > as it works, and it not unsupported in a way that it can suddenly break and > not be quickly fixed...) I agree with Tom that there is no hurry to switch main SCM to something else. At least git users are served via repo.or.cz very well, I hope there is something like that for Mercurial too. Only problem is that there is a certain historical obstacle for DSCM users who want to submit their patches... -- marko
David Fetter wrote: > I'm not picking a DSCM. I'm saying we already have tools in place for > a DSCM *without* having a "flag day." If Mercurial has a similar > migration/legacy support path, then by all means, let's try that out, > too. :) There's at least on Mercurial repo, here: http://www.ubiobio.cl/~gpoo/pgsql/pgsql/ I'm not sure how often it syncs. For those of you outside Chile my bet is that it will be a bit slow ... -- Alvaro Herrera http://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc.
Marko Kreen escribió: > As we seem discussing developement in general, there is one > obstacle in the way of individual use of DSCMs - context diff > format as only one accepted. Both leading DSCMs - GIT and Mercurial > do not support it. Hmm, in Subversion you can specify a separate diff command and args, which can be used to generate context diffs. Is it not possible with git/Hg? -- Alvaro Herrera http://www.amazon.com/gp/registry/DXLWNGRJD34J "La realidad se compone de muchos sueños, todos ellos diferentes, pero en cierto aspecto, parecidos..." (Yo, hablando de sueños eróticos)
> As we seem discussing developement in general, there is one > obstacle in the way of individual use of DSCMs - context diff > format as only one accepted. Both leading DSCMs - GIT and Mercurial > do not support it. > Really? I just started playing around with git, and the output from git diff produced the same kind of diff file I would normally get from `svn di` or `cvs di -c`. Maybe I'm misunderstanding what you meant by "context diff", but this stuff git is producing looks like a context diff to me. Regards, BJ
Brendan Jurd escribió: > > As we seem discussing developement in general, there is one > > obstacle in the way of individual use of DSCMs - context diff > > format as only one accepted. Both leading DSCMs - GIT and Mercurial > > do not support it. > > > > Really? I just started playing around with git, and the output from > git diff produced the same kind of diff file I would normally get from > `svn di` ... which is a unified diff. > or `cvs di -c`. Huh, strange. That's a context diff. -- Alvaro Herrera http://www.CommandPrompt.com/ PostgreSQL Replication, Consulting, Custom Development, 24x7 support
Alvaro Herrera write: > Marko Kreen escribió: > >> As we seem discussing developement in general, there is one >> obstacle in the way of individual use of DSCMs - context diff >> format as only one accepted. Both leading DSCMs - GIT and Mercurial >> do not support it. > > Hmm, in Subversion you can specify a separate diff command and args, > which can be used to generate context diffs. Is it not possible with > git/Hg? It's possible in Mercurial, too. There is a bundled extension (extdiff, http://www.selenic.com/mercurial/wiki/index.cgi/ExtdiffExtension), which can do that. You can configure your own command aliases, e.g. "hg cdiff ..." to do -c diffs. Best Regards Michael Paesold
On 10/24/07, Alvaro Herrera <alvherre@commandprompt.com> wrote: > Brendan Jurd escribió: > > Really? I just started playing around with git, and the output from > > git diff produced the same kind of diff file I would normally get from > > `svn di` > > ... which is a unified diff. > > > or `cvs di -c`. > > Huh, strange. That's a context diff. Right. I was confusing unified and context, because the unified diffs I'm looking at do actually show "context" lines above and below each change. Sorry for the noise.
* Brendan Jurd <direvus@gmail.com> [071024 01:41]: > How up to date is the Git repos? Does it pull individual commits from > CVS, or does it resync the whole history periodically? If so, what's > the lag? It's updated hourly, which is the same rate the public CVS is updated. > An important part of (2) is that the mirrored repos be sufficiently up > to date that using it for your dev work and producing patches doesn't > put you significantly behind HEAD. > > I think the Subversion repos resyncs every six hours, but I'm not sure > (strangely there's not a whole lot of information about it on the > wiki). To me, six hours seems a little slow. > > If you hit (3) on Git I for one will gladly start using the Git > repository for my dev work. I work on minor upgrades, not the major > stuff, so I think the benefits of a distributed system might be slim > for me (because I'm not going to be switching across large branches). > But I'd be happy to give it a shot. Please - feel free to use the Git repository for your work. It's there, and staying there, based on the PostgreSQL CVS repository. In other projects I'm involved in, some developers are using GIT pretty heavily *as developers*. The projects "main repository" is still in CVS, but that isn't much of a problem, because the tools git offers are excellent at producing patches, patch series, rebasing them, sharing them, and even directly committing them to CVS. The advantages of GIT aren't necessarily that it's a *distributed* SCM/VCS, but that since it is a DSCM/DVCS, it's understanding of history and the tools that enable are extremely powerful. But CVS is (unfortunately) the defacto standard. Everybody understands it (well, the basics of it), and all the tools know how to interact with it. So leave the main repository in CVS, and let each developer use the tools they are most comfortable with. The fact is, no matter *what* the "official" repository is, I would *still* use git to develop. And I'm guessing that others who prefer the tools in darcs/monotone/hg will continue to use their favourite tools too. But the more arcane, unique, or just plain "different" the main repository is, the higher *perceived* barrier to entry will be. a. -- Aidan Van Dyk Create like a god, aidan@highrise.ca command like a king, http://www.highrise.ca/ work like a slave.
On Tue, Oct 23, 2007 at 02:39:43PM -0700, David Fetter wrote: > The one below is already available, so we don't have to do a "flag > day" with it. > > http://repo.or.cz/w/PostgreSQL.git As someone who hasn't used GIT: if I have a modified CVS tree from some time back (>1 year) can I use this to manage the bitrot? Just doing a CVS update will probably just mark everything conflicted, so I'm wondering is GIT can do it better. Have a nice day, -- Martijn van Oosterhout <kleptog@svana.org> http://svana.org/kleptog/ > From each according to his ability. To each according to his ability to litigate.
Aidan Van Dyk <aidan@highrise.ca> writes:
> * Brendan Jurd <direvus@gmail.com> [071024 01:41]:
>> How up to date is the Git repos? Does it pull individual commits from
>> CVS, or does it resync the whole history periodically? If so, what's
>> the lag?
> It's updated hourly, which is the same rate the public CVS is updated.
FWIW, that's no longer the case: there's some sort of trigger mechanism
now that causes the anoncvs mirror to update within a few minutes of a
commit to the master. So it might be worth the trouble to sync more
often.
regards, tom lane
On Wed, 2007-10-24 at 08:33 -0300, Alvaro Herrera wrote: > David Fetter wrote: > > > I'm not picking a DSCM. I'm saying we already have tools in place for > > a DSCM *without* having a "flag day." If Mercurial has a similar > > migration/legacy support path, then by all means, let's try that out, > > too. :) > > There's at least on Mercurial repo, here: > > http://www.ubiobio.cl/~gpoo/pgsql/pgsql/ > > I'm not sure how often it syncs. Once an hour. In short, the way it works is: * Having a CVS copy through rsync. * Any CVS change is added incrementally. You can stop the process and the next time will continue from that point. > For those of you outside Chile my bet is that it will be a bit slow ... Through Internet2 should less slower than Internet, but I guess still slow. But it will be slow the first time you check out the repository. Next time you only will get differences, and Mercurial is network friendly, AFAIU. -- Germán Poó Caamaño Concepción - Chile
Josh Berkus wrote:<br /><blockquote cite="mid200710231551.14349.josh@agliodbs.com" type="cite"><pre wrap="">Folks, </pre><blockquote type="cite"><pre wrap="">You are way ahead of us here. And my vote *still* goes to Mercurial, if we're picking SCMs. </pre></blockquote><pre wrap=""> Will a new SCM actually make this easier, or are people just using it as an excuse? </pre></blockquote> We use mercurial here at work, having switched to it recently, and while I don't claim to be an expert,it does seem nice. For example, you can have a local repository you're checking code into, and can pull from andmerge up with some shared repository. Also, you can pull from one repository and check into another- so, for example,we have a staging repository and a compiles repository (unless you welcome the pain)- you pull from the compilesrepository, but push changes back to the staging repository. Then we have a script that pulls recent changes fromthe staging repository, make sure they compile and the unit tests run, before moving them over to the compiles repository. This way, the version you're pulling at least compiles and passes some minimal unit tests.<br /><br /> A similiarprocess could work for postgres- except instead of "staging" and "compiles" you'd have a "sumbitted" and "accepted"repositories. And instead of a compile daemon, it'd be reviewers who would move code from one to the other. <br/><br /> Note that everything I'm talking about here is not unique to Mercurial- you can do this just about as easilyin darcs or git (I'd advise against Bazaar/bzr)- so don't take this as being pro-Mercurial, just pro-SCM.<br /><br/> Brian<br /><br />
"Martijn van Oosterhout" <kleptog@svana.org> writes: > On Tue, Oct 23, 2007 at 02:39:43PM -0700, David Fetter wrote: >> The one below is already available, so we don't have to do a "flag >> day" with it. >> >> http://repo.or.cz/w/PostgreSQL.git > > As someone who hasn't used GIT: if I have a modified CVS tree from some time > back (>1 year) can I use this to manage the bitrot? Just doing a CVS > update will probably just mark everything conflicted, so I'm wondering > is GIT can do it better. Well it could except not here where the git repository is being fed from CVS. CVS is just not gathering a lot of the information that helps git and the other distributed systems do their job. Namely it has no idea which changes were committed together as part of the same patch, and it has no idea where those changes came from and whether they're also in your tree. -- Gregory Stark EnterpriseDB http://www.enterprisedb.com
On Tue, 2007-10-23 at 19:24 -0400, Tom Lane wrote: > Josh Berkus <josh@agliodbs.com> writes: > > Mind you, I'm in favor of one. A new SCM would make some other development > > tasks easier. However, I'm reluctant to open the can-of-worms which is the > > "what SCM should we use" discussion again, and complicate something which > > we seem to have consensus on. > > As near as I can tell, the arguments for a new SCM mostly apply to work > which individual developers are doing outside the main tree. So, given > the existence of stuff like git-cvsimport, I don't see a strong reason > why anyone who wants to work that way can't already sync the core CVS > with a local SCM-of-their-choice and get on with it. According the proposal of reduce/avoid working with big patches, it would help. While it's even possible to handle separately a SCM and then send a big patch, it's a pain merging it using CVS. And finally the history of that patch is lost. The workflow shouldn't change at all, but the tools can help to improve the process of reviewing/merging patches. If that process may improve, then a 'fixed' release schedule is even more possible, IMVVHO. -- Germán Poó Caamaño Concepción - Chile
"Marko Kreen" <markokr@gmail.com> writes:
> As we seem discussing developement in general, there is one
> obstacle in the way of individual use of DSCMs - context diff
> format as only one accepted.
Well, that's not a hard-and-fast rule, just a preference. At least for
me, unidiff is vastly harder to read than cdiff for anything much beyond
one-line changes. (For one-liners it's great ;-), but beyond that it
intermixes old and new lines too freely.) That's not merely an
impediment to quick review of the patch; if there's any manual
patch-merging to be done, it significantly increases the risk of error.
I don't recall that we've rejected any patches lately just because they
were unidiffs. But I'd be sad if a large fraction of incoming patches
started to be unidiffs.
regards, tom lane
On Wed, Oct 24, 2007 at 09:24:47AM -0400, Tom Lane wrote: > I don't recall that we've rejected any patches lately just because they > were unidiffs. But I'd be sad if a large fraction of incoming patches > started to be unidiffs. We bounce them back to the author pretty m uch every time with "context diff please". But I don't recall any case where the patch was actually rejected because of a submitter refusing to provide -c diff. //Magnus
On Wed, 2007-10-24 at 14:32 +0200, Martijn van Oosterhout wrote: > On Tue, Oct 23, 2007 at 02:39:43PM -0700, David Fetter wrote: > > The one below is already available, so we don't have to do a "flag > > day" with it. > > > > http://repo.or.cz/w/PostgreSQL.git > > As someone who hasn't used GIT: if I have a modified CVS tree from some time > back (>1 year) can I use this to manage the bitrot? Just doing a CVS > update will probably just mark everything conflicted, so I'm wondering > is GIT can do it better. The way you can work could be (it's the same if you are using git/hg/bzr): * Have a copy (clone) of the main repo, which is a pristine copy of CVS. call it 'master'. * Clone or create a branch from the 'master'. This is the working copy. Master can be updated regularly. And also, you can merge the changes in your branch or even, only look the changes without touching anything. What can be merge automatically, it will. Otherwise, a external tool is called (such as meld or whatever you use). Evenmore, you can try merging with previous cvs' commits to avoid the pain. -- Germán Poó Caamaño Concepción - Chile
"Tom Lane" <tgl@sss.pgh.pa.us> writes: > "Marko Kreen" <markokr@gmail.com> writes: >> As we seem discussing developement in general, there is one >> obstacle in the way of individual use of DSCMs - context diff >> format as only one accepted. > > Well, that's not a hard-and-fast rule, just a preference. At least for > me, unidiff is vastly harder to read than cdiff for anything much beyond > one-line changes. (For one-liners it's great ;-), but beyond that it > intermixes old and new lines too freely.) That's not merely an > impediment to quick review of the patch; if there's any manual > patch-merging to be done, it significantly increases the risk of error. > > I don't recall that we've rejected any patches lately just because they > were unidiffs. But I'd be sad if a large fraction of incoming patches > started to be unidiffs. It seems hard to believe this would be a hard problem to overcome. It's not like either format contains more or less information than the other. In fact Emacs's diff-mode can convert between them on the fly. -- Gregory Stark EnterpriseDB http://www.enterprisedb.com
On 10/24/07, Tom Lane <tgl@sss.pgh.pa.us> wrote: > "Marko Kreen" <markokr@gmail.com> writes: > > As we seem discussing developement in general, there is one > > obstacle in the way of individual use of DSCMs - context diff > > format as only one accepted. > > Well, that's not a hard-and-fast rule, just a preference. At least for > me, unidiff is vastly harder to read than cdiff for anything much beyond > one-line changes. (For one-liners it's great ;-), but beyond that it > intermixes old and new lines too freely.) That's not merely an > impediment to quick review of the patch; if there's any manual > patch-merging to be done, it significantly increases the risk of error. > > I don't recall that we've rejected any patches lately just because they > were unidiffs. But I'd be sad if a large fraction of incoming patches > started to be unidiffs. Thanks, maybe the DEVFAQ can be changed that both -u and -c are accepted but -c is preferred. The matter of -c vs. -u is mostly a matter of taste and habit but there is also a technical argument - you can always clean up hard-to-read unidiff with simple /^-/d. But there is no simple way to make hard-to-read context diff readable. -- marko
Marko Kreen wrote: > On 10/24/07, Tom Lane <tgl@sss.pgh.pa.us> wrote: > >> "Marko Kreen" <markokr@gmail.com> writes: >> >>> As we seem discussing developement in general, there is one >>> obstacle in the way of individual use of DSCMs - context diff >>> format as only one accepted. >>> >> Well, that's not a hard-and-fast rule, just a preference. At least for >> me, unidiff is vastly harder to read than cdiff for anything much beyond >> one-line changes. (For one-liners it's great ;-), but beyond that it >> intermixes old and new lines too freely.) That's not merely an >> impediment to quick review of the patch; if there's any manual >> patch-merging to be done, it significantly increases the risk of error. >> >> I don't recall that we've rejected any patches lately just because they >> were unidiffs. But I'd be sad if a large fraction of incoming patches >> started to be unidiffs. >> > > Thanks, maybe the DEVFAQ can be changed that both -u and -c are > accepted but -c is preferred. > > The matter of -c vs. -u is mostly a matter of taste and habit but > there is also a technical argument - you can always clean up > hard-to-read unidiff with simple /^-/d. But there is no simple > way to make hard-to-read context diff readable. > > I would rather stick generally to one style. It's a question of whose convenience prevails, the author's or the reviewer's. I think it should be the reviewer's, and since Tom reviews far more than anyone else his voice accordingly matters most. cheers andrew
Magnus Hagander <magnus@hagander.net> writes:
> On Wed, Oct 24, 2007 at 09:24:47AM -0400, Tom Lane wrote:
>> I don't recall that we've rejected any patches lately just because they
>> were unidiffs. But I'd be sad if a large fraction of incoming patches
>> started to be unidiffs.
> We bounce them back to the author pretty m uch every time with "context
> diff please".
We have, and will continue to, bounce patches that arrive as whole files
or no-context-lines patches. But I know we've taken unidiffs without
complaint. Personally, if I have to read one that's more than isolated
one-line changes, I apply it locally and then use "cvs diff -c" to get a
version I can read ... which makes unidiff only a minor annoyance *as
long as it applies cleanly*. If it doesn't then it's a PITA ...
regards, tom lane
Tom Lane wrote: > Magnus Hagander <magnus@hagander.net> writes: >> On Wed, Oct 24, 2007 at 09:24:47AM -0400, Tom Lane wrote: >>> I don't recall that we've rejected any patches lately just because they >>> were unidiffs. But I'd be sad if a large fraction of incoming patches >>> started to be unidiffs. > >> We bounce them back to the author pretty m uch every time with "context >> diff please". > > We have, and will continue to, bounce patches that arrive as whole files > or no-context-lines patches. But I know we've taken unidiffs without > complaint. Personally, if I have to read one that's more than isolated > one-line changes, I apply it locally and then use "cvs diff -c" to get a > version I can read ... which makes unidiff only a minor annoyance *as > long as it applies cleanly*. If it doesn't then it's a PITA ... You can use "filterdiff -v --format=context". Or emacs's built in command to do the conversion. Because it's easy to convert from one to another, I think the unified vs. context diff issue is a non-issue. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Wed, Oct 24, 2007 at 02:32:13PM +0200, Martijn van Oosterhout wrote: > On Tue, Oct 23, 2007 at 02:39:43PM -0700, David Fetter wrote: > > The one below is already available, so we don't have to do a "flag > > day" with it. > > > > http://repo.or.cz/w/PostgreSQL.git > > As someone who hasn't used GIT: if I have a modified CVS tree from some time > back (>1 year) can I use this to manage the bitrot? Just doing a CVS > update will probably just mark everything conflicted, so I'm wondering > is GIT can do it better. It's no magic, but three-way merges[1] and automatic tracking of file renames do help quite a bit. - ------ [1] most distributed SCMs keep enough info around to dig out the last common ancestor of the two files to be merged and do then a three-way merge, which is better at resolving conflicts than a naïve direct merge. Regards - -- tomás -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) iD8DBQFHH2mpBcgs9XrR2kYRAg/uAJoCSjQ3GDIEZidWYG9IX5J7RF4N9QCfXp+i 614RlRkN38tPTBhZ9ctir0I= =AsX7 -----END PGP SIGNATURE-----
Heikki Linnakangas <heikki@enterprisedb.com> writes:
> You can use "filterdiff -v --format=context".
Cool, I'll have to get a copy of that.
> Because it's easy to convert from one to another, I think the unified
> vs. context diff issue is a non-issue.
Fair enough then; we should just change the policy.
regards, tom lane
Marko Kreen wrote: > On 10/24/07, Tom Lane <tgl@sss.pgh.pa.us> wrote: > > "Marko Kreen" <markokr@gmail.com> writes: > > > As we seem discussing developement in general, there is one > > > obstacle in the way of individual use of DSCMs - context diff > > > format as only one accepted. > > > > Well, that's not a hard-and-fast rule, just a preference. At least for > > me, unidiff is vastly harder to read than cdiff for anything much beyond > > one-line changes. (For one-liners it's great ;-), but beyond that it > > intermixes old and new lines too freely.) That's not merely an > > impediment to quick review of the patch; if there's any manual > > patch-merging to be done, it significantly increases the risk of error. > > > > I don't recall that we've rejected any patches lately just because they > > were unidiffs. But I'd be sad if a large fraction of incoming patches > > started to be unidiffs. > > Thanks, maybe the DEVFAQ can be changed that both -u and -c are > accepted but -c is preferred. > > The matter of -c vs. -u is mostly a matter of taste and habit but > there is also a technical argument - you can always clean up > hard-to-read unidiff with simple /^-/d. But there is no simple > way to make hard-to-read context diff readable. Context diff shows you the old block and new block in its entirety. Unidiff does not, hence the context diff preference. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://postgres.enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
Tom Lane wrote: > Andrew Dunstan <andrew@dunslane.net> writes: > > I'm fairly resistant to putting less-than-ready code in the tree, I must > > say. > > Me too, at least if "less than ready" means "unstable". The committed > code has to always be solid enough to let everybody continue working on > their own bits. However, in the past we've tended to refuse stuff if > it "didn't do anything interesting", and I think that attitude will need > adjustment. We'll have to be prepared to accept patches that are only > interesting as waystations to some long-term goal. I know for the Win32 port I committed things in parts, e.g. fix absolute path detection. It would have been impossible to submit one large patch to do the entire Win32 port, and in fact the feature spread over two major releases. I think if someone took on a similar-sized project we would just have to give them CVS commit access. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://postgres.enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
Tom Lane wrote: > Josh Berkus <josh@agliodbs.com> writes: > > Anyway, is there anyone who thinks the "cycle the queue every 6 weeks or 2 > > months or suitable short period" is a *bad* idea? It might be hard to pull > > off, but we won't know until we try. > > It seems worth a try --- we can certainly abandon it easily if it > doesn't work. > > Personally I feel every six weeks would be too short: we'd be talking > only a month of work between commit-fests. I like a two-month cycle > partly because it wouldn't rotate relative to the calendar: we'd always > know that the first half of every odd-numbered month, or something like > that, is commit-fest time. Sounds fine to me. Basically it is a "reviewers, get your heads out of your monitors and help other patch submitters". Not a lot of additional fun for reviewers, of course, but probably necessary. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://postgres.enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
Tom Lane wrote: > [ chewing on this a bit... ] The curious thing about that is that > despite this being designed to be a short release cycle, we ended up > landing a bunch of major patches that weren't on the radar screen at > all at the start of the cycle. This suggests to me that there's > something wrong with the concept that no one can get anything major done > without a long release cycle to do it in. > > Indeed, the thing that seemed to me to be killing us in this freeze > cycle was that the patches coming in were already at the upper limit of > what we can review and digest. Yep. Hopefully with the patches applied in 8.3 we can have more confidence in several patch submitters so there is less reviewing required, and these new people can help review too. > I'd rather encourage people to work in an incremental, not-so-big-bang > fashion. Obviously one of the requirements for that will be quicker > review turnaround and commit, so that there's time to build on a > previous patch... Sounds good. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://postgres.enterprisedb.com + If your life is a hard drive, Christ can be your backup. +
"Tom Lane" <tgl@sss.pgh.pa.us> writes: > Heikki Linnakangas <heikki@enterprisedb.com> writes: >> You can use "filterdiff -v --format=context". > > Cool, I'll have to get a copy of that. Huh, I hadn't heard of that either. The Debian package patchutils says it was downloaded from: http://cyberelk.net/tim/data/patchutils What's really cool is that patchutils also appears to have the utility I've been looking for for a while: a tool to view the differences between two diffs! >> Because it's easy to convert from one to another, I think the unified >> vs. context diff issue is a non-issue. > > Fair enough then; we should just change the policy. It would be cool if the mail list software (optionally) automatically stripped out diffs and replaced them with links to download the diff in either unified or context format from the mail archives. -- Gregory Stark EnterpriseDB http://www.enterprisedb.com
* Tom Lane <tgl@sss.pgh.pa.us> [071024 08:45]: > Aidan Van Dyk <aidan@highrise.ca> writes: > > * Brendan Jurd <direvus@gmail.com> [071024 01:41]: > >> How up to date is the Git repos? Does it pull individual commits from > >> CVS, or does it resync the whole history periodically? If so, what's > >> the lag? > > > It's updated hourly, which is the same rate the public CVS is updated. > > FWIW, that's no longer the case: there's some sort of trigger mechanism > now that causes the anoncvs mirror to update within a few minutes of a > commit to the master. So it might be worth the trouble to sync more > often. Note that it's working off the anoncvs rsync. But I've set it to sync every 15 minutes for now, we'll see how it goes... -- Aidan Van Dyk Create like a god, aidan@highrise.ca command like a king, http://www.highrise.ca/ work like a slave.
tomas@tuxteam.de wrote: > On Wed, Oct 24, 2007 at 02:32:13PM +0200, Martijn van Oosterhout wrote: > > On Tue, Oct 23, 2007 at 02:39:43PM -0700, David Fetter wrote: > > > The one below is already available, so we don't have to do a "flag > > > day" with it. > > > > > > http://repo.or.cz/w/PostgreSQL.git > > > > As someone who hasn't used GIT: if I have a modified CVS tree from some time > > back (>1 year) can I use this to manage the bitrot? Just doing a CVS > > update will probably just mark everything conflicted, so I'm wondering > > is GIT can do it better. > > It's no magic, but three-way merges[1] and automatic tracking of file > renames do help quite a bit. The other thing that helps is being able to apply (merge, propagate) the changesets from mainline into your branch, one at a time, automatically if they merge cleanly, and require manual intervention when they don't. That way the changes required to merge each time are likely trivial. -- Alvaro Herrera http://www.amazon.com/gp/registry/DXLWNGRJD34J "Investigación es lo que hago cuando no sé lo que estoy haciendo" (Wernher von Braun)
On Wed, Oct 24, 2007 at 02:18:42PM -0300, Alvaro Herrera wrote: > tomas@tuxteam.de wrote: > > On Wed, Oct 24, 2007 at 02:32:13PM +0200, Martijn van Oosterhout wrote: > > > On Tue, Oct 23, 2007 at 02:39:43PM -0700, David Fetter wrote: > > > > The one below is already available, so we don't have to do a "flag > > > > day" with it. > > > > > > > > http://repo.or.cz/w/PostgreSQL.git > > > > > > As someone who hasn't used GIT: if I have a modified CVS tree > > > from some time back (>1 year) can I use this to manage the > > > bitrot? Just doing a CVS update will probably just mark > > > everything conflicted, so I'm wondering is GIT can do it better. > > > > It's no magic, but three-way merges[1] and automatic tracking of > > file renames do help quite a bit. > > The other thing that helps is being able to apply (merge, propagate) > the changesets from mainline into your branch, one at a time, > automatically if they merge cleanly, and require manual intervention > when they don't. That way the changes required to merge each time > are likely trivial. Yes :) Cheers, David. -- David Fetter <david@fetter.org> http://fetter.org/ Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter Skype: davidfetter XMPP: david.fetter@gmail.com Remember to vote! Consider donating to Postgres: http://www.postgresql.org/about/donate
Brendan Jurd wrote: > On 10/24/07, Alvaro Herrera <alvherre@commandprompt.com> wrote: >> Brendan Jurd escribió: >>> Really? I just started playing around with git, and the output from git >>> diff produced the same kind of diff file I would normally get from `svn >>> di` >> ... which is a unified diff. >> >>> or `cvs di -c`. >> Huh, strange. That's a context diff. > > Right. I was confusing unified and context, because the unified diffs I'm > looking at do actually show "context" lines above and below each change. > Sorry for the noise. You can get git to produce a context diff by using GIT_EXTERNAL_DIFF to specify an external diff program. I use a little shell script called git-context-diff.sh, containing: #!/bin/bash diff --context=5 -L a/$1 -L b/$1 -p $2 $5 exit 0 and then create a diff with: GIT_EXTERNAL_DIFF=git-context-diff.sh git diff <head to diff against> Just in case you wondered - writing that script as just "exec diff ..." fails, because diff seems to exit with a nonzero exitcode if the files differ, and "git diff" aborts in that case... regards, Florian Pflug
On 10/25/07, Gregory Stark <stark@enterprisedb.com> wrote: > Huh, I hadn't heard of that either. The Debian package patchutils says it was > downloaded from: > > http://cyberelk.net/tim/data/patchutils > > What's really cool is that patchutils also appears to have the utility I've > been looking for for a while: a tool to view the differences between two > diffs! I wouldn't trust that too much. I've played with it quite a bit and there are quite a few corner cases where it gives the wrong answer (or at least a different diff than I get when I manually apply them and then take the diff) -- Kris Shannon <kris.shannon@gmail.com>
Tom, > This seems pretty entirely orthogonal to the commit-fest proposal. > I see no reason to think that snapshots taken at those times would > be any better than any other nightly snapshot, nor any reason > to memorialize them in an archive. I can see that. And it would be pretty hard to keep members of the public from treating the persistent snapshots as releases, and then flooding the lists when they break. So, scratch an idea with unintended consequences. I'll just have to keep the snapshots on our CVS slave. I do hope that the commit-fests will lead to more testing, though, just because people waiting for a specific feature will want to try out the initial commit of that feature. -- --Josh Josh Berkus PostgreSQL @ Sun San Francisco