Re: Two weeks to feature freeze - Mailing list pgsql-hackers

> -----Original Message-----
> From: Jason Earl [mailto:jason.earl@simplot.com]
> Sent: Friday, June 20, 2003 3:32 PM
> To: Dann Corbit
> Cc: Jason Earl; The Hermit Hacker; PostgreSQL-development
> Subject: Re: [HACKERS] Two weeks to feature freeze
>
>
> "Dann Corbit" <DCorbit@connx.com> writes:
> >>
> >> Why couldn't you just release the win32 version of 7.4 when
> >> it was finished.  If it takes an extra month then that just
> >> gives you guys the chance to circulate *two* press releases.
> >> The Native Win32 port is likely to make a big enough splash
> >> all by itself.
> >
> > A formal release needs a big testing effort.  Two separate releases
> > will double the work of validation.
>
> There are several problems with that statement.  The first is
> that PostgreSQL's "testing effort" happens right here on this
> mailing list.

That's not exactly reassuring.  There is no regression test that gets
formal acceptance?!

> The various PostgreSQL hackers code stuff up,
> and we try and break it. There's very little /effort/
> involved.  People that want the new features go out on a limb
> and start using them.  If they don't work, then they bring it
> up on the list.  If they do work then very little gets said.
>
> As it now stands Tom Lane is on the record as stating that
> the new Win32 version isn't going to be ready for production
> anyhow.  If anything the Win32 version *should* get released
> separately simply because we don't want people mistaking the
> Win32 version as being up to the PostgreSQL teams high
> standards.  Those people that want the Win32 version to
> become production ready are going to have to risk their
> precious data.  Otherwise, the Win32 version will likely
> remain a second class citizen forever.
>
> The fact of the matter is that the Win32 specific bits are
> the parts that are likely to break in the new port.  If
> anything the Windows version will *benefit* from an earlier
> *nix release because the *nix users will chase down the bugs
> in the new PostgreSQL features.  Once the *nix version is up
> to 7.4.2 (or so) then a Windows release of 7.4.2 should allow
> the PostgreSQL hackers to simply chase down Windows specific problems.

Then using the same logic, the new Windows version should wait
indefinitely, since the *nix version will always be shaking out bugs.
> Adding a new platform--especially a platform as diverse from
> the rest of PostgreSQL's supported platforms as Windows--is
> what adds the work. Testing the new platform is relatively
> easy.  All you need to do is to start using the Win32 version
> with real live data.

That is not testing.  Using the world as your beta team seems to be a
business model used by a few software giants that is largely frowned
upon.  I would think that there is an opportunity to do things
differently. [Read 'properly'].

We (at CONNX Solutions Inc.) have a formal release procedure that
includes many tens of thousands of automated tests using dozens of
different platforms.  There are literally dozens of machines (I would
guess 70 or so total) running around the clock for 7 days before we even
know if we have a release candidate.  The QA team is distinct from the
development team, and if they say "FLOP!" the release goes nowhere.  No
formal release until QA passes it.

If there is no procedure for PostgreSQL of this nature, then there
really needs to be.  I am sure that MySQL must have something in place
like that.  Their "Crash-Me" test suite shows (at least) that they have
put a large effort into testing.