Re: 9.5 release scheduling (was Re: logical column ordering) - Mailing list pgsql-hackers

On Thu, Dec 11, 2014 at 10:24:20AM -0800, Jeff Janes wrote:
> On Thu, Dec 11, 2014 at 8:03 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> > 2. The amount of pre-release testing we get from people outside the
> > hard-core development crowd seems to be continuing to decrease.
> > We were fortunate that somebody found the JSONB issue before it was
> > too late to do anything about it.

> We are not particularly inviting of feedback for whatever testing has been
> done.
> 
> The definitive guide seems to be
> https://wiki.postgresql.org/wiki/HowToBetaTest, and says:
> 
> You can report tests by email. You can subscribe to any PostgreSQL mailing
> list from the subscription form <http://www.postgresql.org/community/lists/>
> .
> 
>    - pgsql-bugs: this is the preferred mailing list if you think you have
>    found a bug in the beta. You can also use the Bug Reporting Form
>    <http://www.postgresql.org/support/submitbug/>.
>    - pgsql-hackers: bugs, questions, and successful test reports are
>    welcome here if you are already subscribed to pgsql-hackers. Note that
>    pgsql-hackers is a high-traffic mailing list with a lot of development
>    discussion.
> 
> 
> =========
> 
> So if you find a bug, you can report it on the bug reporting form (which
> doesn't have a drop-down entry for 9.4RC1).

Let's get 9.5 alpha/beta/rc releases into that drop-down as we release them.

> If you have positive results rather than negative ones (or even complaints
> that are not actually bugs), you can subscribe to a mailing list which
> generates a lot of traffic which is probably over your head and not
> interesting to you.

Feel welcome to revise that part.  Don't messages from non-subscribed people
make it to the list after manual moderation?  Testers might want to create a
no-delivery subscription to avoid moderation delay, but the decision to
receive all -hackers mail is separate.

> Does the core team keep a mental list of items they want to see tested by
> the public, and they will spend their own time testing those things
> themselves if they don't hear back on some positive tests for them?

Not sure about the core team.  I myself would test essentially the same things
during beta regardless of what end users report having tested, because end
users will pick different test scenarios for the same features.

> If we find reports of public testing that yields good results (or at least
> no bugs) to be useful, we should be more clear on how to go about doing
> it.  But are positive reports useful?  If I report a bug, I can write down
> the steps to reproduce it, and then follow my own instructions to make sure
> it does actually reproduce it.  If I find no bugs, it is just "I did a
> bunch of random stuff and nothing bad happened, that I noticed".

Positive reports have potential to be useful.  In particular, mention the new
features you took action to try.  Areas like BRIN, pg_rewind, foreign tables,
event triggers, CREATE POLICY, INSERT ... ON CONFLICT, and GROUPING SETS are
either completely new or have new sub-features.  If nothing else, we can CC
reporters when considering changes to features they reported upon.  Other
analysis would become attractive given a larger corpus of positive reports.

Thanks,
nm