Re: When to switch to Postgres 8.0? - Mailing list pgsql-general

Christopher Browne <cbbrowne@acm.org> writes:
> The world rejoiced as jao@geophile.com wrote:
>> Suppose 8.0 is released in December or January. Is it a sane thing
>> to put 8.0 in a shipping product within the first two months of its
>> release?
>> ...
>> Suppose I don't use any new features. The question is then whether
>> I'm likely to be affected by a regression.
>> ...
>> We'll be doing extensive testing between code freeze and our ship
>> date.

> Certainly a good move...

My advice would be "trust, but verify" ... that is, use 8.0 and test it
as hard as you can.  You will be contributing something back to the
community by so testing, and you will probably end up with a better
product than if you took the "conservative" route of using 7.4.

While it's certainly true that 8.0 includes some major changes, so did
the last several releases; and nonetheless my feeling is that each
release has been higher quality on average than its predecessor.  We
are on average introducing fewer (post-release) bugs than we fix.

To try to provide some hard evidence, I dug through the recent CVS
change logs.  Attached are all the post-release 7.4 bug fixes that
did not either (a) represent a bug that also existed in 7.3, or
(b) represent a fix in a feature that did not exist in 7.3.  Thus
this list would be a fair gauge of your risk had you updated to 7.4
on Day Zero.  (Disclaimer: I didn't try to evaluate some ecpg and
tsearch bugs that were fixed in this interval; maybe they were in
7.3 too, or maybe not.)

There are twenty bugs in this list.  It is worth pointing out that in
the same interval there were also twenty nontrivial patches applied to
both 7.3 and 7.4 branches (ie, bugs older than 7.4), as well as several
bugs that were fixed in 7.4 but not in 7.3 because the change seemed too
invasive/risky for the older branch.  Moreover, only one of the
new-in-7.4 bugs poses any risk of data corruption, whereas three of the
7.4-and-7.3 bugs did.

I'm too lazy to do a comparable analysis of the prior releases, but my
gut feeling is that every release since 6.5 or so has been as good or
better than its predecessor, even on day zero.  Yeah, there are
localized regressions, but there are also lots of bugs fixed.  The way
to defend yourself against being bit by a localized regression is to get
involved with testing your applications against the beta releases, as
early and aggressively as possible.

The only aspect of 8.0 that I don't care to play Pollyanna about is the
native Windows port.  That's a whole lot of new code with no track
record and not that much field testing yet.  But as long as you are
thinking of running on a non-Microsoft OS, I'd say "go for it".

            regards, tom lane


2004-08-17 19:15  tgl

    * src/backend/access/nbtree/: nbtinsert.c (REL7_4_STABLE),
    nbtinsert.c: Fix bug introduced into _bt_getstackbuf() on
    2003-Feb-21: the initial value of 'start' could be past the end of
    the page, if the page was split by some concurrent inserting
    process since we visited it.  In this situation the code could look
    at bogus entries and possibly find a match (since after all those
    entries still contain what they had before the split).    This would
    lead to 'specified item offset is too large' followed by 'PANIC:
    failed to add item to the page', as reported by Joe Conway for
    scenarios involving heavy concurrent insertion activity.

7.4.3

2004-05-10 22:21  tgl

    * src/backend/optimizer/plan/planner.c (REL7_4_STABLE): Repair
    recalculation failure for nested sub-SELECTs, per bug report from
    Didier Moens.  Bug is new in 7.4, and was caused by not updating
    everyplace I should've when replacing locParam markers by allParam.

2004-03-12 19:54  tgl

    * src/backend/executor/: nodeAgg.c (REL7_4_STABLE), nodeAgg.c:
    Repair memory leakage introduced into the non-hashed aggregate case
    by 7.4 rewrite for hashed aggregate support.  If the transition
    data type is pass-by-reference, the transValue must be pfreed when
    starting a new group boundary, else we have a one-value-per-group
    leakage.  Thanks to Rae Steining for providing a reproducible test
    case.

7.4.2

2004-03-02 13:56  tgl

    * src/: backend/executor/execAmi.c, backend/executor/execMain.c,
    include/executor/executor.h (REL7_4_STABLE),
    backend/executor/execAmi.c, backend/executor/execMain.c,
    include/executor/executor.h: Junkfilter logic to force a projection
    step during SELECT INTO was too simplistic; it recognized SELECT *
    FROM but not SELECT * FROM LIMIT.  Per bug report from Jeff Bohmer.

2004-02-29 12:36  tgl

    * src/backend/optimizer/plan/: createplan.c (REL7_4_STABLE),
    createplan.c: make_sort_from_pathkeys()'s method for choosing which
    of several equivalent sort expressions to use was broken: you can't
    just look at the relation membership, you have to actually grovel
    over the individual Vars in each expression.  I think this did work
    when it was written, but it was broken by subsequent optimizations
    that made join relations not propagate every single input variable
    upward.  Must find the Var that got propagated, not choose one at
    random.  Per bug report from Daniel O'Neill.

2004-02-24 17:59  tgl

    * src/include/catalog/: pg_type.h (REL7_4_STABLE), pg_type.h:
    anyarray really needs to be declared with typalign = 'd', so that
    entries in pg_statistic are correctly aligned if they contain
    values that require double alignment.  Too bad we cannot force
    initdb for this in 7.4 branch.

2004-02-15 10:40  meskes

    * src/interfaces/ecpg/preproc/preproc.y (REL7_4_STABLE): - Allowed
    some C keywords to be used as SQL column names. This used to work
    in 7.3.*

2004-02-13 17:26  tgl

    * src/backend/optimizer/plan/: planner.c (REL7_4_STABLE),
    planner.c: Repair optimization bug I introduced in a moment of
    brain fade back in Nov 2002: when constant-expression
    simplification removes all the aggregate function calls from a
    query, that doesn't mean we can act as though there never were any
    aggregates.  Per bug report from Gabor Szucs.

2004-01-28 02:46  tgl

    * src/backend/parser/: parse_agg.c (REL7_4_STABLE), parse_agg.c:
    Fix oversight in check_ungrouped_columns optimization that avoids
    unnecessary checks for complex grouping expressions: we cannot
    check whether the expressions are simple Vars until after we apply
    flatten_join_alias_vars, because in the case of FULL JOIN that
    routine can introduce non-Var expressions.  Per example from Joel
    Knight.

2004-01-23 19:37  tgl

    * src/backend/optimizer/path/: joinrels.c (REL7_4_STABLE),
    joinrels.c: Repair planner failure for cases involving Cartesian
    products inside IN (sub-SELECT) constructs.  We must force a
    clauseless join of the sub-select member relations, but it wasn't
    happening because the code thought it would be able to use the join
    clause arising from the IN.

2004-01-21 21:23  tgl

    * src/: backend/executor/execMain.c, backend/executor/execScan.c,
    backend/executor/execUtils.c, backend/executor/nodeAppend.c,
    include/executor/executor.h, include/nodes/execnodes.h
    (REL7_4_STABLE), backend/executor/execMain.c,
    backend/executor/execScan.c, backend/executor/execUtils.c,
    backend/executor/nodeAppend.c, include/executor/executor.h,
    include/nodes/execnodes.h: Fix oversight in optimization that
    avoids an unnecessary projection step when scanning a table that we
    need all the columns from.  In case of SELECT INTO, we have to
    check that the hasoids flag matches the desired output type, too.
    Per report from Mike Mascari.

2004-01-17 21:15  tgl

    * src/backend/commands/: copy.c (REL7_4_STABLE), copy.c: Don't use
    %s-with-precision format spec to truncate data being displayed in a
    COPY error message.  It seems that glibc gets indigestion if it is
    asked to truncate strings that contain invalid UTF-8 encoding
    sequences.  vsnprintf will return -1 in such cases, leading to
    looping and eventual memory overflow in elog.c.  Instead use our
    own, more robust pg_mbcliplen routine.    I believe this problem
    accounts for several recent reports of unexpected 'out of memory'
    errors during COPY IN.

2004-01-17 19:31  tgl

    * src/backend/optimizer/plan/createplan.c (REL7_4_STABLE): Repair
    faulty plan generation in cases where we choose to implement an IN
    clause by mergejoin, and a type coercion is needed just above the
    subplan.  A more extensive patch will follow in HEAD.

2003-12-30 15:05  tgl

    * src/backend/executor/: nodeIndexscan.c (REL7_4_STABLE),
    nodeIndexscan.c: Avoid running out of memory during hash_create, by
    not passing a number-of-buckets that exceeds the size we actually
    plan to allow the hash table to grow to.  Per trouble report from
    Sean Shanny.

2003-12-28 12:43  tgl

    * src/interfaces/libpq/: fe-protocol3.c (REL7_4_STABLE),
    fe-protocol3.c: Fix sanity-check code that mistakenly assumed error
    and notice messages could never exceed 30K.  Per report from
    Andreas Pflug.

7.4.1

2003-12-17 14:49  tgl

    * src/backend/parser/: parse_coerce.c (REL7_4_STABLE),
    parse_coerce.c: Reorder tests in parse_coerce so that
    ANY/ANYELEMENT/ANYARRAY coercion does not affect UNKNOWN-type
    literals or Params.  This fixes the recent complaint about
    count('x') being broken, and improves consistency in a few other
    respects too.

2003-12-17 12:07  tgl

    * src/backend/optimizer/path/: allpaths.c, joinrels.c
    (REL7_4_STABLE), allpaths.c, joinrels.c: Repair planner failure
    when there are multiple IN clauses, each with a join in its
    subselect.  In this situation we *must* build a bushy plan because
    there are no valid left-sided or right-sided join trees.
    Accordingly, hoary sanity check needs an update.  Per report from
    Alessandro Depase.

2003-12-08 13:19  tgl

    * src/backend/optimizer/util/: relnode.c (REL7_4_STABLE),
    relnode.c: Whole-row references were broken for subqueries and
    functions, because attr_needed/attr_widths optimization failed to
    allow for Vars with attno zero in this case.  Per report from
    Tatsuo Ishii.

2003-12-03 12:45  tgl

    * src/: backend/optimizer/path/costsize.c,
    backend/optimizer/path/pathkeys.c, backend/utils/cache/lsyscache.c,
    include/utils/lsyscache.h (REL7_4_STABLE),
    backend/optimizer/path/costsize.c,
    backend/optimizer/path/pathkeys.c, backend/utils/cache/lsyscache.c,
    include/utils/lsyscache.h: Planner failed to be smart about
    binary-compatible expressions in pathkeys and hash bucket-size
    estimation.  Issue has been there awhile but is more critical in
    7.4 because it affects varchar columns.  Per report from Greg
    Stark.

2003-11-25 14:17  tgl

    * src/backend/: executor/nodeHashjoin.c,
    optimizer/plan/createplan.c (REL7_4_STABLE): Band-aid solution for
    problems with SubPlans used in hash join clauses, per report from
    Andrew Holm-Hansen.  The difficulty arises from the fact that the
    planner allowed a Hash node's hashkeys to share substructure with
    the parent HashJoin node's hashclauses, plus some rather bizarre
    choices about who initializes what during executor startup.  A
    cleaner but more invasive solution is to not store hashkeys
    separately in the plan tree at all, but let the HashJoin node
    deconstruct hashclauses during executor startup.  I plan to fix it
    that way in HEAD.