Re: Bison 3.0 updates - Mailing list pgsql-hackers

On Mon, Jul 29, 2013 at 11:05:52AM -0400, Tom Lane wrote:
> Andrew Dunstan <andrew@dunslane.net> writes:
> > On 07/29/2013 10:28 AM, Marti Raudsepp wrote:
> >> Hmm? Anchovy is upgrading automatically to newest Arch Linux packages daily.
> >> I assumed that's a good thing -- the purpose of build farm is to test
> >> PostgreSQL in various different real-life environments? Arch Linux is
> >> one such environment that adopts new packages very quickly. If Arch
> >> users are unable to build PostgreSQL then surely it's good to be
> >> notified by the build farm before real users start reporting problems?
> 
> > The buildfarm is principally designed to detect when some change in the 
> > Postgres code breaks something. As such, the expectation is that the 
> > animals will provide a fairly stable platform.  A totally moving target 
> > will present us with false positives, since the failure could be due to 
> > no action of ours at all.
> 
> I can see both sides of this.  It's definitely nice to get early warning
> when toolchain changes create new problems for Postgres, but it's not
> clear that the buildfarm is the best way to get such notifications.

Early notification when a dependency's compatibility break affects PostgreSQL
is a Very Good Thing.

> The big problem here is that it might take a long time before we have
> a suitable fix, and in the meantime that buildfarm member is basically
> useless: it can't tell us anything new, and even if it tried, probably
> nobody would notice because they'd not realize the failure cause changed.
> We've had cases before where a buildfarm member that was failing for
> some known reason was unable to detect a later-introduced bug, so this
> isn't hypothetical.  (And I'm not too thrilled that we've let the
> back-branch failures on anchovy persist for months, though I suppose
> that's not as bad as a breakage in HEAD.)

True, but the solution, if any, is more buildfarm members.  anchovy plays the
important role of a sentinel for trouble with bleeding-edge dependency
packages.  Doing that with excellence inevitably makes it less useful for
detecting other kinds of problems.

> On the other hand, this
> doesn't seem like a big enough problem to justify building some
> different infrastructure for it, so I'm not seeing a practical way
> to do better.

Agreed.  Let's stick an "Updates OS packages daily, regularly acquiring
bleeding-edge upstream releases" note on the buildfarm status page, like we
have for the CLOBBER_CACHE_ALWAYS members.  That should be enough for folks to
realize that a failure in this animal alone is more likely the fault of a new
package version than of the latest commit.

-- 
Noah Misch
EnterpriseDB                                 http://www.enterprisedb.com