Thread: Buildfarm's cross-version-upgrade tests
I broke $SUBJECT again today by removing regress_putenv from
regress.so. I don't really want to put it back, but that means
we need a way to drop the corresponding SQL function from the
old version's regression database before attempting an upgrade.
Since I'd just seen Noah's commit 52202bb39 go by, I tried
modifying src/bin/pg_upgrade/test.sh to do the drop, but that
isn't helping. I recall now from prior discussions that we
have to modify code that's embedded in the buildfarm client
script to make this go. (I wonder what test scenario Noah had
in mind, exactly.)
Realizing that this has happened before and will happen again,
I'm now wondering if we can't devise a fixup method that is
controllable by committers and doesn't require a buildfarm
client rollout to update. I'm thinking maybe we could extract
the fixup logic from test.sh into a standalone SQL script that's
part of the regular source tree so it can be updated by committers,
and then both test.sh and the buildfarm script could invoke it.
Maybe that won't work for some reason, but I'd sure like to find
some way of handling this without making you get involved every time.
regards, tom lane
On 12/30/20 3:42 PM, Tom Lane wrote: > I broke $SUBJECT again today by removing regress_putenv from > regress.so. I don't really want to put it back, but that means > we need a way to drop the corresponding SQL function from the > old version's regression database before attempting an upgrade. > > Since I'd just seen Noah's commit 52202bb39 go by, I tried > modifying src/bin/pg_upgrade/test.sh to do the drop, but that > isn't helping. I recall now from prior discussions that we > have to modify code that's embedded in the buildfarm client > script to make this go. (I wonder what test scenario Noah had > in mind, exactly.) > > Realizing that this has happened before and will happen again, > I'm now wondering if we can't devise a fixup method that is > controllable by committers and doesn't require a buildfarm > client rollout to update. I'm thinking maybe we could extract > the fixup logic from test.sh into a standalone SQL script that's > part of the regular source tree so it can be updated by committers, > and then both test.sh and the buildfarm script could invoke it. > Maybe that won't work for some reason, but I'd sure like to find > some way of handling this without making you get involved every time. > > Well, it's not hugely onerous, although it does seem to have happened a bit more lately than once it used to. Basically, there are two sets of adjustments. First there are the things we do when we're saving a cluster for later upgrade testing. These can be found at lines 238 - 300 of the module (see <https://github.com/PGBuildFarm/client-code/blob/master/PGBuild/Modules/TestUpgradeXversion.pm>). Then, when it comes time to do an upgrade test, we make a copy of the cluster and make further adjustments depending on the target version as well as the source version. See lines 363 - 496. Some of this is a bit more complicated because we test all the way back to 9.2, even though it's past EOL. It seems reasonable to want to institutionalize this knowledge. I'll see if I can extract it into one or two perl scripts suitable for inclusion in core code. I'll try to fix the test for the latest breakage shortly. cheers andrew -- Andrew Dunstan EDB: https://www.enterprisedb.com
On Wed, Dec 30, 2020 at 04:28:44PM -0500, Andrew Dunstan wrote: > On 12/30/20 3:42 PM, Tom Lane wrote: > > Since I'd just seen Noah's commit 52202bb39 go by, I tried > > modifying src/bin/pg_upgrade/test.sh to do the drop, but that > > isn't helping. I recall now from prior discussions that we > > have to modify code that's embedded in the buildfarm client > > script to make this go. (I wonder what test scenario Noah had > > in mind, exactly.) Commit 52202bb39 affected the cross-version test harness described in src/bin/pg_upgrade/TESTING. That harness is disused, having been broken at head for at least 40 months. (One can work around the remaining breakage by locally back-patching e78900a into REL_13_STABLE.) > > I'm now wondering if we can't devise a fixup method that is > > controllable by committers and doesn't require a buildfarm > > client rollout to update. Ideally, the buildfarm code should curate per-version source and installation trees, provide their locations to the test harness, and collect logs. Other test logic (i.e. the kinds of steps that appear in test.sh) should live in postgresql.git code. (That's the separation of concerns for all run-by-default buildfarm tests.)
On 12/30/20 4:28 PM, Andrew Dunstan wrote: > > I'll try to fix the test for the latest breakage shortly. See <https://github.com/PGBuildFarm/client-code/commit/fa86d0b1bc7a8d7b9f15b1da8b8e43f4d3a08e2b> I'm going to get a new release out before next Thursday come hell or high water. cheers andrew -- Andrew Dunstan EDB: https://www.enterprisedb.com