Thread: Re: Regression tests fail with tzdata 2024b

Wolfgang Walther <walther@technowledgy.de> writes:
> Building --with-system-tzdata and the latest tzdata 2024b fails the 
> regression tests for me (see attached .diffs). This seems to be because 
> of [1], which changed the way "PST8PDT" is handled. This is the timezone 
> that the regression tests are run with.

That's quite annoying, especially since it was not mentioned in the
2024b release notes.  (I had read the notes and concluded that 2024b
didn't require any immediate attention on our part.)

>  From 2024b on, "PST8PDT" is the same as "America/Los_Angeles", so by 
> changing the regression tests to use the latter as the default, we're 
> getting consistent output on at least 2024a and 2024b.

I'm fairly un-thrilled with this answer, not least because it exposes
that zone's idiosyncratic "LMT" offset of -7:52:58 for years before
1883.  (I'm surprised that that seems to affect only one or two
regression results.)  Also, as a real place to a greater extent
than "PST8PDT" is, it's more subject to historical revisionism when
somebody turns up evidence of local law having been different than
TZDB currently thinks.

We may not have a lot of choice though.  I experimented with using
full POSIX notation, that is "PST8PDT,M3.2.0,M11.1.0", but that is
actually worse in terms of the number of test diffs, since it doesn't
match the DST transition dates that the tests expect for years before
2007.  Another objection is that the tests would then not exercise any
of the mainstream tzdb-file-reading code paths within the timezone
code itself.

Grumble.

            regards, tom lane

Tom Lane:
> Also, as a real place to a greater extent
> than "PST8PDT" is, it's more subject to historical revisionism when
> somebody turns up evidence of local law having been different than
> TZDB currently thinks.

I now tried all versions of tzdata which we had in tree back to 2018g, 
they all work fine with the same regression test output. 2018g was an 
arbitrary cutoff, I just didn't try any further.

In the end, we don't need a default timezone that will never change. We 
just need one that didn't change in a reasonable number of releases 
going backwards. Once America/Los_Angeles is changed, we need to switch 
to a different zone, which could be one that wouldn't work today. Kind 
of a sliding window.

One positive might be: With this timezone, we are more likely to see 
relevant changes mentioned in the upstream release notes.

Best,

Wolfgang

Wolfgang Walther <walther@technowledgy.de> writes:
> Tom Lane:
>> Also, as a real place to a greater extent
>> than "PST8PDT" is, it's more subject to historical revisionism when
>> somebody turns up evidence of local law having been different than
>> TZDB currently thinks.

> I now tried all versions of tzdata which we had in tree back to 2018g, 
> they all work fine with the same regression test output. 2018g was an 
> arbitrary cutoff, I just didn't try any further.

Yeah, my belly-aching above is just about hypothetical future
instability.  In reality, I'm sure America/Los_Angeles is pretty
well researched and so it's unlikely that there will be unexpected
changes in its zone history.

> In the end, we don't need a default timezone that will never change.

We do, really.  For example, there's a nonzero chance the USA will
cancel DST altogether at some future time.  (This would be driven by
politicians who don't remember the last time, but there's no shortage
of those.)  That'd likely break some of our test results, and even if
it happened not to, it'd still be bad because we'd probably lose some
coverage of the DST-transition-related code paths in src/timezone/.
So I'd really be way happier with a test timezone that never changes
but does include DST behavior.  I thought PST8PDT fit those
requirements pretty well, and I'm annoyed at Eggert for having tossed
it overboard for no benefit whatever.  But I don't run tzdb, so
here we are.

> We just need one that didn't change in a reasonable number of
> releases going backwards.

We've had this sort of fire-drill before, e.g. commit 8d7af8fbe.
It's no fun, and the potential surface area for unexpected changes
is now much greater than the few tests affected by that change.

But here we are, so I pushed your patch with a couple of other
cosmetic bits.  There are still a couple of references to PST8PDT
in the tree, but they don't appear to care what the actual meaning
of that zone is, so I left them be.

            regards, tom lane

Sven Klemm <sven@timescale.com> writes:
> This is an unfortunate change as this will break extensions tests using
> pg_regress for testing. We run our tests against multiple minor versions
> and this getting backported means our tests will fail with the next minor
> pg release. Is there a workaround available to make the timezone for
> pg_regress configurable without going into every test?

Configurable to what?  If your test cases are dependent on the
historical behavior of PST8PDT, you're out of luck, because that
simply isn't available anymore (or won't be once 2024b reaches
your platform, anyway).

            regards, tom lane

Sven Klemm <sven@timescale.com> writes:
> On Mon, Sep 16, 2024 at 5:19 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> Configurable to what?  If your test cases are dependent on the
>> historical behavior of PST8PDT, you're out of luck, because that
>> simply isn't available anymore (or won't be once 2024b reaches
>> your platform, anyway).

> I was wondering whether the timezone used by pg_regress could be made
> configurable.

Yes, I understood that you were suggesting that.  My point is that
it wouldn't do you any good: you will still have to change any
regression test cases that depend on behavior PST8PDT has/had that
is different from America/Los_Angeles.  That being the case,
I don't see much value in making it configurable.

            regards, tom lane

Tom Lane:
>> I was wondering whether the timezone used by pg_regress could be made
>> configurable.
> 
> Yes, I understood that you were suggesting that.  My point is that
> it wouldn't do you any good: you will still have to change any
> regression test cases that depend on behavior PST8PDT has/had that
> is different from America/Los_Angeles.  That being the case,
> I don't see much value in making it configurable.

Just changing it back to PST8PDT wouldn't really help as Tom pointed 
out. You'd still get different results depending on which tzdata version 
you are running with.

The core regression tests need to be run with a timezone that tests 
special cases in the timezone handling code. But that might not be true 
for extensions - all they want could be a stable output across major and 
minor versions of postgres and versions of tzdata. It could be helpful 
to set pg_regress' timezone to UTC, for example?

Best,

Wolfgang

Wolfgang Walther <walther@technowledgy.de> writes:
> The core regression tests need to be run with a timezone that tests 
> special cases in the timezone handling code. But that might not be true 
> for extensions - all they want could be a stable output across major and 
> minor versions of postgres and versions of tzdata. It could be helpful 
> to set pg_regress' timezone to UTC, for example?

I would not recommend that choice.  It would mask simple errors such
as failing to apply the correct conversion between timestamptz and
timestamp values.  Also, if you have test cases that are affected by
this issue at all, you probably have a need/desire to test things like
DST transitions.

            regards, tom lane