Thread: Redhat 7.3 time manipulation bug
Hi,
Something is pretty broken in redhat 7.3 but I'm not sure what and I
don't have time to dig further
masm@test=# select cast('1967-04-18' as timestamptz); timestamptz
------------------------1967-04-17 18:00:00-06
(1 row)
masm@test=# select cast(cast('1967-04-18' as date) as timestamp);
ERROR: Unable to convert date to tm
masm@test=#
Both cases works correctly in Redhat 7.2. Sorry if this is not the
correct forum however an alert is nice for people planning an Redhat
upgrade. I hope to see pretty soon an update since I don't want to
downgrade my server.
Regards,
Manuel.
On Saturday 18 May 2002 12:34 am, Manuel Sugawara wrote: > Something is pretty broken in redhat 7.3 but I'm not sure what and I > don't have time to dig further > Both cases works correctly in Redhat 7.2. Sorry if this is not the > correct forum however an alert is nice for people planning an Redhat > upgrade. I hope to see pretty soon an update since I don't want to > downgrade my server. Filed on Red Hat's Bugzilla system as bug# 65227, as I can reliably reporoduce this bug here, and PostgreSQL 7.2.1 on Red Hat 6.2 on SPARC does not exhibit the bug. Tom (or Thomas): Where would we go to ferret out the source of this bug? More to the point: we need a test case in C that could expose this as a glibc bug. Methinks Red Hat would want this bug ferretted out, as it would likely cause problems with RedHat Database on RH 7.3's glibc..... -- Lamar Owen WGCR Internet Radio 1 Peter 4:11
Lamar Owen <lamar.owen@wgcr.org> writes: > Filed on Red Hat's Bugzilla system as bug# 65227, as I can reliably > reporoduce this bug here, and PostgreSQL 7.2.1 on Red Hat 6.2 on > SPARC does not exhibit the bug. Thanks for filling that report. I couldn't remember what had forgotten ;-) Regards, Manuel.
Lamar Owen <lamar.owen@wgcr.org> writes: > Where would we go to ferret out the source of this bug? More to the > point: we need a test case in C that could expose this as a glibc > bug. Seems like mktime(3) is having problems with dates before the epoch. Attached is the a program to test this. The glibc source is now downloading I will try to hunt down this bug but not until the next week. Regards, Manuel.
Attachment
On Monday 20 May 2002 08:08 pm, Manuel Sugawara wrote: > > Where would we go to ferret out the source of this bug? More to the > > point: we need a test case in C that could expose this as a glibc > > bug. > Seems like mktime(3) is having problems with dates before the > epoch. Attached is the a program to test this. The glibc source is now > downloading I will try to hunt down this bug but not until the next > week. It's not a bug. At least not according to the ISO C standard. See http://www.opengroup.org/onlinepubs/007904975/basedefs/xbd_chap04.html#tag_04_14 for the definition of 'Seconds Since the Epoch', then cross-reference to the man page of mktime. I don't like it any more than you do, but that is the letter of the standard. Thomas, any comments? Our implementation is broken, then. Thomas, is this fixable for a 7.2.x release, or something for 7.3? -- Lamar Owen WGCR Internet Radio 1 Peter 4:11
...
> Our implementation is broken, then. Thomas, is this fixable for a 7.2.x
> release, or something for 7.3?
"Our implementation is broken, then" is really not a conclusion to be
reached. The de facto behavior of mktime() on all world-class
implementations is to support pre-1970 times. This has been true forever
afaik, certainly far longer than PostgreSQL (or Postgres) has been in
existance.
Any standard which chooses to ignore pre-1970 dates is fundamentally
broken imho, and I'm really ticked off that the glibc folks have so
glibly introduced a change of this nature and magnitude without further
discussion.
- Thomas
Lamar Owen <lamar.owen@wgcr.org> writes: > On Monday 20 May 2002 08:08 pm, Manuel Sugawara wrote: > > > Where would we go to ferret out the source of this bug? More to the > > > point: we need a test case in C that could expose this as a glibc > > > bug. > > > Seems like mktime(3) is having problems with dates before the > > epoch. Attached is the a program to test this. The glibc source is now > > downloading I will try to hunt down this bug but not until the next > > week. > > It's not a bug. At least not according to the ISO C standard. See > http://www.opengroup.org/onlinepubs/007904975/basedefs/xbd_chap04.html#tag_04_14 > for the definition of 'Seconds Since the Epoch', then > cross-reference to the man page of mktime. I see. This behavior is consistent with the fact that mktime is supposed to return -1 on error, but then is broken in every other Unix implementation that I know. Any other workaround than downgrade or install FreeBSD? Regards, Manuel.
On Tue, 21 May 2002, Lamar Owen wrote: > On Tuesday 21 May 2002 11:04 am, Manuel Sugawara wrote: > > I see. This behavior is consistent with the fact that mktime is > > supposed to return -1 on error, but then is broken in every other Unix > > implementation that I know. > > > Any other workaround than downgrade or install FreeBSD? > > Complain to Red Hat. Loudly. However, as this is a glibc change, other > distributors are very likely to fold in this change sooner rather than > later. Relying on nonstandardized/nondocumented behaviour is a program bug, not a glibc bug. PostgreSQL needs fixing. Since we ship both, we're looking at it, but glibc is not the component with a problem. -- Trond Eivind Glomsrød Red Hat, Inc.
On Tuesday 21 May 2002 11:04 am, Manuel Sugawara wrote: > I see. This behavior is consistent with the fact that mktime is > supposed to return -1 on error, but then is broken in every other Unix > implementation that I know. > Any other workaround than downgrade or install FreeBSD? Complain to Red Hat. Loudly. However, as this is a glibc change, other distributors are very likely to fold in this change sooner rather than later. Try using timestamp without timezone? -- Lamar Owen WGCR Internet Radio 1 Peter 4:11
On Tuesday 21 May 2002 12:31 pm, Trond Eivind Glomsrød wrote: > On Tue, 21 May 2002, Lamar Owen wrote: > > However, as this is a glibc change, other > > distributors are very likely to fold in this change sooner rather than > > later. > Relying on nonstandardized/nondocumented behaviour is a program bug, not a > glibc bug. PostgreSQL needs fixing. Since we ship both, we're looking at > it, but glibc is not the component with a problem. In your opinion. Not everyone agrees with the new glibc behavior. In fact, some here are rather livid about it. But that's a digression. The matter at hand is making it work right again. It seems to me someone should have thought about this before making the glibc change that had the potential for breaking a large application -- regardless of who is at fault, it's egg in Red Hat's face for not catching it sooner (and egg in my face as well, as I must admit that I of all people should have caught this earlier). Is the change in glibc behavior noted in the release notes? The man page isn't changed either, from Red Hat 6.2, in fact. The only change is adhering to the ISO definition of 'Seconds Since the Epoch' rather than the defacto industry-accepted definition that has been with us a very long time. Like PostgreSQL's refusal to upgrade in a sane manner, this should have received release notes billing, IMHO. Then I, nor anyone else, could reasonably complain. But this does show the need for the regression testing packge, no? :-) And it also shows the danger in becoming too familiar with certain regression tests failing due to locale -- to the extent that a locale issue was the first thing thought of. To the extent that I'm going to change my build process to include regression testing as a part of the build -- and any failure will abort the build. Maybe that will get my attention. And anyone else's rebuilding from the source RPM. SuSE already does this. I wonder how they've handled this issue with 8.0? In any case, this isn't just a Red Hat problem, as it's going to cause problems with the use of timestamps on ANY glibc 2.2.5 dist. That's more than Red Hat, by a large margin. And I think that it is naive to think that PostgreSQL is the only program that has used mktime's behavior in the negative-time_t zone. Other packages are likely impacted, to a greater or lesser extent. -- Lamar Owen WGCR Internet Radio 1 Peter 4:11
Trond Eivind Glomsrød <teg@redhat.com> writes:
> Relying on nonstandardized/nondocumented behaviour is a program bug,
> not a glibc bug.
The question is: how this thing didn't show up before? ISTM that
someone is not doing his work correctly.
> PostgreSQL needs fixing.
Arguably, however, right now is *a lot easier* to fix glibc, and it's
really needed for production systems using postgreSQL and working on
RedHat. But redhat users doesn't matter, the most important thing is
*strict* conformace to standars, right?
> Since we ship both, we're looking at it, but glibc is not the
^^^^^^^^^^^^^^^^^^^
The sad true is: you only answered when the 'Complain to Red Hat'
statement appeared, not a single word before and not a single word
when the bug report were closed. I'm really disappointed.
The nice thing is: glibc is free software and we don't have to wait or
relay on some of the redhat staff members (thanks god) for this to get
fixed or say: for the standard to get extended again. The patch to
glibc is pretty straightforward and attached.
Regards,
Manuel.
Attachment
On 21 May 2002, Manuel Sugawara wrote: > Trond Eivind Glomsrød <teg@redhat.com> writes: > > > Relying on nonstandardized/nondocumented behaviour is a program bug, > > not a glibc bug. > > The question is: how this thing didn't show up before? ISTM that > someone is not doing his work correctly. FWIW, I ran the regressions tests some time ago(probably before that change to glibc) . Since the tests are known to be broken wrt. time issues anyway (as well as currency, math and sorting), it's easy to overlook. > > PostgreSQL needs fixing. > > Arguably, however, right now is *a lot easier* to fix glibc, and it's > really needed for production systems using postgreSQL and working on > RedHat. You're not "fixing" glibc, you're reintroducing non-standardized, upstream removed behaviour. That's typically a very bad thing. If anything, it demonstrates the importance of not using or relying on unstandardized/undocumented behaviour (and given that time_t is pretty restrictive anyway, you'll need something else to keep dates. It doesn't even cover all living people, and definitely not historical dates). > > Since we ship both, we're looking at it, but glibc is not the > ^^^^^^^^^^^^^^^^^^^ > The sad true is: you only answered when the 'Complain to Red Hat' > statement appeared, not a single word before and not a single word > when the bug report were closed. I'm really disappointed. The bug wasn't open for long, and was closed by someone else. > The nice thing is: glibc is free software Also, notice that this was where the fix came from: The upstream maintainers (some of whom work for us, others don't). -- Trond Eivind Glomsrød Red Hat, Inc.
On Tuesday 21 May 2002 03:09 pm, Trond Eivind Glomsrød wrote: > FWIW, I ran the regressions tests some time ago(probably before that > change to glibc) . Since the tests are known > to be broken wrt. time issues anyway (as well as currency, math and > sorting), it's easy to overlook. The time tests have never broken in this manner before on Red Hat. When the original regression failure report was posted, I saw right away that this was not the run of the mill locale issue -- this was a real problem. Regression testing must become a regularly scheduled activity, methinks. In the RPM build process, we can control the locale to the extent that the tests will pass (except on DST days) reliably. I am going to implement this for my next RPM set. Along with a patch to this problem -- we _can_ patch around this, I believe, but it's not likely going to be an easy one. We have gotten blind to the regular locale-induced failures -- this is not a good thing. -- Lamar Owen WGCR Internet Radio 1 Peter 4:11
Trond Eivind Glomsrød <teg@redhat.com> writes:
> Relying on nonstandardized/nondocumented behaviour is a program bug, not a
> glibc bug. PostgreSQL needs fixing. Since we ship both, we're looking at
> it, but glibc is not the component with a problem.
A library that can no longer cope with dates before 1970 is NOT my idea
of a component without a problem. We will be looking at ways to get
around glibc's breakage at the application level, since we have little
alternative other than to declare Linux an unsupported platform;
but it's still glibc (and the ISO spec:-() that are broken.
regards, tom lane
On Tue, 2002-05-21 at 21:31, Trond Eivind Glomsrød wrote: > On Tue, 21 May 2002, Lamar Owen wrote: > > > On Tuesday 21 May 2002 11:04 am, Manuel Sugawara wrote: > > > I see. This behavior is consistent with the fact that mktime is > > > supposed to return -1 on error, but then is broken in every other Unix > > > implementation that I know. > > > > > Any other workaround than downgrade or install FreeBSD? > > > > Complain to Red Hat. Loudly. However, as this is a glibc change, other > > distributors are very likely to fold in this change sooner rather than > > later. > > Relying on nonstandardized/nondocumented behaviour is a program bug, not a > glibc bug. PostgreSQL needs fixing. Since we ship both, we're looking at > it, but glibc is not the component with a problem. Still it seems kind of silly to have a function that works different from all other implementations and forces people to use their own function of the same name (lifted from BSD and also compliant). Speaking of nonstandardized/nondocumented behaviour, I read from "The Open Sources" book that if you implement TCP/IP strictly from the RFCs then it won't interoperate with any other TCP/IP stack. I hope that Red Hat is not going to be "standards compliant" here ;) -------------- Hannu
On Tue, 2002-05-21 at 18:24, Lamar Owen wrote: > In any case, this isn't just a Red Hat problem, as it's going to cause > problems with the use of timestamps on ANY glibc 2.2.5 dist. That's more > than Red Hat, by a large margin. I'm running glibc 2.2.5 on Debian and all regression tests pass OK (with make check). I don't see any note in the glibc Debian changelog about reversing an upstream change to mktime(). I missed the first messages in this thread and I can't find them in the archive. What should I be looking for to see if I have the problem you have encountered or to see why I don't have it if I ought to have? -- Oliver Elphick Oliver.Elphick@lfix.co.uk Isle of Wight http://www.lfix.co.uk/oliver GPG: 1024D/3E1D0C1C: CA12 09E0 E8D5 8870 5839 932A 614D 4C34 3E1D 0C1C "O come, let us worship and bow down; let us kneel before the LORD our maker." Psalms 95:6
On Tuesday 21 May 2002 06:09 pm, Oliver Elphick wrote: > On Tue, 2002-05-21 at 18:24, Lamar Owen wrote: > > In any case, this isn't just a Red Hat problem, as it's going to cause > > problems with the use of timestamps on ANY glibc 2.2.5 dist. That's more > > than Red Hat, by a large margin. > I'm running glibc 2.2.5 on Debian and all regression tests pass OK (with > make check). I don't see any note in the glibc Debian changelog about > reversing an upstream change to mktime(). > I missed the first messages in this thread and I can't find them in the > archive. What should I be looking for to see if I have the problem you > have encountered or to see why I don't have it if I ought to have? Hmmm. Compile and run the attached program. If you get -1, it's the new behavior. It might be interesting to see the differences here..... -- Lamar Owen WGCR Internet Radio 1 Peter 4:11
Manuel Sugawara <masm@fciencias.unam.mx> writes:
> +#if 0
> /* Only years after 1970 are defined.
> If year is 69, it might still be representable due to
> timezone differnces. */
> if (year < 69)
> return -1;
> +#endif
Hm. If that fixes it, it implies that all the other support for
pre-1970 dates is still there (notably, entries in the timezone tables).
Should we assume that future glibc releases will rip out the timezone
database entries and other support for pre-1970 dates? Or is the
breakage going to stop with mktime?
regards, tom lane
Lamar Owen writes: > SuSE already does this. I wonder how they've handled this issue with > 8.0? Their glibc doesn't have that problem. Personally, I think if you need time (zone) support before 1970, obtain one of the various operating systems that support it. There's little value in hacking around it in PostgreSQL, since the rest of your system will be broken as well. -- Peter Eisentraut peter_e@gmx.net
> > SuSE already does this. I wonder how they've handled this issue with
> > 8.0?
> Their glibc doesn't have that problem.
My strong recollection is that a SuSE guy was the one applying the
change. So this is coming to those systems too. I may not remember that
correctly though...
> Personally, I think if you need time (zone) support before 1970, obtain
> one of the various operating systems that support it. There's little
> value in hacking around it in PostgreSQL, since the rest of your system
> will be broken as well.
Yes, I'm afraid I agree. In practice, maybe most applications won't
notice. But after getting the Linux time zone databases set up to be
better than most (Solaris has the best I've found for fidelity to
pre-1970 year-to-year conventions) throwing that work away is just plain
silly. I consider this a major gaff on the part of the commercial Linux
houses to not see this coming and to contribute to a better solution.
- Thomas
On Tue, 2002-05-21 at 23:47, Lamar Owen wrote: > Hmmm. Compile and run the attached program. If you get -1, it's the new > behavior. It might be interesting to see the differences here..... $ a.out The system thinks 11/30/1969 is a timestamp of -176400 $ dpkg -l libc6 ... ||/ Name Version Description +++-==============-==============-============================================ ii libc6 2.2.5-6 GNU C Library: Shared libraries and Timezone -- Oliver Elphick Oliver.Elphick@lfix.co.uk Isle of Wight http://www.lfix.co.uk/oliver GPG: 1024D/3E1D0C1C: CA12 09E0 E8D5 8870 5839 932A 614D 4C34 3E1D 0C1C "We are troubled on every side, yet not distressed; we are perplexed, but not in despair; persecuted, but not forsaken; cast down, but not destroyed; Always bearing about in the body the dying of the Lord Jesus, that thelife also of Jesus might be made manifest in our body." II Corinthians 4:8-10
On Wed, 2002-05-22 at 02:14, Tom Lane wrote: > Trond Eivind Glomsrød <teg@redhat.com> writes: > > Relying on nonstandardized/nondocumented behaviour is a program bug, not a > > glibc bug. PostgreSQL needs fixing. Since we ship both, we're looking at > > it, but glibc is not the component with a problem. > > A library that can no longer cope with dates before 1970 is NOT my idea > of a component without a problem. We will be looking at ways to get > around glibc's breakage at the application level, since we have little > alternative other than to declare Linux an unsupported platform; > but it's still glibc (and the ISO spec:-() that are broken. IIRC the spec is not _really_ broken - it still allows the correct behaviour :) The fact the ISO spec is broken usually means that at least one of the big vendors involved in ISO spec creation must have had a broken implementation at that time. Most likely they have fixed it by now ... Does anyone know _any_ other libc that has this behaviour ? -------------- Hannu
> IIRC the spec is not _really_ broken - it still allows the correct
> behaviour :)
Yes.
> The fact the ISO spec is broken usually means that at least one of the
> big vendors involved in ISO spec creation must have had a broken
> implementation at that time.
Right. IBM.
> Most likely they have fixed it by now ...
Nope, though I don't know for sure. Anyone here have a recent AIX
machine to test?
> Does anyone know _any_ other libc that has this behaviour ?
AIX and (I think) Irix.
Trond, do you have a suggestion on how to get this addressed at the
glibc level? Does someone within RH participate in glibc development? If
so, can we get them to champion changes which would comply with the
standard but remove this arbitrary breakage?
The changes would involve returning -1 from mktime() for dates before
1970, and using the tm_isdst flag to indicate whether a time zone
translation was not possible.
- Thomas
On Wed, 2002-05-22 at 15:30, Thomas Lockhart wrote: > > IIRC the spec is not _really_ broken - it still allows the correct > > behaviour :) > > Yes. > > > The fact the ISO spec is broken usually means that at least one of the > > big vendors involved in ISO spec creation must have had a broken > > implementation at that time. > > Right. IBM. > > > Most likely they have fixed it by now ... > > Nope, though I don't know for sure. Anyone here have a recent AIX > machine to test? > > > Does anyone know _any_ other libc that has this behaviour ? > > AIX and (I think) Irix. How do we currently support AIX/Irix ? ---------------- Hannu
Thomas Lockhart writes:> Right. IBM.> > Most likely they have fixed it by now ...> Nope, though I don't know for sure. Anyonehere have a recent AIX> machine to test? Well on AIX 4.3.3 the output from Lamar's earlier test program is: The system thinks 11/30/1969 is a timestamp of -1 and tm_isdst is left at -1... I could boot the machine into 5.0 too, but going by the AIX 5L manpages it still returns -1: Note: The mktime subroutine cannot convert time values before00:00:00 UTC, January 1, 1970 and after 03:14:07 UTC, January19,2038. And getting an Irix 5.3 box up and running would be a chore! Lee.
...
> > AIX and (I think) Irix.
> How do we currently support AIX/Irix ?
Dates and times prior to 1970 have no time zone (they are in GMT, as is
the case for all platforms on dates before 1903 and after 2038). We have
separate regression test results for those platforms.
- Tom
On Wed, 22 May 2002, Thomas Lockhart wrote: > > IIRC the spec is not _really_ broken - it still allows the correct > > behaviour :) > > Yes. > > > The fact the ISO spec is broken usually means that at least one of the > > big vendors involved in ISO spec creation must have had a broken > > implementation at that time. > > Right. IBM. > > > Most likely they have fixed it by now ... > > Nope, though I don't know for sure. Anyone here have a recent AIX > machine to test? > > > Does anyone know _any_ other libc that has this behaviour ? > > AIX and (I think) Irix. > > Trond, do you have a suggestion on how to get this addressed at the > glibc level? Does someone within RH participate in glibc development? Jakub Jelinek, Ulrich Drepper and others. > If so, can we get them to champion changes which would comply with the > standard but remove this arbitrary breakage? Unlikely. They already saw (and participated, at least Ulrich) a thread on this with Lamar. Their take is "this is the standard, you should do what the standard says and not rely on undocumented, non-standardized sideeffects. > The changes would involve returning -1 from mktime() for dates before > 1970, and using the tm_isdst flag to indicate whether a time zone > translation was not possible. -- Trond Eivind Glomsrød Red Hat, Inc.
On Wednesday 22 May 2002 01:12 pm, Ulrich Drepper wrote: > On Wed, 2002-05-22 at 08:00, Thomas Lockhart wrote: > > > > If so, can we get them to champion changes which would comply with > > > > the standard but remove this arbitrary breakage? > > > Unlikely. They already saw (and participated, at least Ulrich) a thread > > > on this with Lamar. Their take is "this is the standard, you should do > > > what the standard says and not rely on undocumented, non-standardized > > > sideeffects. > > OK. They must be new guys. > :-) Very funny. What isn't funny is Oliver Elphick's results on Debian, running glibc 2.2.5 (same as Red Hat 7.3's version). They are different. And, IMO, those results are the 'expected' results on a unixoid system, ISO or no ISO. -- Lamar Owen WGCR Internet Radio 1 Peter 4:11
Ulrich Drepper <drepper@redhat.com> writes:
> If you are misusing interfaces you get what you deserve. At no time was
> it correct to use these functions for general date manipulation. It
> always only was allowed to use them to represent system times and there
> was no Unix system before the epoch. Therefore you argumentation is
> completely wrong.
> If you need date manipulation write your own code which work for all the
> times you want to represent.
We may indeed end up doing that, if glibc fails to provide the
functionality we need; but your argument is nonsense. Unix systems have
*always* interpreted time_t as a signed offset from the epoch. Do you
really think that when Unixen were first built in the early 70s, there
was no interest in working with pre-1970 dates? Hardly likely.
glibc has just taken a large step backwards by comparison to every other
libc on the planet. Claiming that you are okay because you conform to
a lowest-common-denominator ISO spec is beside the point. ISO specs are
only pieces of paper. You have removed functionality that is de facto
standard, important in practice, and *provided by most of your
competition*. People will start going to the competition. Or perhaps
there will be a glibc fork.
Postgres is not the only application that will be complaining loudly
about this change; anyone who has to deal with historical information
is going to be unhappy. We just happen to be the earliest bearers of
the bad news. But you will end up reverting this change due to pushback
from users. Want to make a side bet?
regards, tom lane
On Wednesday 22 May 2002 01:58 pm, Ulrich Drepper wrote: > On Wed, 2002-05-22 at 10:51, Lamar Owen wrote: > > What isn't funny is Oliver Elphick's results on Debian, running glibc > > 2.2.5 (same as Red Hat 7.3's version). > This is a completely different version. Once Debian updates (in a few > years) they'll get the same result. A completely different version with the same version number? Or is this a case of a Red Hat version number really meaning something different Shouldn't glibc 2.2.5 be the same as glibc 2.2.5 regardless of distribution? And who's to stop them patching out the new stuff and reinstating the old behavior? :-) > If you are misusing interfaces you get what you deserve. At no time was > it correct to use these functions for general date manipulation. It > always only was allowed to use them to represent system times and there > was no Unix system before the epoch. Therefore you argumentation is > completely wrong. If it is completely wrong, then tell Sun, HP, and all the rest of the Unix vendors, including the authors of the original AT&T code as lifted by Berkeley, that they're wrong and you're right. They'll laugh you to scorn. And just which 'major Unixen' other than AIX and Irix that follow the letter of the 'seconds since the epoch' definition of the ISO standard? > If you need date manipulation write your own code which work for all the > times you want to represent. The mktime bug doesn't effect our representation of dates and times other than the timezone at this time. What's aggravating to me is the surprise factor. -- Lamar Owen WGCR Internet Radio 1 Peter 4:11
This thread is getting pretty annoying rather than constructive. By the mean time I can see the users of many db's running under linux loudly complaining. As a user of both products (glibc and postgres), I would like to see a good compromise in both sides. For instance: postgreSQL will implement it's own time engine in one or to releases (In a few months I can do it if no one else volunteers) and glibc will revert this change for one or two releases (I can do it right now). That way everyone will be happy; particularly me and many others using glibc and postgres. Regards, Manuel.
> > OK. They must be new guys.
> :-) Very funny.
Hey, it worked. Got you out of the woodwork. :))
- Thomas
> On Wed, 2002-05-22 at 15:30, Thomas Lockhart wrote: > > > IIRC the spec is not _really_ broken - it still allows the correct > > > behaviour :) > > > > Yes. > > > > > The fact the ISO spec is broken usually means that at least one of the > > > big vendors involved in ISO spec creation must have had a broken > > > implementation at that time. > > > > Right. IBM. > > > > > Most likely they have fixed it by now ... > > > > Nope, though I don't know for sure. Anyone here have a recent AIX > > machine to test? > > > > > Does anyone know _any_ other libc that has this behaviour ? > > > > AIX and (I think) Irix. > > How do we currently support AIX/Irix ? Why should we rely on broken glibc and the standard? Why don't we make our own mktime() and use it on all platforms. -- Tatsuo Ishii
...
> Why should we rely on broken glibc and the standard? Why don't we make
> our own mktime() and use it on all platforms.
The downside to doing that is that we then take over maintenance of the
code and, more importantly, the timezone database.
But it might be the best thing to do. It looks like the zic package
pointed to the other day could be used, though we may have to change
some of the variables and entry points to avoid library conflicts. But
we could also blow past the y2038 limit afaict which would be nice.
- Thomas
Thomas Lockhart <lockhart@fourpalms.org> writes:
>> Why should we rely on broken glibc and the standard? Why don't we make
>> our own mktime() and use it on all platforms.
> The downside to doing that is that we then take over maintenance of the
> code and, more importantly, the timezone database.
> But it might be the best thing to do.
I've been sorta thinking the same thing. We could get out from under
the Y2038 issue, and also eliminate a whole lot of platform
dependencies. Not to mention sillinesses like being unable to recognize
a bad timezone name when it's fed to us.
Exactly how much work (and code bulk) would we be taking on? I've
never looked at how big the timezone databases are...
regards, tom lane
Tom Lane <tgl@sss.pgh.pa.us> wrote: [snip] > > Exactly how much work (and code bulk) would we be taking on? I've > never looked at how big the timezone databases are... > Some answers on database sizes, if this is any help... I did "du -sh /usr/share/zoneinfo/" on them all. OpenBSD 3.1, sparc64: 1.3M /usr/share/zoneinfo/ Linux, i686, oldish mandrake (6.x?), glibc 2.1.3: 478k /usr/share/zoneinfo Linux, i686, newish redhat 7.2, glibc 2.2.4: 4.9M /usr/share/zoneinfo Linux, alpha EV56, oldish redhat 6.2, glibc 2.1.3 1.4M /usr/share/zoneinfo > regards, tom lane Magnus -- -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-Programmer/Networker [|] Magnus Naeslund -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
On Thu, May 23, 2002 at 04:42:13AM +0200, Magnus Naeslund(f) wrote: > Some answers on database sizes, if this is any help... > I did "du -sh /usr/share/zoneinfo/" on them all. > > OpenBSD 3.1, sparc64: > 1.3M /usr/share/zoneinfo/ > > Linux, i686, oldish mandrake (6.x?), glibc 2.1.3: > 478k /usr/share/zoneinfo > > Linux, i686, newish redhat 7.2, glibc 2.2.4: > 4.9M /usr/share/zoneinfo ^^^^ What do they do with that di? > Linux, alpha EV56, oldish redhat 6.2, glibc 2.1.3 > 1.4M /usr/share/zoneinfo Here's what my Debian Woody system says: 1.5M /usr/share/zoneinfo And this is with glibc 2.2.5. Of course this wouldn't be the first time that RedHat uses the same version number for a different version. Michael -- Michael Meskes Michael@Fam-Meskes.De Go SF 49ers! Go Rhein Fire! Use Debian GNU/Linux! Use PostgreSQL!
On Wed, 2002-05-22 at 08:00, Thomas Lockhart wrote: > ... > > > If so, can we get them to champion changes which would comply with the > > > standard but remove this arbitrary breakage? > > Unlikely. They already saw (and participated, at least Ulrich) a thread on > > this with Lamar. Their take is "this is the standard, you should do what > > the standard says and not rely on undocumented, non-standardized sideeffects. > > OK. They must be new guys. :-) Very funny. -- ---------------. ,-. 1325 Chesapeake Terrace Ulrich Drepper \ ,-------------------' \ Sunnyvale, CA 94089 USA Red Hat `--' drepper at redhat.com `------------------------
On Wed, 2002-05-22 at 10:51, Lamar Owen wrote: > What isn't funny is Oliver Elphick's results on Debian, running glibc 2.2.5 > (same as Red Hat 7.3's version). This is a completely different version. Once Debian updates (in a few years) they'll get the same result. If you are misusing interfaces you get what you deserve. At no time was it correct to use these functions for general date manipulation. It always only was allowed to use them to represent system times and there was no Unix system before the epoch. Therefore you argumentation is completely wrong. If you need date manipulation write your own code which work for all the times you want to represent. -- ---------------. ,-. 1325 Chesapeake Terrace Ulrich Drepper \ ,-------------------' \ Sunnyvale, CA 94089 USA Red Hat `--' drepper at redhat.com `------------------------
On Wed, 2002-05-22 at 11:23, Tom Lane wrote: > Unix systems have > *always* interpreted time_t as a signed offset from the epoch. No. This always was an accident if it happens. > Do you > really think that when Unixen were first built in the early 70s, there > was no interest in working with pre-1970 dates? Hardly likely. There never were files or any system events with these dates. Yes. And just to educate you and your likes: the majority of systems on this planet use mktime this way. I hate using this as an argument, but beside major Unixes M$ systems also do this. > But you will end up reverting this change due to pushback > from users. Want to make a side bet? Sure. Especially not everybody is that stubborn. -- ---------------. ,-. 1325 Chesapeake Terrace Ulrich Drepper \ ,-------------------' \ Sunnyvale, CA 94089 USA Red Hat `--' drepper at redhat.com `------------------------
On Wed, May 22, 2002 at 10:58:15AM -0700, Ulrich Drepper wrote: > On Wed, 2002-05-22 at 10:51, Lamar Owen wrote: > > > What isn't funny is Oliver Elphick's results on Debian, running glibc 2.2.5 > > (same as Red Hat 7.3's version). > > This is a completely different version. Once Debian updates (in a few > years) they'll get the same result. Ulrich, how shall I understand this? I'm pretty sure Oliver does not use a Debian 2.2 system with glibc-2.1.3 but a pretty up-to-date one. The glibc version in the soon to be released Woody release is 2.2.5. This seems to be the very same version that RedHat uses. So what could/should Debian update? Besides, the "in a few years" comment looks like FUD to me. It may be a few years since we talked the last time, but I cannot imagine you changed that much that you spread FUD nowadays. So I probably misunderstood this sentence, but nevertheless would like to know what Debian should update. Or do you mean that once Debian updates to glibc 2.3 (or whatever the next release will be) it will show the same results? Does RedHat 7.3 already run on that new release? But then I would think they changed the version number. Michael -- Michael Meskes Michael@Fam-Meskes.De Go SF 49ers! Go Rhein Fire! Use Debian GNU/Linux! Use PostgreSQL!
On 22 May 2002, Ulrich Drepper wrote: > On Wed, 2002-05-22 at 10:51, Lamar Owen wrote: > > > What isn't funny is Oliver Elphick's results on Debian, running glibc 2.2.5 > > (same as Red Hat 7.3's version). > > This is a completely different version. Once Debian updates (in a few > years) they'll get the same result. > > If you are misusing interfaces you get what you deserve. At no time was > it correct to use these functions for general date manipulation. It > always only was allowed to use them to represent system times and there > was no Unix system before the epoch. Therefore you argumentation is > completely wrong. > > If you need date manipulation write your own code which work for all the > times you want to represent. We are Redhat, you will be assimilated
On 22 May 2002, Ulrich Drepper wrote: > On Wed, 2002-05-22 at 11:23, Tom Lane wrote: > > > Unix systems have > > *always* interpreted time_t as a signed offset from the epoch. > > No. This always was an accident if it happens. > > > Do you > > really think that when Unixen were first built in the early 70s, there > > was no interest in working with pre-1970 dates? Hardly likely. > > There never were files or any system events with these dates. Yes. > > And just to educate you and your likes: the majority of systems on this > planet use mktime this way. I hate using this as an argument, but > beside major Unixes M$ systems also do this. M$ systems crashes regularly too ... is Redhat going to adopt that too?
> --=-Z1lifK4QZqKV8kHqHfYT > Content-Type: text/plain > Content-Transfer-Encoding: quoted-printable > > On Wed, 2002-05-22 at 10:51, Lamar Owen wrote: > > > What isn't funny is Oliver Elphick's results on Debian, running glibc 2.2= > .5=20 > > (same as Red Hat 7.3's version). > > This is a completely different version. Once Debian updates (in a few > years) they'll get the same result. > > If you are misusing interfaces you get what you deserve. At no time was > it correct to use these functions for general date manipulation. It > always only was allowed to use them to represent system times and there > was no Unix system before the epoch. Therefore you argumentation is > completely wrong. > > If you need date manipulation write your own code which work for all the > times you want to represent. This is indeed a problem with dates on LIBC, because even if it is theoretically satisfactory to describe dates within some range within 2^31 seconds of 1970, that is certainly NOT satisfactory for describing all dates of interest unless you're being terribly parochial about what is to be considered "of interest." My father's birth falls within 2^31 seconds of 1970, but the Battle of Agincourt doesn't. Any backup of any Unix system in history falls within 2^31 seconds of 1970, but there are people alive whose births don't. People get away with using Unix dates as a "general" date type when they don't have to work outside a limited range. If/when there is a move to 64 bit values, that will provide something with enough range to cover history back to ridiculously early times, relieving the limit. But anybody using Unix dates as "general dates" has leaped into exactly the same sort of trap that caused people to get so paranoid about Y2K. -- (concatenate 'string "cbbrowne" "@acm.org") http://www.cbbrowne.com/info/oses.html Do Roman paramedics refer to IV's as "4's"? -- (concatenate 'string "aa454" "@freenet.carleton.ca") http://www.ntlug.org/~cbbrowne/linuxxian.html "So, when you typed in the date, it exploded into a sheet of blue flame and burned the entire admin wing to the ground? Yes, that's a known bug. We'll be fixing it in the next release. Until then, try not to use European date format, and keep an extinguisher handy." -- slam@pobox.com (Tequila Rapide)
> On 22 May 2002, Ulrich Drepper wrote: > > > On Wed, 2002-05-22 at 11:23, Tom Lane wrote: > > > > > Unix systems have > > > *always* interpreted time_t as a signed offset from the epoch. > > > > No. This always was an accident if it happens. > > > > > Do you > > > really think that when Unixen were first built in the early 70s, there > > > was no interest in working with pre-1970 dates? Hardly likely. > > > > There never were files or any system events with these dates. Yes. > > > > And just to educate you and your likes: the majority of systems on this > > planet use mktime this way. I hate using this as an argument, but > > beside major Unixes M$ systems also do this. > > M$ systems crashes regularly too ... is Redhat going to adopt that too? Harbison and Steele indicates that: "Although the traditional return type of time is long, the value returned is usually of type unsigned long." That is NOT a "Linux" reference; it was published before Linus had started working on his kernel project. ANSI does not indicate that time_t is a signed int, signed long, or whatever. It is only defined to be an arithmetic type. It would not be a bug for GLIBC to define time_t to be an unsigned int, with an epoch beginning of January 1, 1999. That definition would conform to ANSI C. Since that definition can conform to ANSI C, and Unix systems have more particularly been known to use unsigned ints with epoch of 1970, it is NOT reasonable to assume that time_t can be used to express dates that are at all ancient in the past. Indeed, it is fairly _useful_ for different libc implementations to make different assumptions about things whose definitions are permitted to vary, as this makes it easier to call out as WRONG those systems that make up their own definitions. People will no doubt get defensive about their own non-standard implementations of things; it is certainly far easier to cry "They're trying to play Microsoft!" than it is to be honest and actually look at the standards. -- (concatenate 'string "cbbrowne" "@ntlug.org") http://www.cbbrowne.com/info/advocacy.html When confronted by a difficult problem, solve it by reducing it to the question, "How would the Lone Ranger handle this?" -- (reverse (concatenate 'string "gro.mca@" "enworbbc")) http://www.cbbrowne.com/info/linuxxian.html As of next Monday, COMSAT will be flushed in favor of a string and two tin cans. Please update your software.
On Thu, 23 May 2002 cbbrowne@cbbrowne.com wrote: > > On 22 May 2002, Ulrich Drepper wrote: > > > > > On Wed, 2002-05-22 at 11:23, Tom Lane wrote: > > > > > > > Unix systems have > > > > *always* interpreted time_t as a signed offset from the epoch. > > > > > > No. This always was an accident if it happens. > > > > > > > Do you > > > > really think that when Unixen were first built in the early 70s, there > > > > was no interest in working with pre-1970 dates? Hardly likely. > > > > > > There never were files or any system events with these dates. Yes. > > > > > > And just to educate you and your likes: the majority of systems on this > > > planet use mktime this way. I hate using this as an argument, but > > > beside major Unixes M$ systems also do this. > > > > M$ systems crashes regularly too ... is Redhat going to adopt that too? < stuff deleted > > People will no doubt get defensive about their own non-standard > implementations of things; it is certainly far easier to cry "They're trying > to play Microsoft!" than it is to be honest and actually look at the standards. Just to clarify, if this was directed at my comment, I wasn't the one that brought up the fact that "Redhat is trying to play Microsoft", Ulrich was the one that brought it into the argument ... I was just curious as to how far they planned on getting to what M$ systems do ...
On Thu, 2002-05-23 at 15:20, Michael Meskes wrote: > On Wed, May 22, 2002 at 10:58:15AM -0700, Ulrich Drepper wrote: > > On Wed, 2002-05-22 at 10:51, Lamar Owen wrote: > > > > > What isn't funny is Oliver Elphick's results on Debian, running glibc 2.2.5 > > > (same as Red Hat 7.3's version). > > > > This is a completely different version. Once Debian updates (in a few > > years) they'll get the same result. > > Ulrich, how shall I understand this? I'm pretty sure Oliver > does not use a Debian 2.2 system with glibc-2.1.3 but a pretty > up-to-date one. The glibc version in the soon to be released Woody > release is 2.2.5. In fact, I run "unstable" and the version is 2.2.5-6. I couldn't see any reference in the Debian changelog, but I'm wondering if the change was commented out. I haven't had the time to download the source and go looking. -- Oliver Elphick Oliver.Elphick@lfix.co.uk Isle of Wight http://www.lfix.co.uk/oliver GPG: 1024D/3E1D0C1C: CA12 09E0 E8D5 8870 5839 932A 614D 4C34 3E1D 0C1C "I will praise thee; for I am fearfully and wonderfully made..." Psalms 139:14
On Thu, May 23, 2002 at 09:29:06AM -0700, Ulrich Drepper wrote: > On Thu, 2002-05-23 at 07:20, Michael Meskes wrote: > > > The glibc version in the soon to be released Woody > > release is 2.2.5. > > The version in RHL7.3 is 2.2.5-34. This is not what Debian uses. Maybe > you should read the changelog for the version. So with your arithmetics 2.2.5 != 2.2.5. Hey that's great. How about naming the next Linux kernel release 2.0 just to confuse some people? :-) Seriously though, while this is offtopic on this list, I wonder what's going on. If I have a system with glibc 2.2.5, I cannot expect this to be the same version as on another system with glibc 2.2.5. Is this the correct understanding? And which changelog should I read? The RedHat one, the Debian one or the glibc one? Or does the -34 mean more than just the RedHat version number? The Debian version is correctly named 2.2.5-6 where the -6 means that this is the 6th release of glibc 2.2.5 for Debian, Michael -- Michael Meskes Michael@Fam-Meskes.De Go SF 49ers! Go Rhein Fire! Use Debian GNU/Linux! Use PostgreSQL!
...
> But anybody using Unix dates as "general dates" has leaped into exactly the
> same sort of trap that caused people to get so paranoid about Y2K.
Certainly true. We don't use Unix dates as "general dates", we use the
Unix time zone database and API for dates and times within the year
range of 1903 to 2038. Well, up until now anyway...
Prior to the 1900s, the concept of time zones was more localized and not
universally adopted. In the US, a first round of time zone
standardization came with the transcontinental railroads in the 1860s.
After 2038, it is a good bet that time zones will resemble those in use
today, but they are as much a political construct as a physical one so
the details are subject to change.
- Thomas
On Fri, 24 May 2002 06:10:39 PDT, the world broke into rejoicing as Thomas Lockhart <lockhart@fourpalms.org> said: > ... > > But anybody using Unix dates as "general dates" has leaped into exactly the > > same sort of trap that caused people to get so paranoid about Y2K. > Certainly true. We don't use Unix dates as "general dates", we use the > Unix time zone database and API for dates and times within the year > range of 1903 to 2038. Well, up until now anyway... I don't think going past 1970 is particularly safe; it certainly doesn't seem to fit with ANSI... By the way, the seemingly relevant link to look at for TZ info is http://www.twinsun.com/tz/tz-link.htm, linking to the data used by various Unix implementations. > Prior to the 1900s, the concept of time zones was more localized and > not universally adopted. In the US, a first round of time zone > standardization came with the transcontinental railroads in the 1860s. > After 2038, it is a good bet that time zones will resemble those in > use today, but they are as much a political construct as a physical > one so the details are subject to change. Some of the zones are quite peculiar if you head to Africa and Asia; there are some sitting on 15 minute intervales, rather than the usual 1h intervals. (The classic Canadian timezone joke is "World ends at 9:00; 9:30 in Newfoundland". For more information, see TZ='America/St_Johns') -- (concatenate 'string "chris" "@cbbrowne.com") http://www.ntlug.org/~cbbrowne/spreadsheets.html "Heuristics (from the French heure, "hour") limit the amount of time spent executing something. [When using heuristics] it shouldn't take longer than an hour to do something."
cbbrowne@cbbrowne.com writes: > By the way, the seemingly relevant link to look at for TZ info is > http://www.twinsun.com/tz/tz-link.htm, linking to the data used by > various Unix implementations. Oh, this is interesting: it claims that : This database (often called tz or zoneinfo) is used by several : implementations, including GNU/Linux, FreeBSD, NetBSD, OpenBSD, DJGPP, : HP-UX, IRIX, Open UNIX, Solaris, and Tru64. The actual timezone database seems to consist of about half a meg of heavily commented text data files. The accompanying code (probably providing far more functionality than we actually need) is under 400k. (Both figures are for uncompressed source.) Not large at all. I cannot find any sign of a copyright or license in the files; I think it is intended to be public domain, and in any case if the *BSDs are using it then it must have a BSD-compatible license. It seems to me that it'd be really practical to just take what we need out of this distribution and forgo all dependency on system-provided timezone databases. And, since there's a mailing list maintaining it, we could expect someone else to handle updates ;-) ... we'd just have to be careful to use the database files unmodified, so that we could drop in new releases from time to time. Comments? Anyone want to do the legwork? regards, tom lane
Tom Lane wrote:> It seems to me that it'd be really practical to just take what we> need out of this distribution and forgoall dependency on> system-provided timezone databases. And, since there's a mailing> list maintaining it, we couldexpect someone else to handle updates> ;-) ... we'd just have to be careful to use the database files> unmodified, sothat we could drop in new releases from time to time.>> Comments? Anyone want to do the legwork?> I don't understand precisely what need to be done, but I'll give it a shot if you get me pointed in the right direction. <downloads and looks at code> I see that tzcode2002c.tar.gz includes a mktime() function. Is the idea to pull this out (with just whatever support it needs), incorporate it into PostgreSQL source (perhaps in a new src/backend/utils/tz directory) and use this in place of the system provided mktime()? Joe
Joe Conway <mail@joeconway.com> writes:
> I don't understand precisely what need to be done, but I'll give it a
> shot if you get me pointed in the right direction.
> <downloads and looks at code>
> I see that tzcode2002c.tar.gz includes a mktime() function. Is the idea
> to pull this out (with just whatever support it needs), incorporate it
> into PostgreSQL source (perhaps in a new src/backend/utils/tz directory)
> and use this in place of the system provided mktime()?
Well, that's the zeroth-order approximation. We should take the
opportunity to get out from under the mktime()/tzset() API. The real
idea here is to make use of the timezone database info in the ways that
Postgres needs. Some things that are not good about mktime()/tzset():
* Arbitrary restrictions on range of dates. We certainly don't want to
be limited by a 32-bit time_t, whether you think it's signed or not.
The APIs should be recast in terms of PG's preferred internal
representations. (Lockhart would be the man to point you in the right
direction here, not me.)
* No way to tell whether a user-provided timezone name is actually good.
* No support for concurrent access to multiple zones, short of flushing
all memory of one zone to load the next. Although we do not really need
this now, I can foresee wanting it. I'd be inclined to store all the
info about a particular zone in some struct that can be referenced by
a pointer; that would give us the flexibility to have multiple such
structs floating around a backend in the future (perhaps living in a
hashtable indexed by timezone name?)
My guess is that we want to borrow the parts of the tzcode library that
are associated with reading a tz database file and loading it into some
suitable internal representation; there's probably not a lot else that
we'll want to use as-is.
regards, tom lane
Michael Meskes writes: > Or does the -34 mean more than just the RedHat version number? The > Debian version is correctly named 2.2.5-6 where the -6 means that this > is the 6th release of glibc 2.2.5 for Debian, Just for general amusement: I run SuSE's glibc 2.2.5-38 which contains neither the questionable code in the original sources nor is there any reference to it in the patch set. Go figure. -- Peter Eisentraut peter_e@gmx.net
Tom Lane wrote: > Well, that's the zeroth-order approximation. We should take the > opportunity to get out from under the mktime()/tzset() API. The real > idea here is to make use of the timezone database info in the ways that > Postgres needs. Some things that are not good about mktime()/tzset(): > > * Arbitrary restrictions on range of dates. We certainly don't want to > be limited by a 32-bit time_t, whether you think it's signed or not. > The APIs should be recast in terms of PG's preferred internal > representations. (Lockhart would be the man to point you in the right > direction here, not me.) > > * No way to tell whether a user-provided timezone name is actually good. > > * No support for concurrent access to multiple zones, short of flushing > all memory of one zone to load the next. Although we do not really need > this now, I can foresee wanting it. I'd be inclined to store all the > info about a particular zone in some struct that can be referenced by > a pointer; that would give us the flexibility to have multiple such > structs floating around a backend in the future (perhaps living in a > hashtable indexed by timezone name?) > > My guess is that we want to borrow the parts of the tzcode library that > are associated with reading a tz database file and loading it into some > suitable internal representation; there's probably not a lot else that > we'll want to use as-is. Well, this does sound a bit more involved than I was envisioning. There are a few items wrt SRFs that I should finish first, but I'll come back to this afterward if no one else does first. Joe
...
> Well, this does sound a bit more involved than I was envisioning. There
> are a few items wrt SRFs that I should finish first, but I'll come back
> to this afterward if no one else does first.
The first cut might be to reproduce the functionality we already have.
That would allow us to (optionally) use this internal code *or* the
system-provided code with a configure-time switch. We could then strip
out two of the three time zone interface styles we support. And we could
(possibly) allow folks to use their built-in time zone databases if they
want, to minimize inconsistancies between PostgreSQL and other programs
on the system. We might need to modify function and variable signatures
to avoid conflicts with system-supplied libraries.
The next step would be to see how to generalize this past Y2038 (as
mentioned previously, time zone info for pre-1900 is not likely to be
interesting). If it involves mass substitution of time_t for, say,
pg_time_t, then that might be all we need for the second phase, at which
time we've broken the y2038 limit by moving to 64-bit integer time.
The last phase could be extending the API to allow multiple simultaneous
time zones, detection of bad time zones, etc etc. This would involve API
changes or extensions, and breaks compatibility with system-supplied
infrastructure.
- Thomas
Thomas Lockhart <lockhart@fourpalms.org> writes:
> The last phase could be extending the API to allow multiple simultaneous
> time zones, detection of bad time zones, etc etc. This would involve API
> changes or extensions, and breaks compatibility with system-supplied
> infrastructure.
One thing that wasn't clear to me, but could use investigation: if so
many systems are using the same underlying timezone database info, maybe
there is some commonality at a level below the ISO mktime/tzset/etc API.
If we could make use of the system-provided TZ database at a lower level
while still using our own APIs not tied to time_t, it'd answer the issue
of compatibility with the surrounding system. (Which is a real issue,
I agree --- we should be able to accept the system's standard TZ setting
if possible.)
regards, tom lane
> > The last phase could be extending the API to allow multiple simultaneous
> > time zones, detection of bad time zones, etc etc. This would involve API
> > changes or extensions, and breaks compatibility with system-supplied
> > infrastructure.
> One thing that wasn't clear to me, but could use investigation: if so
> many systems are using the same underlying timezone database info, maybe
> there is some commonality at a level below the ISO mktime/tzset/etc API.
> If we could make use of the system-provided TZ database at a lower level
> while still using our own APIs not tied to time_t, it'd answer the issue
> of compatibility with the surrounding system. (Which is a real issue,
> I agree --- we should be able to accept the system's standard TZ setting
> if possible.)
The fundamental problem (which of course can have a fundamental solution
;) is that a time zone database built with a 32-bit time_t will have
time zone info through 2038 only (it is a binary file with 32-bit time
fields -- almost certainly anyway). So if we have an extended time zone
infrastructure using something different for time_t we would need to
handle the case of reading non-extended time zones databases, which puts
us back to having limitations.
I'm guessing that a better approach might be to have our time zone stuff
inside our own API, which then could choose to call, for example,
mktime() or pg_mktime(), which could each have different signatures.
Then the heuristics for matching one to the other are isolated to our
thin API implementation, not to the underlying system- or pg-provided
libraries.
afaik there is no API provision for the "inverse time zone" problem of
matching "stringy time zones" to numeric offsets for input date/times.
The time zone databases themselves don't lend themselves to this, since
the tables have those stringy zones somewhere on the right hand side of
each row of information and the fields can change from year to year.
- Thomas
Thomas Lockhart <lockhart@fourpalms.org> writes:
> The fundamental problem (which of course can have a fundamental solution
> ;) is that a time zone database built with a 32-bit time_t will have
> time zone info through 2038 only (it is a binary file with 32-bit time
> fields -- almost certainly anyway).
I'm not sure that the time zone database tables store time_t's at all.
They certainly could be coded not to use 'em; but I do not know exactly
how various vendors have chosen to represent the data.
A random extract from the tzdata2002c files looks like:
Zone America/Chicago -5:50:36 - LMT 1883 Nov 18 12:00 -6:00 US C%sT 1920 -6:00
Chicago C%sT 1936 Mar 1 2:00 -5:00 - EST 1936 Nov 15 2:00 -6:00 Chicago C%sT 1942
-6:00 US C%sT 1946 -6:00 Chicago C%sT 1967 -6:00 US C%sT
which might well be represented with separate y/m/d/h/m fields...
certainly we'd choose some such thing if we have to implement it
ourselves.
regards, tom lane
> > > The last phase could be extending the API to allow multiple simultaneous > > > time zones, detection of bad time zones, etc etc. This would involve API > > > changes or extensions, and breaks compatibility with system-supplied > > > infrastructure. > > One thing that wasn't clear to me, but could use investigation: if so > > many systems are using the same underlying timezone database info, maybe > > there is some commonality at a level below the ISO mktime/tzset/etc API. > > If we could make use of the system-provided TZ database at a lower level > > while still using our own APIs not tied to time_t, it'd answer the issue > > of compatibility with the surrounding system. (Which is a real issue, > > I agree --- we should be able to accept the system's standard TZ setting > > if possible.) > The fundamental problem (which of course can have a fundamental > solution ;) is that a time zone database built with a 32-bit time_t > will have time zone info through 2038 only (it is a binary file with > 32-bit time fields -- almost certainly anyway). So if we have an > extended time zone infrastructure using something different for time_t > we would need to handle the case of reading non-extended time zones > databases, which puts us back to having limitations. Ah, but the database in question _doesn't_ consist of 32 bit time_t values. It consists of things like: # @(#)zone.tab 1.26 # # TZ zone descriptions # # From Paul Eggert <eggert@twinsun.com> (1996-08-05): # # This file contains a table with the following columns: # 1. ISO 3166 2-character country code. See the file `iso3166.tab'. # 2. Latitude and longitude of the zone's principal location # in ISO 6709 sign-degrees-minutes-seconds format, # either +-DDMM+-DDDMM or +-DDMMSS+-DDDMMSS, # first latitude (+ is north), then longitude (+ is east). # 3. Zone name used in value of TZ environment variable. # 4. Comments; present if and only if the country has multiple rows. # # Columns are separated by a single tab. # The table is sorted first by country, then an order within the country that # (1) makes some geographical sense, and # (2) puts the most populous zones first, where that does not contradict (1). # # Lines beginning with `#' are comments. # #country- #code coordinates TZ comments AD +4230+00131 Europe/Andorra AE +2518+05518 Asia/Dubai AF +3431+06912 Asia/Kabul AG +1703-06148 America/Antigua AI +1812-06304 America/Anguilla AL +4120+01950 Europe/Tirane AM +4011+04430 Asia/Yerevan AN +1211-06900 America/Curacao AO -0848+01314 Africa/Luanda Then a "leapseconds" table, looking like: # The correction (+ or -) is made at the given time, so lines # will typically look like: # Leap YEAR MON DAY 23:59:60 + R/S # or # Leap YEAR MON DAY 23:59:59 - R/S # If the leapsecond is Rolling (R) the given time is local time # If the leapsecond is Stationary (S) the given time is UTC # Leap YEAR MONTH DAY HH:MM:SS CORR R/S Leap 1972 Jun 30 23:59:60 + S Leap 1972 Dec 31 23:59:60 + S Leap 1973 Dec 31 23:59:60 + S Leap 1974 Dec 31 23:59:60 + S Leap 1975 Dec 31 23:59:60 + S Leap 1976 Dec 31 23:59:60 + S And then a set of rules about timezone adjustments for all sorts of localities, including the following: # Rule NAME FROM TO TYPE IN ON AT SAVE LETTER/S # Summer Time Act, 1916 Rule GB-Eire 1916 only - May 21 2:00s 1:00 BST Rule GB-Eire 1916 only - Oct 1 2:00s 0 GMT # S.R.&O. 1917, No. 358 Rule GB-Eire 1917 only - Apr 8 2:00s 1:00 BST Rule GB-Eire 1917 only - Sep 17 2:00s 0 GMT # Zone NAME GMTOFF RULES FORMAT [UNTIL] Zone Antarctica/Casey 0 - zzz 1969 8:00 - WST # Western (Aus) Standard Time Zone Antarctica/Davis 0 - zzz 1957 Jan 13 7:00 - DAVT 1964 Nov # Davis Time 0 - zzz 1969 Feb 7:00 - DAVT Zone Antarctica/Mawson 0 - zzz 1954 Feb 13 6:00 - MAWT # Mawson Time > I'm guessing that a better approach might be to have our time zone > stuff inside our own API, which then could choose to call, for > example, mktime() or pg_mktime(), which could each have different > signatures. Then the heuristics for matching one to the other are > isolated to our thin API implementation, not to the underlying system- > or pg-provided libraries. > matching "stringy time zones" to numeric offsets for input date/times. > The time zone databases themselves don't lend themselves to this, > since the tables have those stringy zones somewhere on the right hand > side of each row of information and the fields can change from year to > year. The ultimate goal would seem likely to be to store dates internally in some form like UTC, with some reasonably huge dynamic range, that is, not limited to 32 bit timestamps, but rather using something like a proleptic Gregorian calendar (per _Calendrical Calculations_, page 50). Some reasonable treatments would include: - 32 bits is an signed int indicating number of days since GREG_EPOCH, where logical epochs would include January 1, 1,January 1, 1900, or perhaps even something actually proleptic (proleptic indicates "future"), such as January 1, 2038. - 8 bits indicating the month; 8 bits indicating the day of month; 16 bits providing a range of years from -32767 to 32768. Both have merits... Timestamps would then forcibly expand things by _at least_ 22 bits, the minimum needed to express 1/100ths of seconds. Might as well head on to 32 bits for the time and so have something that can easily represent values down to well below a millisecond. The "stringy stuff" indicates how values are to be displayed or parsed. It does nothing about what is stored internally, or at least shouldn't. -- (reverse (concatenate 'string "gro.gultn@" "enworbbc")) http://www.cbbrowne.com/info/emacs.html In the name of the Lord-High mutant, we sacrifice this suburban girl -- `Future Schlock'
On Fri, 24 May 2002, Peter Eisentraut wrote: > Michael Meskes writes: > > > Or does the -34 mean more than just the RedHat version number? The > > Debian version is correctly named 2.2.5-6 where the -6 means that this > > is the 6th release of glibc 2.2.5 for Debian, > > Just for general amusement: I run SuSE's glibc 2.2.5-38 which contains > neither the questionable code in the original sources nor is there any > reference to it in the patch set. Go figure. You've got to remember that you're talking about systems where, a long time ago now, certain groups felt it necessary to supply nonstandard versions of the core component (the kernel). Sure they helped development of the kernel but only through bastardisation of version numbers where 2.0.1 didn't really mean a Linux 2.0.1 kernel. Is it really surprising the system support stuff has been mangled beyond sense? Anyway, I've composed several and aborted all but this message on this subject and I'm not going to persue it. I have my own views on the right and wrongs off the change in glibc but they wouldn't have advanced anything so I'm keeping quiet on it, still. It seems there is a solution forming. Plus, I'd hate to side with the baddies from the first paragraph :) -- Nigel J. Andrews Director --- Logictree Systems Limited Computer Consultants
On Fri, May 24, 2002 at 12:15:47PM -0700, Ulrich Drepper wrote: > This is getting silly. Does nobody here understand that the release Yes, but I'm not sure on which side. > number is local for each distribution. Comparing them does not lead to No, this is simply not true. The version number is what the upstream gives its release. No more no less. What RH does is becoming as subtly incompatible a possible. If that's the goal, it doesn't look like free software for me. Sure all changes are published, but why forcing this kind of difference between linux distributions? Why not calling it 2.2.6 or something if there has to be some changes that are not compatible? Michael -- Michael Meskes Michael@Fam-Meskes.De Go SF 49ers! Go Rhein Fire! Use Debian GNU/Linux! Use PostgreSQL!
On Sat, 25 May 2002, Michael Meskes wrote: > No, this is simply not true. The version number is what the upstream > gives its release. No more no less. What RH does is becoming as subtly > incompatible a possible. If that's the goal, it doesn't look like free > software for me. Sure all changes are published, but why forcing this > kind of difference between linux distributions? Why not calling it 2.2.6 > or something if there has to be some changes that are not compatible? Or rlibc? :)
On Thu, 2002-05-23 at 07:20, Michael Meskes wrote: > The glibc version in the soon to be released Woody > release is 2.2.5. The version in RHL7.3 is 2.2.5-34. This is not what Debian uses. Maybe you should read the changelog for the version. -- ---------------. ,-. 1325 Chesapeake Terrace Ulrich Drepper \ ,-------------------' \ Sunnyvale, CA 94089 USA Red Hat `--' drepper at redhat.com `------------------------
On Fri, 2002-05-24 at 12:03, Peter Eisentraut wrote: > > Or does the -34 mean more than just the RedHat version number? The > > Debian version is correctly named 2.2.5-6 where the -6 means that this > > is the 6th release of glibc 2.2.5 for Debian, > > Just for general amusement: I run SuSE's glibc 2.2.5-38 which contains > neither the questionable code in the original sources nor is there any > reference to it in the patch set. Go figure. This is getting silly. Does nobody here understand that the release number is local for each distribution. Comparing them does not lead to anything. If you want to find out run rpm -q --changelog glibc | less on a RH system. Don't know what other systems provide in this direction. You'll see that the glibc in RHL7.3 contains a lot of the code from the glibc 2.3 branch. It's not named 2.2.90 because major pieces are missing. If you still don't know that version numbers are meaningless for determining feature lists you might want to consider going back to your CS101 class and revisit software configuration management. -- ---------------. ,-. 1325 Chesapeake Terrace Ulrich Drepper \ ,-------------------' \ Sunnyvale, CA 94089 USA Red Hat `--' drepper at redhat.com `------------------------
On Friday 24 May 2002 03:15 pm, Ulrich Drepper wrote: > This is getting silly. Yes, Ulrich, it is. Very silly. And Red Hat's stance is one of the silliest, IMHO. >You'll see that the glibc in RHL7.3 contains a lot of the > code from the glibc 2.3 branch. It's not named 2.2.90 because major > pieces are missing. > If you still don't know that version numbers are meaningless for > determining feature lists you might want to consider going back to your > CS101 class and revisit software configuration management. IOW, Red Hat's glibc 2.2.5 isn't really pristine glibc 2.2.5 as found straight from the GNU repository. In fact, Red Hat glibc 2.2.5 isn't really 2.2.5 -- how about 2.2.96? :-) .96 was good enough for gcc.... Furthermore, Red Hat glibc 2.2.5 isn't even fully compatible with GNU glibc 2.2.5 -- at least in the area of time_t stuff. In the open source world, version numbers are actually supposed to mean something -- at least for package dependencies. Of course, I also have read the kernel-2.4.18 source RPM and its 21.8MB 'ac-bits' patch. You do realize that this sort of thing doesn't help Red Hat's PR state amongst the greater open source community, right? Nor would it help Mandrake, SuSE, or any other Linux distributor (I specifically excluded Debian due to its unique community supported state). But, if you don't care about the greater open source community, well... And I say all of that while running and enjoying the greater part of Red Hat 7.3. For the most part it is extraordinarily stable. And I know that that 21.8MB kernel patch is one of the reasons it is so stable. But I still question the versioning of glibc. So, in summary, the glibc version number in any particular linux distribution is meaningless because the distributor is free to patch the bloody daylights out of it at any time. Sweet. And so standard. But, if glibc 2.3 is where this bit came from, it is just a matter of time before all Linux distributions (that aren't willing to patch away) get this braindead behavior. Oh well. The general solution will happen. -- Lamar Owen WGCR Internet Radio 1 Peter 4:11
Tom Lane wrote: > Thomas Lockhart <lockhart@fourpalms.org> writes: > >> Why should we rely on broken glibc and the standard? Why don't we make > >> our own mktime() and use it on all platforms. > > > The downside to doing that is that we then take over maintenance of the > > code and, more importantly, the timezone database. > > > But it might be the best thing to do. > > I've been sorta thinking the same thing. We could get out from under > the Y2038 issue, and also eliminate a whole lot of platform > dependencies. Not to mention sillinesses like being unable to recognize > a bad timezone name when it's fed to us. > > Exactly how much work (and code bulk) would we be taking on? I've > never looked at how big the timezone databases are... I am not really excited about distributing a timezone database as part of PostgreSQL, and it wouldn't match the OS's timezone. (We do need a 64-time time_t, but I think we can wait to get closer to 2038.) Can we detect if glibc is being used for the compile (easy), and substitute a non-broken mktime in the link path ahead of glibc's mktime? Seems that would be the easiest solution. Of course, pre-1970 dates then wouldn't match the OS on glibc systems, but that seems like a win. :-) -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 853-3000+ If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup. | Drexel Hill, Pennsylvania19026
Trond Eivind Glomsr�d wrote: > On Tue, 21 May 2002, Lamar Owen wrote: > > > On Tuesday 21 May 2002 11:04 am, Manuel Sugawara wrote: > > > I see. This behavior is consistent with the fact that mktime is > > > supposed to return -1 on error, but then is broken in every other Unix > > > implementation that I know. > > > > > Any other workaround than downgrade or install FreeBSD? > > > > Complain to Red Hat. Loudly. However, as this is a glibc change, other > > distributors are very likely to fold in this change sooner rather than > > later. > > Relying on nonstandardized/nondocumented behaviour is a program bug, not a > glibc bug. PostgreSQL needs fixing. Since we ship both, we're looking at > it, but glibc is not the component with a problem. No one has really answered the question --- if the way PostgreSQL is using mktime() for pre-1970 dates is wrong, why do timezone databases have pre-1970 timezone information? I assume Linux does or the old mktime() wouldn't have worked for pre-1970 dates. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 853-3000+ If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup. | Drexel Hill, Pennsylvania19026