Re: Re: Inconsistency between JDBC and ODBC drivers when dealing with TIMESTAMP WITH TIME ZONE - Mailing list pgsql-odbc

From Walter Couto
Subject Re: Re: Inconsistency between JDBC and ODBC drivers when dealing with TIMESTAMP WITH TIME ZONE
Date
Msg-id 7FA4F9E5512F214C801E596ECD8790E066A83019@ETNAMAIL01.embarcadero.com
Whole thread Raw
In response to Re: Re: Inconsistency between JDBC and ODBC drivers when dealing with TIMESTAMP WITH TIME ZONE  (David Johnston <david.g.johnston@gmail.com>)
Responses Re: Re: Inconsistency between JDBC and ODBC drivers when dealing with TIMESTAMP WITH TIME ZONE  ("Inoue, Hiroshi" <inoue@tpf.co.jp>)
List pgsql-odbc

Yes, I have to workaround this issue in my application since the existing issue as the current drivers have this bug. And to avoid overhead caching the server time zone is the best option to avoid repeated calls (thought their is a risk of you being wrong as you can changes the session time zone via a statement which will make the cached value wrong if it was run before that.

 

As apposed to showing my code to get a SQL_C_CHAR  I'll show you the ODBC code that is perplexing:

 

In convert.c's copy_and_convert_field() line 946: This case statement block states that for PG_TYPE_ABSTIME, PG_TYPE_DATETIME, PG_TYPE_TIMESTAMP_NO_TMZONE and PG_TYPE_TIMESTAMP that the string given by the server is:

1) -Infinity

2) -Infinity

3) invalid

4) %4d-%2d-%2d %2d:%2d:%2d%31s %15s (YYYY-MM-dd hh:mm:ss[.ffffff][-/+zh[:zm] [BC]) (max 35 characters for year < 10000)

 

What that code does for 1) - 3) is perfectly fine, you have to choose some timestamp value and those values are as good as anything else. 4) is dealt by the function timestamp2stime on line 224 of the convert_field.  The only bug I see in this code is how it handles the time zone parsing, it only handles +/-zh, but not the possible but rare +/-zh:zm format/ That bug is in the switch statement on line 245 of that file. The cases for '+', '-' and '.' all don't check for the existence of ":" in the string.  But doesn't matter currently as we throw away the time zone, so no fix is needed in this method at the current time, just a note that it is not handling a possible form of the string correctly.

 

Going back to copy_and_convert_field() on line 1147 this case happens if we are returning the text version of a value as indicated by the local variable text_handling. We then get to line 1170 where it handles PG_TYPE_ABSTIME, PG_TYPE_DATETIME, PG_TYPE_TIMESTAMP_NO_TMZONE and PG_TYPE_TIMESTAMP again and it calls a function, stime2timestamp, to rebuild the string in the format of 4) but with no zone information! (NOTE: stime2timestamp also has a bug with time zone it doesn`t handle offset that need the minute, i.e. -3:30 but not a concern as zone is never used currently)  If I was writing this part of the code I would have something like the logic shown below as I don`t understand what is wrong with the string the server gave us and why we would want to give it in an alternate form:

 

case PG_TYPE_ABSTIME:

case PG_TYPE_DATETIME:

case PG_TYPE_TIMESTAMP_NO_TMZONE:

case PG_TYPE_TIMESTAMP:

   len = 19;

   if (cbValueMax > len)

   {

      if ( (strnicmp(value, INFINITY_STRING, 8) == 0) ||

           (strnicmp(value, MINFINITY_STRING, 9) == 0) ||

           (strnicmp(value, "invalid", 7) == 0) // Keep old behaviour

      {      

         /* sprintf(rgbValueBindRow, "%.4d-%.2d-%.2d %.2d:%.2d:%.2d",

         std_time.y, std_time.m, std_time.d, std_time.hh, std_time.mm, std_time.ss); */

         stime2timestamp(&std_time, rgbValueBindRow, FALSE,

                         PG_VERSION_GE(conn, 7.2) ? (int) cbValueMax - len - 2 : 0);

         len = strlen(rgbValueBindRow);

      }

      else

      {

         len = strlen(value);

         if (cbValueMax > len)

         {

            // Just give back what the server gave us

            strcpy(rgbValueBindRow, value);

         }

         else

         {      

            /* fall back to generating the string */

            stime2timestamp(&std_time, rgbValueBindRow, FALSE,

                         PG_VERSION_GE(conn, 7.2) ? (int) cbValueMax - len - 2 : 0);

            len = strlen(rgbValueBindRow);

         } 

      }

   }

   break;

 

I hope that clears things up.

 

Walter

 


From: David Johnston [david.g.johnston@gmail.com]
Sent: Tuesday, June 10, 2014 1:04 AM
To: Inoue, Hiroshi
Cc: Adrian Klaver; Walter Couto; pgsql-odbc@postgresql.org
Subject: Re: [ODBC] Re: Inconsistency between JDBC and ODBC drivers when dealing with TIMESTAMP WITH TIME ZONE

On Tuesday, June 10, 2014, Inoue, Hiroshi <inoue@tpf.co.jp> wrote:
(2014/06/10 12:18), Adrian Klaver wrote:
On 06/09/2014 07:49 PM, Walter Couto wrote:
I have to strongly disagree that this is an enhancement.  It is a
severe bug. In PostgreSQL, if you look at both the JDBC and ODBC
source code, for timestamp with time zone, the string that the server
gives to the driver for my example is "2002-04-11 02:33:08.12345+3"
if the session time zone is set to +3.

Asking for a timestamp with time zone as a timestamp in either ODBC
and JDBC is done knowing you are accepting the lose of time zone info.
I have NO issue with this (as a side note the JDBC and ODBC driver did
behave the same at one point, in both cases they returned the
timestamp in the server time zone, but surprise change in a recent
version to correct this to be JDBC spec compliant caused some
headaches for some people using the driver as they wrote code assuming
this behaviour and now were all wrong).

I have an issue with the retuned string value from ODBC....the driver
got a perfectly fine string, it could have given it to me. There is no
sane reason to have the time zone part stripped out before giving it
to me.  At least with JDBC it gave me an EQUALLY ACCURATE string of
"2002-04-10 19:22:08.12345-04", with ODBC I need to run a separate
query to get the session time zone to correct the string that was
perfectly fine when ODBC was given it.

Changing the query is nice for internal queries and is a good work
around for this bug, but when non-internal queries are given to our
application, we can't expect to ask the customers to work around
driver bugs and change all their perfectly working queries. I expect
the driver to give information as accurately as the server gave to the
driver.  That is what I am asking here and that is what is not happing
in the case of ODBC when asking for the string form of a timestamp
with time zone.  This is why I see this as a severe bug as the driver
is giving me CORRUPT data that is not the same or even equally the
same as the data it retrieved from the server that the client tool has
to "fix" by issuing another query.

Currently psqlodbc retrieves TIMESTAMP WITH TIME ZONE pretty much
the same as TIMESTAMP WITHOUT TIME ZONE. Though there are some code
 about time zone handling, it is disabled.

The data type TIMESTAMP_STRUCT which correponds to SQL_C_TIMESTAMP
has no information about time zone. If we would retrieve time zone
explictly, a consensus is needed which time zone is used to store
TIMESTAMP_STRUCT type data.

1. UTC
2. client local time zone
3. server time zone

The driver's behavior has been 3. So UTC seems unfavorable.
Probably most people have used the driver under environment
2 == 3. Should we use client local time zone instead?


There is no functional difference between 2 and 3 if the information sent from the server cannot be embedded into the stored typed.

That is the issue since what the driver is providing when a string is requested is not the raw server data but the string  representation of the server content after it has been converted into a local timestamp object and the timezone stripped.

If indeed the timezone is silently dropped then it matters not what you call resolving the issue - enhancement or bugfix. In the end someone has to volunteer to dive into the open souce code and develop a patch.  Until then you live with the limitation, write a wrapper to accomodate a local workaround (say by caching the server timezone to avoid repetitive queries), and/or choose a different driver/protocol.

Note I doubt any acceptable solution would be simply returning supposedly raw text from the server. I think one problem is that the data is not always in text format.  It is possible that an internal type could be created but if you go that far you might as well go make the whole thing work normally and not just via a getString method.

David J.


CONFIDENTIALITY NOTICE: This email message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message.   ­­  

pgsql-odbc by date:

Previous
From: David Johnston
Date:
Subject: Re: Re: Inconsistency between JDBC and ODBC drivers when dealing with TIMESTAMP WITH TIME ZONE
Next
From: Heikki Linnakangas
Date:
Subject: Re: Problem retrieving a numeric(38,0) value as SQL_NUMERIC_STRUCT if value needs to use all 16 SQLCHAR elements of the val array