Re: UTF8 conversion differences from v8.1.3 to v8.1.4 - Mailing list pgsql-general

From Eric Faulhaber
Subject Re: UTF8 conversion differences from v8.1.3 to v8.1.4
Date
Msg-id 44BD76E7.1040708@goldencode.com
Whole thread Raw
In response to Re: UTF8 conversion differences from v8.1.3 to v8.1.4  (Tom Lane <tgl@sss.pgh.pa.us>)
Responses Re: UTF8 conversion differences from v8.1.3 to v8.1.4  (Martijn van Oosterhout <kleptog@svana.org>)
List pgsql-general
Tom Lane wrote:
> Eric Faulhaber <ecf@goldencode.com> writes:
>> OK, but this particular issue is something quite new to the latest
>> version.
>
> Again, PG has never stored such data correctly.
>

Perhaps not, but it silently tolerated such data until this release, at
least at the encoding conversion level.  I don't know what happened to
the embedded nulls beyond that point (ignorance is bliss), but our JDBC
queries were working as expected...

BTW, any idea why we don't see this problem when issuing the same query
from psql?  I've set psql's encoding to UTF8 to try to trigger the
conversion when running against the LATIN1-encoded database.  It happily
returns the result we previously achieved with JDBC on 8.1.3.  Is psql
filtering out embedded nulls before the backend sees them?

>> Am I stuck at 8.1.3 for the time being?  I'd be happy to create a patch
>> to resolve this for a future version, but if it is not considered a
>> defect, it doesn't make sense for me to do that.
>
> It's not a defect ... or at least, it doesn't make sense to change it
> unless you are willing to go through the entire system to make it able
> to store null bytes in text.  We've looked at that in the past and
> always concluded that it was completely impractical :-(
>
>             regards, tom lane

:-( indeed, though I appreciate the dialog, Tom.  Sadly, this would not
be the first completely impractical task on my todo list ;-)

Thanks,
Eric


pgsql-general by date:

Previous
From: Reid Thompson
Date:
Subject: Re: postmaster: StreamConnection: accept: No such device
Next
From: Q
Date:
Subject: Re: Antw: Performance problem with query