Thread: BUG #2120: Crash when doing UTF8<->ISO_8859_8 encoding conversion

BUG #2120: Crash when doing UTF8<->ISO_8859_8 encoding conversion

From
"Sagi Bashari"
Date:
The following bug has been logged online:

Bug reference:      2120
Logged by:          Sagi Bashari
Email address:      sagi@adamnet.co.il
PostgreSQL version: 8.1.1
Operating system:   Debian Sarge
Description:        Crash when doing UTF8<->ISO_8859_8 encoding conversion
Details:

Postgresql crashes when a client with ISO_8859_8 encoding tries to select
data from a utf8 database.

I have compiled postgresql 8.1.1 from scratch with the following commands:
./configure --prefix=/home/sagi/temp/pgtest --enable-debug
make
make install
mkdir /home/sagi/temp/pgtest/data
/home/sagi/temp/pgtest/bin/initdb -D /home/sagi/temp/pgtest/data/
/home/sagi/temp/pgtest/bin/postmaster -D /home/sagi/temp/pgtest/data

Created a utf8 database:
./createdb test -E utf8

And ran 'SET client_encoding = 'ISO_8859_8'; SELECT 'שלום';' inside
`psql test`.

That's ISO_8859_8 hebrew text inside the SELECT. Here's a file containing
the query (incase the mail breaks it):
http://future.adamnet.co.il/~sagi/temp/enc.sql

Psql returned the following message:
server closed the connection unexpectedly
        This probably means the server terminated abnormally
        before or while processing the request.

The database log:
LOG:  server process (PID 1290) was terminated by signal 11

Backtrace:
Core was generated by `postgres: sagi test [local] idle
              '.
Program terminated with signal 11, Segmentation fault.

#0  0x0823a1a6 in compare2 ()
#1  0x40100b52 in bsearch () from /lib/libc.so.6
#2  0x0823a41c in LocalToUtf ()
#3  0x40df8909 in iso8859_to_utf8 () from
/home/sagi/temp/pgtest/lib/postgresql/utf8_and_iso8859.so
#4  0x08233112 in FunctionCall5 ()
#5  0x0823adc8 in perform_default_encoding_conversion ()
#6  0x0823acd3 in pg_client_to_server ()
#7  0x081508d4 in pq_getmsgstring ()
#8  0x081b924e in PostgresMain ()
#9  0x08191b1c in BackendRun ()
#10 0x08191565 in BackendStartup ()
#11 0x0818f7c1 in ServerLoop ()
#12 0x0818eb5a in PostmasterMain ()
#13 0x08150e0e in main ()

Re: BUG #2120: Crash when doing UTF8<->ISO_8859_8 encoding conversion

From
Tom Lane
Date:
"Sagi Bashari" <sagi@adamnet.co.il> writes:
> Postgresql crashes when a client with ISO_8859_8 encoding tries to select
> data from a utf8 database.

It looks like somebody rearranged the pg_enc enum without bothering to
fix the tables that are affected by this.

utf8_and_iso8859.c is certainly broken, and I'm wondering what else
might be.  Tatsuo, can you think of any other places to look?

            regards, tom lane

Re: BUG #2120: Crash when doing UTF8<->ISO_8859_8 encoding conversion

From
Tom Lane
Date:
Tatsuo Ishii <ishii@sraoss.co.jp> writes:
>> It looks like somebody rearranged the pg_enc enum without bothering to
>> fix the tables that are affected by this.

> I will look into this.

Thank you.  It might be worth adding a comment to pg_wchar.h listing all
the places that need to be fixed when enum pg_enc changes.

            regards, tom lane

Re: BUG #2120: Crash when doing UTF8<->ISO_8859_8 encoding

From
Tatsuo Ishii
Date:
> "Sagi Bashari" <sagi@adamnet.co.il> writes:
> > Postgresql crashes when a client with ISO_8859_8 encoding tries to select
> > data from a utf8 database.
>
> It looks like somebody rearranged the pg_enc enum without bothering to
> fix the tables that are affected by this.
>
> utf8_and_iso8859.c is certainly broken, and I'm wondering what else
> might be.  Tatsuo, can you think of any other places to look?

I will look into this.
--
Tatsuo Ishii
SRA OSS, Inc. Japan

Re: BUG #2120: Crash when doing UTF8<->ISO_8859_8 encoding

From
Tatsuo Ishii
Date:
> > "Sagi Bashari" <sagi@adamnet.co.il> writes:
> > > Postgresql crashes when a client with ISO_8859_8 encoding tries to select
> > > data from a utf8 database.
> >
> > It looks like somebody rearranged the pg_enc enum without bothering to
> > fix the tables that are affected by this.
> >
> > utf8_and_iso8859.c is certainly broken, and I'm wondering what else
> > might be.  Tatsuo, can you think of any other places to look?
>
> I will look into this.

Quick check reveals that ISO-8859-5 to ISO-8859-8 are broken.
--
Tatsuo Ishii
SRA OSS, Inc. Japan