testing locales and encodings - Mailing list pgsql-hackers

From Alvaro Herrera
Subject testing locales and encodings
Date
Msg-id 20080714134909.GC4050@alvh.no-ip.org
Whole thread Raw
In response to Re: PATCH: CITEXT 2.0 v3  (Andrew Dunstan <andrew@dunslane.net>)
Responses Re: testing locales and encodings  ("David E. Wheeler" <david@kineticode.com>)
List pgsql-hackers
Andrew Dunstan wrote:

> Some time ago I raised the question of doing locale- and encoding-aware  
> regression tests, and IIRC the consensus seemed to be that it was not  
> worth the effort, but maybe we need to revisit that. We certainly seem  
> to be doing a lot more work now to get Postgres to where it needs to be  
> w.r.t. locale support, and we've cleaned up a lot of the encoding issues  
> we had, so maybe we need to reflect that in our testing regime more.

I agree.  Maybe we need to unearth the src/test/mb or src/test/locale;
perhaps have the buildfarm run those tests when they get to a useful
point.

I notice that src/test/locale is not VPATH-aware, and I'm unsure if it's
got a clear distinction of locales vs. encodings (it calls koi8-r a
locale).

src/test/mb is not VPATH aware either (and it doesn't have a Makefile at
all), and is now broken by the fact that createdb refuses to create a
database with mismatching encoding.

$ LC_ALL=C sh mbregress.sh 
euc_jp .. createdb: database creation failed: ERROR:  encoding EUC_JP does not match server's locale fr_CA.UTF-8
DETAIL:  The server's LC_CTYPE setting requires encoding UTF8.
failed
sjis .. failed
euc_kr .. createdb: database creation failed: ERROR:  encoding EUC_KR does not match server's locale fr_CA.UTF-8
DETAIL:  The server's LC_CTYPE setting requires encoding UTF8.
failed
euc_cn .. createdb: database creation failed: ERROR:  encoding EUC_CN does not match server's locale fr_CA.UTF-8
DETAIL:  The server's LC_CTYPE setting requires encoding UTF8.
failed
euc_tw .. createdb: database creation failed: ERROR:  encoding EUC_TW does not match server's locale fr_CA.UTF-8
DETAIL:  The server's LC_CTYPE setting requires encoding UTF8.
failed
big5 .. failed
utf8 .. ok
mule_internal .. createdb: database creation failed: ERROR:  encoding MULE_INTERNAL does not match server's locale
fr_CA.UTF-8
DETAIL:  The server's LC_CTYPE setting requires encoding UTF8.
failed

So this whole area would seem to need a lot of love ...

-- 
Alvaro Herrera                                http://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support


pgsql-hackers by date:

Previous
From: Andrew Dunstan
Date:
Subject: Re: PATCH: CITEXT 2.0 v3
Next
From: Tom Lane
Date:
Subject: Re: PATCH: CITEXT 2.0 v3