Re: Initdb-cs_CZ.WIN-1250 buildfarm failures - Mailing list pgsql-hackers

On 20.12.2014 17:48, Noah Misch wrote:
> On Sat, Dec 20, 2014 at 05:14:03PM +0100, CSPUG wrote:
>> On 20.12.2014 07:39, Noah Misch wrote:
>>> Buildfarm members magpie, treepie and fulmar went absent on
>>> 2014-10-29. Since returning on 2014-11-16, they have consistently
>>> failed with 'initdb: invalid locale name "cs_CZ.WIN-1250"'. No
>>> commits in that period readily explain a regression in this area. Did
>>> the underlying system configurations change?
>>
>> I'm pretty sure it's because of broken locales at the system level. It was
>> working fine, and I haven't done any substantial changes to the system
>> except for occasional "yum update", so I'm unaware of what went wong :(
>>
>> The issue looks like this:
>>
>> # locale -a | grep en_US
>> en_US
>> en_US.iso88591
>> en_US.iso885915
>> en_US.utf8
>>
>> # locale en_US
>> locale: unknown name "en_US"
>
> The NAME argument to the "locale" tool is something like LC_PAPER, not a
> locale name.  Use "LANG=en_US locale LC_NUMERIC" to test locale loading.

D'oh! That kinda explains the strange `locale` failures, of course ...

But still, the three animals (magpie, fulmar, treepie) are all running
with these locales:

   C en_US
   cs_CZ.UTF-8 cs_CZ.ISO-8859-2 cs_CZ.WIN-1250
   sk_SK.UTF-8 sk_SK.ISO-8859-2 sk_SK.WIN-1250

and the only one missing (so the 'locale LC_NUMERIC' failed) was
sk_SK.WIN-1250. I've fixed that, and now all the locales work fine.

That however does not explain why the tests are failing with
cs_CZ.WIN-1250 because that locale was working for some time.

>
>> The only reasons I can think of is that some of the updates required
>> a reboot, and I haven't done that because that would kill all the VMs
>> running on that HW, including the one with CLOBBER_CACHE_RECURSIVE
>> tests. And that'd throw away tests running for ~3 months.
>>
>> I've disabled the three animals (magpie, fulmar, treepie) for now, because
>> there's no point in running the tests until the locale issues are fixed. If
>> anyone has an idea of what might be wrong, let me know.
>
> Those animals have been successfully completing initdb for several locales,
> including en_US, before failing at cs_CZ.WIN-1250.  You could disable just the
> cs_CZ.WIN-1250 steps.  A CentOS 6.6 system here also lacks such a locale:
>
> $ LANG=cs_CZ.WIN-1250 locale LC_NUMERIC
> locale: Cannot set LC_CTYPE to default locale: No such file or directory
> locale: Cannot set LC_MESSAGES to default locale: No such file or directory
> locale: Cannot set LC_ALL to default locale: No such file or directory
> .
>
> -1
> 46
> 0
> ANSI_X3.4-1968
> $ locale -a|grep cs_
> cs_CZ
> cs_CZ.iso88592
> cs_CZ.utf8
>
> Perhaps an update made the system stricter about rejecting unknown locales.

I believe the locale system (at the OS level) works just like before. I
remember I had to manually create the locales while initially setting up
the animals. Then, ~2 months ago something happened (I asssume a yum
update) and some of the locales disappeared. But I have recreated them,
except for sk_SK.WIN-1250. But the tests fail because of cs_CZ.WIN-1250
which does exist.

Attached is a log of

for l in cs_CZ.UTF-8 sk_SK.UTF-8 cs_CZ.ISO-8859-2 sk_SK.ISO-8859-2
cs_CZ.WIN-1250 sk_SK.WIN-1250; do
  echo $l; LANG=$l locale LC_NUMERIC;
done;

showing that all the locales exist. However when I tried to initialize a
cluster with cs_CZ.WIN-1250, I got an error like this:

[pgbuild@regular-builds ~]$ locale
LANG=cs_CZ.WIN-1250
LC_CTYPE="cs_CZ.WIN-1250"
LC_NUMERIC="cs_CZ.WIN-1250"
LC_TIME="cs_CZ.WIN-1250"
LC_COLLATE="cs_CZ.WIN-1250"
LC_MONETARY="cs_CZ.WIN-1250"
LC_MESSAGES="cs_CZ.WIN-1250"
LC_PAPER="cs_CZ.WIN-1250"
LC_NAME="cs_CZ.WIN-1250"
LC_ADDRESS="cs_CZ.WIN-1250"
LC_TELEPHONE="cs_CZ.WIN-1250"
LC_MEASUREMENT="cs_CZ.WIN-1250"
LC_IDENTIFICATION="cs_CZ.WIN-1250"
LC_ALL=
[pgbuild@regular-builds ~]$ pg_ctl -D tmp-data init
The files belonging to this database system will be owned by user "pgbuild".
This user must also own the server process.

The database cluster will be initialized with locale "cs_CZ.WIN-1250".
could not determine encoding for locale "cs_CZ.WIN-1250": codeset is
"ANSI_X3.4-1968"
initdb: could not find suitable encoding for locale "cs_CZ.WIN-1250"
Rerun initdb with the -E option.
Try "initdb --help" for more information.
pg_ctl: database system initialization failed

So apparently the locale does exist, but we're unable to cope with it
for some reason ...

Tomas