Thread: Locale Issue

Locale Issue

From
Don Parris
Date:
Hi all,

When I install the Kubuntu (13.04) postgresql (9.1) packages, the default template1 encoding turns out to be ASCII, which is not really what I want.  My OS locale command reveals everything to be en_US.UTF-8, except for LC_ALL, which is left open.

I am guessing that my best bet is to change the locale settings of my OS in order to get postgresql server to use a UTF-8 locale setting from the system.  But which setting should I change?

I did run pg_dropcluster, pg_createcluster (setting locale to C.UTF8) - and that seemed to work, except that I could not thereafter make remote connections (despite resetting the hba and postgres.conf files as they should be).

Thanks!
Don
--
D.C. Parris, FMP, Linux+, ESL Certificate
Minister, Security/FM Coordinator, Free Software Advocate
GPG Key ID: F5E179BE

Re: Locale Issue

From
Tom Lane
Date:
Don Parris <parrisdc@gmail.com> writes:
> When I install the Kubuntu (13.04) postgresql (9.1) packages, the default
> template1 encoding turns out to be ASCII, which is not really what I want.
> My OS locale command reveals everything to be en_US.UTF-8, except for
> LC_ALL, which is left open.

initdb will absorb locale/encoding from its environment, unless told
otherwise through a --locale switch.  So the usual expectation would be
that it'd work like you want.  Perhaps the Ubuntu packager set LANG=C in
the postgres user's ~/.profile, or some such?  Poke around a bit in the
package's files, it shouldn't be too hard to find where the damage is
being done.

            regards, tom lane


Re: Locale Issue

From
Don Parris
Date:
On Wed, Aug 21, 2013 at 10:08 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
Don Parris <parrisdc@gmail.com> writes:
<SNIP>
 
initdb will absorb locale/encoding from its environment, unless told
otherwise through a --locale switch.  So the usual expectation would be
that it'd work like you want.  Perhaps the Ubuntu packager set LANG=C in
the postgres user's ~/.profile, or some such?  Poke around a bit in the
package's files, it shouldn't be too hard to find where the damage is
being done.

                        regards, tom lane

Thanks Tom,

I'll check that out.

--
D.C. Parris, FMP, Linux+, ESL Certificate
Minister, Security/FM Coordinator, Free Software Advocate
GPG Key ID: F5E179BE

Re: Locale Issue

From
Stuart Bishop
Date:
On Wed, Aug 21, 2013 at 4:55 PM, Don Parris <parrisdc@gmail.com> wrote:

> I did run pg_dropcluster, pg_createcluster (setting locale to C.UTF8) - and
> that seemed to work, except that I could not thereafter make remote
> connections (despite resetting the hba and postgres.conf files as they
> should be).

Rather than chasing locales, I'd look into why you failed here. When
you install the PostgreSQL packages, it runs pg_createcluster for you.
If you don't like the locale or encoding you used, you run
pg_dropcluster and pg_createcluster as you did. The reason why your
database did not work after doing this is probably obvious from your
log files.


--
Stuart Bishop <stuart@stuartbishop.net>
http://www.stuartbishop.net/


Re: Locale Issue

From
Don Parris
Date:
On Thu, Aug 22, 2013 at 3:23 AM, Stuart Bishop <stuart@stuartbishop.net> wrote:
On Wed, Aug 21, 2013 at 4:55 PM, Don Parris <parrisdc@gmail.com> wrote:

> I did run pg_dropcluster, pg_createcluster (setting locale to C.UTF8) - and
> that seemed to work, except that I could not thereafter make remote
> connections (despite resetting the hba and postgres.conf files as they
> should be).

Rather than chasing locales, I'd look into why you failed here. When
you install the PostgreSQL packages, it runs pg_createcluster for you.
If you don't like the locale or encoding you used, you run
pg_dropcluster and pg_createcluster as you did. The reason why your
database did not work after doing this is probably obvious from your
log files.


Hi Stuart,

 

Hi Stuart,


Your point is valid and well-taken.  I discovered a couple of other issues in my poking around:

<> The pgsql logs turned up empty (literally), so would have to rely on system logs for any hints of connection issues.

<> It seems my iptables was ruined - I kept getting errors about needing an update and the software update manager kept giving me errors on kernel updates.  I could not create iptables rules to open the port.

<> I also had problems with my SSH service.

I really suspect the whole connectivity issue was related to updates and something going South with my iptables setup (among other possible issues).  I had been able to connect remotely before I replaced the cluster, but not after, so I just cannot be too sure what ultimately caused the problem.


But I still come back to the locale issue.  I am glad I was on the right track in replacing the cluster.  Still, how could I have made UTF-8 the default encoding at install time?  Maybe the very first step on a Kubuntu system should be to replace the cluster before doing anything else.  Or maybe there is a locale setting that can be changed to ensure the pgsql package gets the 'right' cues? 



An OpenSUSE 12.3 installation with pgsql 9.2 comes up with a UTF8-encoded template1.  So why does the (K)Ubuntu package default to an ASCII-encoded template1?  When I read the fine manual (and it really is pretty good), I get the impression it is best to let initdb/pg_createcluster take its cue from the OS locale settings.  If my locale on Kubuntu shows up as UTF-8, then why the seeming miscue?  I am certain there is a reasonable explanation, but that eludes me at this point.


Regards,
Don

--
D.C. Parris, FMP, Linux+, ESL Certificate
Minister, Security/FM Coordinator, Free Software Advocate
GPG Key ID: F5E179BE

Re: Locale Issue

From
Vincent Veyron
Date:
Le jeudi 22 août 2013 à 11:29 -0400, Don Parris a écrit :

>   Still, how could I have made UTF-8 the default encoding at install
> time?

I did several recent installations of Postgresql on Debian Wheezy with
UTF8 and LATIN9, and my system's encoding was used every time.

Installation steps outlined below.

Before you install Postgresql, make sure your systems encoding is set to
UTF-8, which you can check with :
locale -a

If it's not, run :
dpkg-reconfigure locales

and select UTF-8 for your language.

Use apt-get to install Postgresql :
apt-get install postgresql postgresql-client

I'm not sure if Kubuntu adds some overhead to the installation process.
You might want to try a regular debian install and add the KDE desktop
to it.


--
Salutations, Vincent Veyron
http://marica.fr/site/demonstration
Logiciel de gestion des contentieux juridiques et des sinistres d'assurance



Re: Locale Issue

From
Don Parris
Date:
On Thu, Aug 22, 2013 at 11:29 AM, Don Parris <parrisdc@gmail.com> wrote:
On Thu, Aug 22, 2013 at 3:23 AM, Stuart Bishop <stuart@stuartbishop.net> wrote:
On Wed, Aug 21, 2013 at 4:55 PM, Don Parris <parrisdc@gmail.com> wrote:

<SNIP>
you install the PostgreSQL packages, it runs pg_createcluster for you.
If you don't like the locale or encoding you used, you run
pg_dropcluster and pg_createcluster as you did. The reason why your
database did not work after doing this is probably obvious from your
log files.


Hi Stuart,

 <SNIP>

 

But I still come back to the locale issue.  I am glad I was on the right track in replacing the cluster.  Still, how could I have made UTF-8 the default encoding at install time?  Maybe the very first step on a Kubuntu system should be to replace the cluster before doing anything else.  Or maybe there is a locale setting that can be changed to ensure the pgsql package gets the 'right' cues? 



<SNIP>
I did find a (bit dated) response on the UbuntuForums site, but assume the basic advice applies:
http://askubuntu.com/questions/20880/how-do-i-create-a-unicode-databases-in-postgresql-8-4
~#export LANGUAGE=en_US.UTF-8
~#export LANG=en_US.UTF-8
~#export LC_ALL=en_US.UTF-8
~#locale-gen en_US.UTF-8
~#dpkg-reconfigure locales

The recommendation is to change the OS locale settings and use dpkg to reconfigure locales *before* recreating the cluster.

I really do think the Ubuntu documentation needs to clarify that.

Finally, I am ever more convinced that a recent update effectively blew up my connectivity.

--
D.C. Parris, FMP, Linux+, ESL Certificate
Minister, Security/FM Coordinator, Free Software Advocate
GPG Key ID: F5E179BE

Re: Locale Issue

From
Don Parris
Date:
On Thu, Aug 22, 2013 at 5:12 PM, Vincent Veyron <vv.lists@wanadoo.fr> wrote:
Le jeudi 22 août 2013 à 11:29 -0400, Don Parris a écrit :

>   Still, how could I have made UTF-8 the default encoding at install
> time?

I did several recent installations of Postgresql on Debian Wheezy with
UTF8 and LATIN9, and my system's encoding was used every time.

Installation steps outlined below.

Before you install Postgresql, make sure your systems encoding is set to
UTF-8, which you can check with :
locale -a

If it's not, run :
dpkg-reconfigure locales

and select UTF-8 for your language.

Use apt-get to install Postgresql :
apt-get install postgresql postgresql-client

I'm not sure if Kubuntu adds some overhead to the installation process.
You might want to try a regular debian install and add the KDE desktop
to it.


Thanks Vincent.  locale showed everything *except* LC_ALL as being en_US.UTF-8.  LC_ALL was empty.  I can only guess that the LC_ALL setting may have played a part in the problem.  See my previous e-mail (sent just a  few minutes before this one).  I may have also been fine if that system had not lost connectivity.  Trying out things on OpenSUSE 12.3, I do have SSH connectivity and the Postgres server came up with template1 encoded as UTF-8 by default.  I'll test out the pgsql remote connectivity soon as well.

Re: Locale Issue

From
Vincent Veyron
Date:
Le jeudi 22 août 2013 à 20:06 -0400, Don Parris a écrit :

>  locale showed everything *except* LC_ALL as being en_US.UTF-8.
> LC_ALL was empty.  I can only guess that the LC_ALL setting may have
> played a part in the problem.

I doubt it, mine was not set either and the installation went fine. See
the output of locale on my machine :

LANG=fr_FR.UTF-8
LANGUAGE=
LC_CTYPE="fr_FR.UTF-8"
LC_NUMERIC="fr_FR.UTF-8"
LC_TIME="fr_FR.UTF-8"
LC_COLLATE="fr_FR.UTF-8"
LC_MONETARY="fr_FR.UTF-8"
LC_MESSAGES="fr_FR.UTF-8"
LC_PAPER="fr_FR.UTF-8"
LC_NAME="fr_FR.UTF-8"
LC_ADDRESS="fr_FR.UTF-8"
LC_TELEPHONE="fr_FR.UTF-8"
LC_MEASUREMENT="fr_FR.UTF-8"
LC_IDENTIFICATION="fr_FR.UTF-8"
LC_ALL=


>   See my previous e-mail (sent just a  few minutes before this one).

Agreed : change your settings, then create the cluster.

I would stick with the standard 'apt-get install', unless you have a
reason not to.


--
Salutations, Vincent Veyron
http://marica.fr/
Gestion des contrats, des contentieux juridiques et des sinistres
d'assurance