Thread: psql file restore - problem with encoding

Hi all,

I have problem on one of our potentional client's linux server with data
encoding. I have played with this for some time but with no success.
Here is the problem:

I have dump file created with pg_dump on my dev linux machine
(PostgreSQL 8.0.1). When I upload the file to the client's server with
psql, the data is uploaded but it seems the strings were encoded once
again though file header states the data is in UNICODE and the database
is in UNICODE too.

The database was initialized with 'initdb --locale=cs_CZ.utf8 -D
/var/lib/pgsql/data'.

Here is a sample how the data look:
Akzent Media spole�nost s ru�ením omezený
and how they should look:
Akzent Media společnost s ručením omezeným

What is strange on the machine is that there are perhaps two versions of
postgresql installed - version 8.0.1 and 7.4.2 so I use the full path to
the 8.0.1. I think that the system is Fedora Core (but I'm not sure -
unfortunatelly I cannot catch the admin now). We have no real control of
the client's server so we cannot install, uninstall etc.

I have also tried to dump some data from our php app - with the same
result. phpinfo() says that php was configured against PostgreSQL 8.0.1
but it also says in PostgreSQL configuration info that there is
"PostgreSQL(libpq) Version 7.4.2".

It seems to me that the problem might be in:
1) the mixture of the two versions of PostgreSQL in the system
2) some problem with locale
3) something different

Can someone please help me to diagnose this problem so I can solve it
and move forward?

Thank you in advance.

Miroslav

Just update - the system is Fedora Core 2, but the PostgreSQL is
compiled from sources.

Miroslav Šulc wrote:

> Hi all,
>
> I have problem on one of our potentional client's linux server with
> data encoding. I have played with this for some time but with no
> success. Here is the problem:
>
> I have dump file created with pg_dump on my dev linux machine
> (PostgreSQL 8.0.1). When I upload the file to the client's server with
> psql, the data is uploaded but it seems the strings were encoded once
> again though file header states the data is in UNICODE and the
> database is in UNICODE too.
>
> The database was initialized with 'initdb --locale=cs_CZ.utf8 -D
> /var/lib/pgsql/data'.
>
> Here is a sample how the data look:
> Akzent Media spole�nost s ru�ením omezený
> and how they should look:
> Akzent Media společnost s ručením omezeným
>
> What is strange on the machine is that there are perhaps two versions
> of postgresql installed - version 8.0.1 and 7.4.2 so I use the full
> path to the 8.0.1. I think that the system is Fedora Core (but I'm not
> sure - unfortunatelly I cannot catch the admin now). We have no real
> control of the client's server so we cannot install, uninstall etc.
>
> I have also tried to dump some data from our php app - with the same
> result. phpinfo() says that php was configured against PostgreSQL
> 8.0.1 but it also says in PostgreSQL configuration info that there is
> "PostgreSQL(libpq) Version 7.4.2".
>
> It seems to me that the problem might be in:
> 1) the mixture of the two versions of PostgreSQL in the system
> 2) some problem with locale
> 3) something different
>
> Can someone please help me to diagnose this problem so I can solve it
> and move forward?
>
> Thank you in advance.
>
> Miroslav
>
>---------------------------(end of broadcast)---------------------------
>TIP 2: you can get off all lists at once with the unregister command
>    (send "unregister YourEmailAddressHere" to majordomo@postgresql.org)
>
>

On Mar 16, 2005, at 9:34 AM, Miroslav Šulc wrote:

> I have problem on one of our potentional client's linux server with
> data encoding. I have played with this for some time but with no
> success. Here is the problem:
>

Maybe try setting the client side encoding explicitly at the top of the
dump file:

\encoding UNICODE

It should also be possible to set this in the psql command line if you
don't want to modify the file.

John DeSoi, Ph.D.
http://pgedit.com/
Power Tools for PostgreSQL

John DeSoi wrote:

>
> On Mar 16, 2005, at 9:34 AM, Miroslav Šulc wrote:
>
>> I have problem on one of our potentional client's linux server with
>> data encoding. I have played with this for some time but with no
>> success. Here is the problem:
>>
>
> Maybe try setting the client side encoding explicitly at the top of
> the dump file:
>
> \encoding UNICODE
>
> It should also be possible to set this in the psql command line if you
> don't want to modify the file.

The dump file already contains the encoding information as the first
command:

SET client_encoding = 'UNICODE';

so I think this is sufficient. What is strange to me is that the
communication is UNICODE -> UNICODE so for me it means "no conversion",
but the result is different.

>
> John DeSoi, Ph.D.
> http://pgedit.com/
> Power Tools for PostgreSQL
>
Miroslav

We have solved the problem. Maybe someone will face the same problem so
just to close this thread, the problem was in php.ini where a default
output character set was set which caused the "second" recoding problem
(just for display using php).

Miroslav Šulc wrote:

> Hi all,
>
> I have problem on one of our potentional client's linux server with
> data encoding. I have played with this for some time but with no
> success. Here is the problem:
>
> I have dump file created with pg_dump on my dev linux machine
> (PostgreSQL 8.0.1). When I upload the file to the client's server with
> psql, the data is uploaded but it seems the strings were encoded once
> again though file header states the data is in UNICODE and the
> database is in UNICODE too.
>
> The database was initialized with 'initdb --locale=cs_CZ.utf8 -D
> /var/lib/pgsql/data'.
>
> Here is a sample how the data look:
> Akzent Media spole�nost s ru�ením omezený
> and how they should look:
> Akzent Media společnost s ručením omezeným
>
> What is strange on the machine is that there are perhaps two versions
> of postgresql installed - version 8.0.1 and 7.4.2 so I use the full
> path to the 8.0.1. I think that the system is Fedora Core (but I'm not
> sure - unfortunatelly I cannot catch the admin now). We have no real
> control of the client's server so we cannot install, uninstall etc.
>
> I have also tried to dump some data from our php app - with the same
> result. phpinfo() says that php was configured against PostgreSQL
> 8.0.1 but it also says in PostgreSQL configuration info that there is
> "PostgreSQL(libpq) Version 7.4.2".
>
> It seems to me that the problem might be in:
> 1) the mixture of the two versions of PostgreSQL in the system
> 2) some problem with locale
> 3) something different
>
> Can someone please help me to diagnose this problem so I can solve it
> and move forward?
>
> Thank you in advance.
>
> Miroslav
>
>---------------------------(end of broadcast)---------------------------
>TIP 2: you can get off all lists at once with the unregister command
>    (send "unregister YourEmailAddressHere" to majordomo@postgresql.org)
>
>