Thread: copy from stdin; bug?

copy from stdin; bug?

From
Rehak Tamas
Date:
Hello

i don't know, whether it is a real bug or what, has been fixed or not, but
i can't find any info about it:

i try to fill my table from a file using copy from stdin and postgresql
corrupt the table. This happen if before the <tab> or end of line there
is word that has a non-standard letter like o with accent and then an
ordinary lette [a-z]. and now i reproduced this error if that o is at the
and of the word. in these cases that word and the next one get into the
same field...

why does it happen?

thanks
tRehak

------------------------------------
E-Mail: Tom Rehak <rehak@sch.bme.hu>
------------------------------------



Re: copy from stdin; bug?

From
Tom Lane
Date:
Rehak Tamas <rehak@sch.bme.hu> writes:
> i try to fill my table from a file using copy from stdin and postgresql
> corrupt the table. This happen if before the <tab> or end of line there
> is word that has a non-standard letter like o with accent and then an
> ordinary lette [a-z]. and now i reproduced this error if that o is at the
> and of the word. in these cases that word and the next one get into the
> same field...

Sounds to me like a multibyte-character translation problem.  What
encoding do you have set for the database?  What have you told it the
client encoding is?
        regards, tom lane


Re: copy from stdin; bug?

From
Rehak Tamas
Date:
Hi

On Sun, 14 Jan 2001, Tom Lane wrote:
> Sounds to me like a multibyte-character translation problem.  What
> encoding do you have set for the database?  What have you told it the
> client encoding is?

sorry, i forgot to write:
i use debian potato, i use the standard debian packege.
version 6.5.3

i haven't set any encoding i think it must be the default (?)
i used psql to copy the texts they are iso-8859-2 encoded...

please write if you need more information.

Udv
tRehak

------------------------------------
E-Mail: Tom Rehak <rehak@sch.bme.hu>
------------------------------------



Re: copy from stdin; bug?

From
Tatsuo Ishii
Date:
> On Sun, 14 Jan 2001, Tom Lane wrote:
> > Sounds to me like a multibyte-character translation problem.  What
> > encoding do you have set for the database?  What have you told it the
> > client encoding is?
> 
> sorry, i forgot to write:
> i use debian potato, i use the standard debian packege.
> version 6.5.3
> 
> i haven't set any encoding i think it must be the default (?)
> i used psql to copy the texts they are iso-8859-2 encoded...
> 
> please write if you need more information.

Can you show me your database name and the output from 'psql -l'?
--
Tatasuo Ishii


Re: copy from stdin; bug?

From
Tatsuo Ishii
Date:
> yes, here are the output:
> 
> datname          |datdba|encoding|datpath
> -----------------+------+--------+-----------------
> template1        |    31|       5|template1
> map              |  1003|       5|map
> helyes           |  1003|       5|helyes
> 
> i found that if i put a space behind the letters ([o with
> accent][a-z][\t]) before the tab, it works correct... but without the
> space it corrupt the database...

The encoding of your databases are all UNICODE. So you need to input
data as UTF-8 in this case. I guess you are trying to input ISO-8859-1
encoded data that is the source of the problem. Here are possible
solutions:

1) input data as UTF-8

2) crete a new databse using encoidng LATIN1. createdb -E LATIN1...

3) upgrade to 7.1 that has the capability to do an automatic  conversion between UTF-8 and ISO-8859-1.
--
Tatsuo Ishii



Re: copy from stdin; bug?

From
Rehak Tamas
Date:
Hello

On Mon, 15 Jan 2001, Tatsuo Ishii wrote:
> Can you show me your database name and the output from 'psql -l'?

yes, here are the output:

datname          |datdba|encoding|datpath
-----------------+------+--------+-----------------
template1        |    31|       5|template1
map              |  1003|       5|map
helyes           |  1003|       5|helyes

i found that if i put a space behind the letters ([o with
accent][a-z][\t]) before the tab, it works correct... but without the
space it corrupt the database...

Udv
tRehak

------------------------------------
E-Mail: Tom Rehak <rehak@sch.bme.hu>
------------------------------------



Re: copy from stdin; bug?

From
Rehak Tamas
Date:
Re

On Tue, 16 Jan 2001, Tatsuo Ishii wrote:
> The encoding of your databases are all UNICODE. So you need to input
> data as UTF-8 in this case. I guess you are trying to input ISO-8859-1
> encoded data that is the source of the problem. Here are possible
> solutions:
> 1) input data as UTF-8
:)

> 2) crete a new databse using encoidng LATIN1. createdb -E LATIN1...
yes, this will be the sollution...

> 3) upgrade to 7.1 that has the capability to do an automatic
>    conversion between UTF-8 and ISO-8859-1.
i like to use deb packages and to use 7.1 i would have to upgrade to woody
(or even sid)...

thank you for your quick help!!!

Udv
tRehak

------------------------------------
E-Mail: Tom Rehak <rehak@sch.bme.hu>
------------------------------------



Re: copy from stdin; bug?

From
ncm@zembu.com (Nathan Myers)
Date:
On Wed, Jan 17, 2001 at 01:40:58AM +0100, Rehak Tamas wrote:
> 
> > 3) upgrade to 7.1 that has the capability to do an automatic
> >    conversion between UTF-8 and ISO-8859-1.
>
> i like to use deb packages and to use 7.1 i would have to upgrade
> to woody (or even sid)...

Not true.  There are Debian source packages, and taking the source
package from Debian 2.x, x>2 (woody/sid), you can easily build it 
on Debian 2.2 (potato).  

In fact, it seems likely that a 2.2 (potato) packaging of 7.1 should be 
available from somebody else anyhow.  Oliver, do you plan to make the 
woody 7.1 package depend on any other package versions not in potato?  
If not, you can just use the 7.1 package directly on your Debian 2.2 
system.  

(Apologies to the rest for the Debian jargon.)

Nathan Myers
ncm@zembu.com


Re: copy from stdin; bug?

From
Tatsuo Ishii
Date:
FYI, if you would like to build 7.1 with the UTF-8 <--> other
encodings capability, you need to configure --enable-multibyte AND
--enable-unicode-conversion, and that would add a 1MB conversion table
linked to the backend.
--
Tatsuo Ishii

> On Wed, Jan 17, 2001 at 01:40:58AM +0100, Rehak Tamas wrote:
> > 
> > > 3) upgrade to 7.1 that has the capability to do an automatic
> > >    conversion between UTF-8 and ISO-8859-1.
> >
> > i like to use deb packages and to use 7.1 i would have to upgrade
> > to woody (or even sid)...
> 
> Not true.  There are Debian source packages, and taking the source
> package from Debian 2.x, x>2 (woody/sid), you can easily build it 
> on Debian 2.2 (potato).  
> 
> In fact, it seems likely that a 2.2 (potato) packaging of 7.1 should be 
> available from somebody else anyhow.  Oliver, do you plan to make the 
> woody 7.1 package depend on any other package versions not in potato?  
> If not, you can just use the 7.1 package directly on your Debian 2.2 
> system.  
> 
> (Apologies to the rest for the Debian jargon.)
> 
> Nathan Myers
> ncm@zembu.com


Re: copy from stdin; bug?

From
Emmanuel Charpentier
Date:
Nathan Myers wrote:
[ ... ]

> Not true.  There are Debian source packages, 

Where are they ? I'm *quite* interested !

>                                              and taking the source
> package from Debian 2.x, x>2 (woody/sid), you can easily build it
> on Debian 2.2 (potato).
> 
> In fact, it seems likely that a 2.2 (potato) packaging of 7.1 should be
> available from somebody else anyhow.  Oliver, do you plan to make the
> woody 7.1 package depend on any other package versions not in potato?
> If not, you can just use the 7.1 package directly on your Debian 2.2
> system.

Oliver Elphick seems awfully busy and once said that 7.1 required a
*lot* of packaging ... Better not bug him right now ...

--
Emmanuel Charpentier


Re: copy from stdin; bug?

From
"Oliver Elphick"
Date:
Nathan Myers wrote: >In fact, it seems likely that a 2.2 (potato) packaging of 7.1 should be  >available from somebody
elseanyhow.  Oliver, do you plan to make the  >woody 7.1 package depend on any other package versions not in potato?  
 

It will automatically depend on whatever is in my machine at the time 
(unstable updated at least weekly), e.g.: libc6 2.2.1.

However, as we have with 7.0.3, we will build a version for potato on a 
potato machine and make that available for those who don't want to risk
up-grading the rest of their systems.

-- 
Oliver Elphick                                Oliver.Elphick@lfix.co.uk
Isle of Wight                              http://www.lfix.co.uk/oliver
PGP: 1024R/32B8FAA1: 97 EA 1D 47 72 3F 28 47  6B 7E 39 CC 56 E4 C1 47
GPG: 1024D/3E1D0C1C: CA12 09E0 E8D5 8870 5839  932A 614D 4C34 3E1D 0C1C
========================================   "And, behold, I come quickly; and my reward is with me,     to give every
manaccording as his work shall be."                                  Revelation 22:12 
 




Re: copy from stdin; bug?

From
"Oliver Elphick"
Date:
Emmanuel Charpentier wrote: >Oliver Elphick seems awfully busy and once said that 7.1 required a >*lot* of packaging
...Better not bug him right now ...
 

I'm working on it at the moment.


-- 
Oliver Elphick                                Oliver.Elphick@lfix.co.uk
Isle of Wight                              http://www.lfix.co.uk/oliver
PGP: 1024R/32B8FAA1: 97 EA 1D 47 72 3F 28 47  6B 7E 39 CC 56 E4 C1 47
GPG: 1024D/3E1D0C1C: CA12 09E0 E8D5 8870 5839  932A 614D 4C34 3E1D 0C1C
========================================   "For the eyes of the LORD run to and fro throughout the     whole earth, to
showhimself strong in the behalf of      them whose heart is perfect toward him..."
        II Chronicles 16:9 
 




Re: copy from stdin; bug?

From
ncm@zembu.com (Nathan Myers)
Date:
On Wed, Jan 17, 2001 at 10:34:45PM +0100, Emmanuel Charpentier wrote:
> Nathan Myers wrote:
> > Not true.  There are Debian source packages, 

> Where are they ? I'm *quite* interested !

> >                                              and taking the source
> > package from Debian 2.x, x>2 (woody/sid), you can easily build it
> > on Debian 2.2 (potato).

They're as near as "apt-get source postgresql", once you have edited 
/etc/apt/sources.list to point to the package repositories you want,
and have run "apt-get update" to synchronize with those repositories.  

Under the best circumstances, "apt-get source -b postgresql" will 
download the sources and build a ".deb" tailored for your system, 
much as in the BSD "ports" system.  (I say "best circumstances"
because the package or the build process may depend on tools and
"-dev" packages you have not "apt-get install"'ed yet, and because
you might prefer to tinker with configuration options before building.)

I'm sure once Oliver has prepared a 7.1 Debian package he will announce
it here.

(This is getting dangerously Debian-specific.)

Nathan Myers
ncm@zembu.com