Thread: copy from stdin; bug?
Hello i don't know, whether it is a real bug or what, has been fixed or not, but i can't find any info about it: i try to fill my table from a file using copy from stdin and postgresql corrupt the table. This happen if before the <tab> or end of line there is word that has a non-standard letter like o with accent and then an ordinary lette [a-z]. and now i reproduced this error if that o is at the and of the word. in these cases that word and the next one get into the same field... why does it happen? thanks tRehak ------------------------------------ E-Mail: Tom Rehak <rehak@sch.bme.hu> ------------------------------------
Rehak Tamas <rehak@sch.bme.hu> writes:
> i try to fill my table from a file using copy from stdin and postgresql
> corrupt the table. This happen if before the <tab> or end of line there
> is word that has a non-standard letter like o with accent and then an
> ordinary lette [a-z]. and now i reproduced this error if that o is at the
> and of the word. in these cases that word and the next one get into the
> same field...
Sounds to me like a multibyte-character translation problem. What
encoding do you have set for the database? What have you told it the
client encoding is?
regards, tom lane
Hi On Sun, 14 Jan 2001, Tom Lane wrote: > Sounds to me like a multibyte-character translation problem. What > encoding do you have set for the database? What have you told it the > client encoding is? sorry, i forgot to write: i use debian potato, i use the standard debian packege. version 6.5.3 i haven't set any encoding i think it must be the default (?) i used psql to copy the texts they are iso-8859-2 encoded... please write if you need more information. Udv tRehak ------------------------------------ E-Mail: Tom Rehak <rehak@sch.bme.hu> ------------------------------------
> On Sun, 14 Jan 2001, Tom Lane wrote: > > Sounds to me like a multibyte-character translation problem. What > > encoding do you have set for the database? What have you told it the > > client encoding is? > > sorry, i forgot to write: > i use debian potato, i use the standard debian packege. > version 6.5.3 > > i haven't set any encoding i think it must be the default (?) > i used psql to copy the texts they are iso-8859-2 encoded... > > please write if you need more information. Can you show me your database name and the output from 'psql -l'? -- Tatasuo Ishii
> yes, here are the output: > > datname |datdba|encoding|datpath > -----------------+------+--------+----------------- > template1 | 31| 5|template1 > map | 1003| 5|map > helyes | 1003| 5|helyes > > i found that if i put a space behind the letters ([o with > accent][a-z][\t]) before the tab, it works correct... but without the > space it corrupt the database... The encoding of your databases are all UNICODE. So you need to input data as UTF-8 in this case. I guess you are trying to input ISO-8859-1 encoded data that is the source of the problem. Here are possible solutions: 1) input data as UTF-8 2) crete a new databse using encoidng LATIN1. createdb -E LATIN1... 3) upgrade to 7.1 that has the capability to do an automatic conversion between UTF-8 and ISO-8859-1. -- Tatsuo Ishii
Hello On Mon, 15 Jan 2001, Tatsuo Ishii wrote: > Can you show me your database name and the output from 'psql -l'? yes, here are the output: datname |datdba|encoding|datpath -----------------+------+--------+----------------- template1 | 31| 5|template1 map | 1003| 5|map helyes | 1003| 5|helyes i found that if i put a space behind the letters ([o with accent][a-z][\t]) before the tab, it works correct... but without the space it corrupt the database... Udv tRehak ------------------------------------ E-Mail: Tom Rehak <rehak@sch.bme.hu> ------------------------------------
Re On Tue, 16 Jan 2001, Tatsuo Ishii wrote: > The encoding of your databases are all UNICODE. So you need to input > data as UTF-8 in this case. I guess you are trying to input ISO-8859-1 > encoded data that is the source of the problem. Here are possible > solutions: > 1) input data as UTF-8 :) > 2) crete a new databse using encoidng LATIN1. createdb -E LATIN1... yes, this will be the sollution... > 3) upgrade to 7.1 that has the capability to do an automatic > conversion between UTF-8 and ISO-8859-1. i like to use deb packages and to use 7.1 i would have to upgrade to woody (or even sid)... thank you for your quick help!!! Udv tRehak ------------------------------------ E-Mail: Tom Rehak <rehak@sch.bme.hu> ------------------------------------
On Wed, Jan 17, 2001 at 01:40:58AM +0100, Rehak Tamas wrote: > > > 3) upgrade to 7.1 that has the capability to do an automatic > > conversion between UTF-8 and ISO-8859-1. > > i like to use deb packages and to use 7.1 i would have to upgrade > to woody (or even sid)... Not true. There are Debian source packages, and taking the source package from Debian 2.x, x>2 (woody/sid), you can easily build it on Debian 2.2 (potato). In fact, it seems likely that a 2.2 (potato) packaging of 7.1 should be available from somebody else anyhow. Oliver, do you plan to make the woody 7.1 package depend on any other package versions not in potato? If not, you can just use the 7.1 package directly on your Debian 2.2 system. (Apologies to the rest for the Debian jargon.) Nathan Myers ncm@zembu.com
FYI, if you would like to build 7.1 with the UTF-8 <--> other encodings capability, you need to configure --enable-multibyte AND --enable-unicode-conversion, and that would add a 1MB conversion table linked to the backend. -- Tatsuo Ishii > On Wed, Jan 17, 2001 at 01:40:58AM +0100, Rehak Tamas wrote: > > > > > 3) upgrade to 7.1 that has the capability to do an automatic > > > conversion between UTF-8 and ISO-8859-1. > > > > i like to use deb packages and to use 7.1 i would have to upgrade > > to woody (or even sid)... > > Not true. There are Debian source packages, and taking the source > package from Debian 2.x, x>2 (woody/sid), you can easily build it > on Debian 2.2 (potato). > > In fact, it seems likely that a 2.2 (potato) packaging of 7.1 should be > available from somebody else anyhow. Oliver, do you plan to make the > woody 7.1 package depend on any other package versions not in potato? > If not, you can just use the 7.1 package directly on your Debian 2.2 > system. > > (Apologies to the rest for the Debian jargon.) > > Nathan Myers > ncm@zembu.com
Nathan Myers wrote: [ ... ] > Not true. There are Debian source packages, Where are they ? I'm *quite* interested ! > and taking the source > package from Debian 2.x, x>2 (woody/sid), you can easily build it > on Debian 2.2 (potato). > > In fact, it seems likely that a 2.2 (potato) packaging of 7.1 should be > available from somebody else anyhow. Oliver, do you plan to make the > woody 7.1 package depend on any other package versions not in potato? > If not, you can just use the 7.1 package directly on your Debian 2.2 > system. Oliver Elphick seems awfully busy and once said that 7.1 required a *lot* of packaging ... Better not bug him right now ... -- Emmanuel Charpentier
Nathan Myers wrote: >In fact, it seems likely that a 2.2 (potato) packaging of 7.1 should be >available from somebody elseanyhow. Oliver, do you plan to make the >woody 7.1 package depend on any other package versions not in potato? It will automatically depend on whatever is in my machine at the time (unstable updated at least weekly), e.g.: libc6 2.2.1. However, as we have with 7.0.3, we will build a version for potato on a potato machine and make that available for those who don't want to risk up-grading the rest of their systems. -- Oliver Elphick Oliver.Elphick@lfix.co.uk Isle of Wight http://www.lfix.co.uk/oliver PGP: 1024R/32B8FAA1: 97 EA 1D 47 72 3F 28 47 6B 7E 39 CC 56 E4 C1 47 GPG: 1024D/3E1D0C1C: CA12 09E0 E8D5 8870 5839 932A 614D 4C34 3E1D 0C1C ======================================== "And, behold, I come quickly; and my reward is with me, to give every manaccording as his work shall be." Revelation 22:12
Emmanuel Charpentier wrote: >Oliver Elphick seems awfully busy and once said that 7.1 required a >*lot* of packaging ...Better not bug him right now ... I'm working on it at the moment. -- Oliver Elphick Oliver.Elphick@lfix.co.uk Isle of Wight http://www.lfix.co.uk/oliver PGP: 1024R/32B8FAA1: 97 EA 1D 47 72 3F 28 47 6B 7E 39 CC 56 E4 C1 47 GPG: 1024D/3E1D0C1C: CA12 09E0 E8D5 8870 5839 932A 614D 4C34 3E1D 0C1C ======================================== "For the eyes of the LORD run to and fro throughout the whole earth, to showhimself strong in the behalf of them whose heart is perfect toward him..." II Chronicles 16:9
On Wed, Jan 17, 2001 at 10:34:45PM +0100, Emmanuel Charpentier wrote: > Nathan Myers wrote: > > Not true. There are Debian source packages, > Where are they ? I'm *quite* interested ! > > and taking the source > > package from Debian 2.x, x>2 (woody/sid), you can easily build it > > on Debian 2.2 (potato). They're as near as "apt-get source postgresql", once you have edited /etc/apt/sources.list to point to the package repositories you want, and have run "apt-get update" to synchronize with those repositories. Under the best circumstances, "apt-get source -b postgresql" will download the sources and build a ".deb" tailored for your system, much as in the BSD "ports" system. (I say "best circumstances" because the package or the build process may depend on tools and "-dev" packages you have not "apt-get install"'ed yet, and because you might prefer to tinker with configuration options before building.) I'm sure once Oliver has prepared a 7.1 Debian package he will announce it here. (This is getting dangerously Debian-specific.) Nathan Myers ncm@zembu.com