Postgres Pro
Downloads
Home   >   mailing lists

Thread: COPY alloc problem in 7.4

COPY alloc problem in 7.4

From
Margus Väli
Date:
Your name               : Margus Väli
Your email address      : mvali at hot.ee

                                                                                                                
System Configuration
--------------------- Architecture (example: Intel Pentium)         :
i686 athlon i386, AthlonXP 1800+ with 1Gb of memory
 Operating System (example: Linux 2.0.26 ELF)  :
2.4.18-24.8.0 #1 Fri Jan 31 07:28:55 EST 2003 i686 athlon i386 GNU/Linux
 PostgreSQL version (example: PostgreSQL-7.4):   PostgreSQL-7.4
PostgreSQL-7.4
 Compiler used (example:  gcc 2.95.2)          :
gcc version 3.2 20020903 (Red Hat Linux 8.0 3.2-7)
Please enter a FULL description of your problem:
------------------------------------------------
COPY command tries to create a buffer too large in size
when the file copied is larger than about 500 bytes.

ERROR:  invalid memory alloc request size 1073741824
Please describe a way to repeat the problem.   Please try to provide a
concise reproducible example, if at all possible:
----------------------------------------------------------------------copy marksona from '/tmp/marksona-aa' with
delimiter'|' null as ''; 
Database was created with -E UNICODE, table marksona definition:

marksona_id  integer  NOT NULL
nextval('public.marksona_marksona_id_seq'::text)
kasikiri_id  integer
sona  character(30)
sona_low  character(30)
liik  character(15)
dummy  character(3)

marksona-aa file is as follows:
1|1|Aavik, Evald|aavik, evald|Adressaat|
2|2|Aleksius|aleksius|Adressaat|
3|2| metropoliit| metropoliit|Adressaat|
4|3|Anastasjev, A.|anastasjev, a.|Adressaat|
5|4|Arbuzov, Aleksei|arbuzov, aleksei|Adressaat|
6|5|Auster, Lydia|auster, lydia|Adressaat|
7|6|NSVL Kultuuriministeeriumi|nsvl kultuuriministeeriumi|Adressaat|
8|6|Badjul, Ivan|badjul, ivan|Adressaat|
9|7|NSVL Kultuuriministeeriumi Tea|nsvl kultuuriministeeriumi
tea|Adressaat|
10|7|Baiterjakova'le, Dinora|baiterjakova'le, dinora|Adressaat|
11|8|Hetagurov'ile|hetagurov'ile|Adressaat|
12|8|Balti Sõjaväeringkonna Ülemjuh|balti sõjaväeringkonna
ülemjuh|Adressaat|
13|9|Barðtein'ile, J|barðtein'ile, j|Adressaat|
14|10|Beek'ile|beek'ile|Adressaat|
15|11|Beekman'ile, A. ja V.|beekman'ile, a. ja v.|Adressaat|
16|12|Bergvald'ile|bergvald'ile|Adressaat|
17|13|"Berliner Festtage" büroo kuns|"berliner festtage" büroo
kuns|Adressaat|
18|14|Bleive'le|bleive'le|Adressaat|
19|15|Breþnev'ile, Leonid|breþnev'ile, leonid|Adressaat|
20|16|Bush'ile, Alan|bush'ile, alan|Adressaat|
\.
         
If you know how this problem might be fixed, list the solution below:
---------------------------------------------------------------------

Re: COPY alloc problem in 7.4

From
Neil Conway
Date:
Margus Väli <mvali@hot.ee> writes:
> COPY command tries to create a buffer too large in size when the
> file copied is larger than about 500 bytes.

Can anyone else reproduce this? I did the following with a fairly
recent CVS snapshot:

    1. Copied the supplied text data to a file, fixed up the extra
       newlines so that COPY doesn't bail out

    2. Created a new database with encoding = unicode

    3. Create a table "marksona" in that DB with the supplied schema

    4. I then got the following:

unicode_test=# copy marksona from '/home/nconway/copy_input.asc' with delimiter '|' null as '';
ERROR:  Unicode characters greater than or equal to 0x10000 are not supported
CONTEXT:  COPY marksona, line 12, column sona: "Balti Sõjaväeringkonna Ülemjuh"

    5. If I load the data into a SQL_ASCII encoded DB, the COPY
       command succeeds.

However, I know absolutely nothing about Unicode, so perhaps I've
mucked something up.

-Neil

Re: COPY alloc problem in 7.4

From
Margus Väli
Date:
I'm sending another file we have the alloc problem with.
mv

Neil Conway kirjutas R, 28.11.2003 kell 23:41:
> unicode_test=# copy marksona from '/home/nconway/copy_input.asc' with delimiter '|' null as '';
> ERROR:  Unicode characters greater than or equal to 0x10000 are not supported
> CONTEXT:  COPY marksona, line 12, column sona: "Balti Sõjaväeringkonna Ülemjuh"
Attachment