Thread: COPY from file

COPY from file

From
Ole Gjerde
Date:
Hey,
I have found what looks to be a bug in the COPY command.

This is current CVS as of monday at 7pm CST.

I have a tab delimited file(It's actually just a dump) that includes
some non-normal ascii characters, the one in question being y with 2 dots.
If the data has ANY of those in a char(or vchar) field the COPY finishes
successfully with no errors, but the table is empty.  If I edit
the data file and remove that y(there was only one), it works just fine.
So basically, COPY silently dropped that line on the floor and didn't say
anything about it.

Postgres was compiled like this: 
./configure --prefix=/home/postgres --with-maxbackends=128 ; make
and ran like this: postmaster -d 9 -o '-F'
on Linux 2.2.14 (Mandrake 7.0)

The above scenario happens when I just try to run on a one line data 
file(which I can provide if necessary).  If I try the big macdaddy file
with around 350 MB of data(and about 12 out of the 3-4 million rows has
that pesky y in it), more interesting things happen.

The backend log with -d 9 says: 
FATAL 1:  Socket command type (y with 2 dots goes here) unknown
NOTICE:  AbortTransaction and not in in-progress state   

And psql says this:psql:parts.dump:42: Backend message type 0x45 arrived while idle 
pqReadData() -- backend closed the channel unexpectedly.       This probably means the backend terminated abnormally
   before or while processing the request. 
 
PQsendQuery() -- There is no connection to the backend. 
PQsendQuery() -- There is no connection to the backend. 
PQsendQuery() -- There is no connection to the backend.
PQsendQuery() -- There is no connection to the backend. 
PQsendQuery() -- There is no connection to the backend. 
PQsendQuery() -- There is no connection to the backend. 
PQsendQuery() -- There is no connection to the backend. 
PQsendQuery() -- There is no connection to the backend.
PQsendQuery() -- There is no connection to the backend.                         

2 weird things here.  psql takes a good 30-40 seconds to notice that the
backend has died and I had another session(idle) of psql going, and that
backend died as well, but the postmaster was still running.

All of the data above will import just fine into 6.5.3(on the same
machine/os).

Thanks,
Ole Gjerde



Re: [HACKERS] COPY from file

From
Peter Eisentraut
Date:
On 2000-02-08, Ole Gjerde mentioned:

> I have a tab delimited file(It's actually just a dump) that includes
> some non-normal ascii characters, the one in question being y with 2 dots.
> If the data has ANY of those in a char(or vchar) field the COPY finishes
> successfully with no errors, but the table is empty.  If I edit
> the data file and remove that y(there was only one), it works just fine.

Fixed it. Thanks for the report, we'd never have found that one.


-- 
Peter Eisentraut                  Sernanders väg 10:115
peter_e@gmx.net                   75262 Uppsala
http://yi.org/peter-e/            Sweden



Re: [HACKERS] COPY from file

From
Tom Lane
Date:
Ole Gjerde <gjerde@icebox.org> writes:
> I have found what looks to be a bug in the COPY command.

Ooops.  Fixed.  ('char c' -> 'int c' ... EOF doesn't fit in a char ...)
        regards, tom lane


Re: [HACKERS] COPY from file

From
Peter Eisentraut
Date:
Okay, somebody beat me to it. Seems we can't fix these bugs fast enough.
;)

On 2000-02-09, Peter Eisentraut mentioned:

> On 2000-02-08, Ole Gjerde mentioned:
> 
> > I have a tab delimited file(It's actually just a dump) that includes
> > some non-normal ascii characters, the one in question being y with 2 dots.
> > If the data has ANY of those in a char(or vchar) field the COPY finishes
> > successfully with no errors, but the table is empty.  If I edit
> > the data file and remove that y(there was only one), it works just fine.
> 
> Fixed it. Thanks for the report, we'd never have found that one.
> 
> 
> 

-- 
Peter Eisentraut                  Sernanders väg 10:115
peter_e@gmx.net                   75262 Uppsala
http://yi.org/peter-e/            Sweden