Re: Cannot import logs from csv - Mailing list pgsql-general

From GRIFFITHS H.P.
Subject Re: Cannot import logs from csv
Date
Msg-id ADE4CEE57ABD5F408EB0FA146C62F96F05B33A9C@CISCMRMBS02.mds.ad.dur.ac.uk
Whole thread Raw
In response to Re: Cannot import logs from csv  (Helen Griffiths <helen.griffiths@durham.ac.uk>)
List pgsql-general
Something else that causes a similar error is if the PostgreSQL daemon is restarted in the middle of logging a query.
Thelog line is truncated and the next log line starts straight after the truncated line without even a newline in
between. The problematic line can be many lines before where the import process shows an error. 

The last problem I found that shows this error is when control characters end up in your queries, usually because
someoneisn't validating their input >:-(  Putting the logfiles through "iconv -c -f UTF8 -t ASCII" seems to fix this. 

--
Helen Griffiths


> -----Original Message-----
> From: pgsql-general-owner@postgresql.org [mailto:pgsql-general-
> owner@postgresql.org] On Behalf Of Helen Griffiths
> Sent: 18 October 2013 18:27
> To: pgsql-general@postgresql.org
> Subject: Re: [GENERAL] Cannot import logs from csv
>
> On Fri, 18 Oct 2013, Adrian Klaver wrote:
>
> > This came up before recently in this thread:
> >
> > http://www.postgresql.org/message-
> id/CADK3HHJNEWKD9gNyXmjv9ABbn+37rY3M
> > vp9=1j7msg9YpoBBBw@mail.gmail.com
> >
> > To cut to the chase, in that case the OP found:
> >
> > "Ok, I found the offending line. It was not the pgadmin line. There
> > was a line with a large binary insert."
> >
> > Not sure if that helps.
>
> It might: there are then some lines like:
> 2013-10-11 02:44:26.164
> BST,"postgres","course_records",19216,"[local]",525740b3.4b10,124887,"idle
> in transaction",2013-10-11 01:05:07 BST,7/16388,0,LOG,00000,"fastpath
> function call: ""loread"" (OID 954)",,,,,,,,
>
> I suspect the ""loread"" is the offender.
>
> Thanks to all for the help.  I would never have thought to suspect the lines after
> the one given in the error message.
>
> --
> Helen Griffiths
>
>
> --
> Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-general


pgsql-general by date:

Previous
From: Brian Crowell
Date:
Subject: Re: GSSAPI server side on Linux, SSPI client side on Windows
Next
From: Brian Crowell
Date:
Subject: Re: GSSAPI server side on Linux, SSPI client side on Windows