Request a Demo
Home   >   mailing lists

Re: Speed up COPY FROM text/CSV parsing using SIMD - Mailing list pgsql-hackers

From Nathan Bossart
Subject Re: Speed up COPY FROM text/CSV parsing using SIMD
Date
Msg-id apWOr1kgVSqweBnU@nathan
Whole thread
In response to Re: Speed up COPY FROM text/CSV parsing using SIMD  (Nathan Bossart <nathandbossart@gmail.com>)
List pgsql-hackers
Tree view 
Claude found this bug:

    CREATE TABLE t (a TEXT);
    COPY t FROM PROGRAM $$printf 'abcdefghijklmnopqrst\n\\.\nxxxxxxxxxxxx\200'$$;

fails with

    ERROR:  invalid byte sequence for encoding "UTF8": 0x80
    CONTEXT:  COPY t, line 2

even though COPY should ignore everything after the \.

The best fix I could find involves teaching CopyLoadInputBuf() to avoid
erroring in the read-ahead path.  AFAICT that doesn't meaningfully change
the performance characteristics.  Patch attached.

-- 
nathan
Attachment

pgsql-hackers by date:

Previous
From: Matthias van de Meent
Date:
Subject: Re: Commit Sequence Numbers and Visibility
Next
From: Nathan Bossart
Date:
Subject: Re: WAIT FOR command should do some query jumbling