Re: New "single" COPY format - Mailing list pgsql-hackers

From Joel Jacobson
Subject Re: New "single" COPY format
Date
Msg-id 826954e1-f800-4871-8a9a-e692c45085cf@app.fastmail.com
Whole thread Raw
List pgsql-hackers
On Thu, Nov 7, 2024, at 17:15, Joel Jacobson wrote:
> Attachments:
> * v18-0001-Introduce-CopyFormat-and-replace-csv_mode-and-binary.patch
> * v18-0002-Add-COPY-format-single.patch
> * v18-0003-Reorganize-option-validations.patch

I want to bring up a potential problem with v18, which has been discussed
before:

On Tue, Oct 15, 2024, at 19:30, Jacob Champion wrote:
> Hi,
>
> Idle thoughts from a design perspective -- feel free to ignore, since
> I'm not the target audience for the feature:
>
> - If the column data stored in Postgres contains newlines, it seems
> like COPY TO won't work "correctly". Is that acceptable?

Example:

CREATE TABLE log (line TEXT);
INSERT INTO log (line) VALUES (E'foo\nbar'), ('baz');
COPY log TO '/tmp/log.txt' (FORMAT 'single');
COPY 2

% cat log.txt
foo
bar
baz

TRUNCATE log;
COPY log FROM '/tmp/log.txt' (FORMAT 'single');
SELECT * FROM log;
 line
------
 foo
 bar
 baz
(3 rows)

It would be nice if we could come up with an approach, that didn't introduce
this footgun, while at the same time being convenient for the common use cases.

Ideas?

/Joel



pgsql-hackers by date:

Previous
From: Jim Nasby
Date:
Subject: Re: Vacuum statistics
Next
From: "Daniel Verite"
Date:
Subject: Re: New "single" COPY format