Re: confusing / inefficient "need_transcoding" handling in copy - Mailing list pgsql-hackers

From Michael Paquier
Subject Re: confusing / inefficient "need_transcoding" handling in copy
Date
Msg-id ZcvlgMEjt3qY8eiL@paquier.xyz
Whole thread Raw
In response to Re: confusing / inefficient "need_transcoding" handling in copy  (Sutou Kouhei <kou@clear-code.com>)
Responses Re: confusing / inefficient "need_transcoding" handling in copy
List pgsql-hackers
On Thu, Feb 08, 2024 at 05:25:01PM +0900, Sutou Kouhei wrote:
> In <20240206222445.hzq22pb2nye7rm67@awork3.anarazel.de>
>   "Re: confusing / inefficient "need_transcoding" handling in copy" on Tue, 6 Feb 2024 14:24:45 -0800,
>   Andres Freund <andres@anarazel.de> wrote:
>
>> One unfortunate issue: We don't have any tests verifying that COPY FROM
>> catches encoding issues.
>
> How about the attached patch for it?
>
> +CREATE TABLE test (t text);
> +COPY test FROM stdin WITH (ENCODING 'EUC_JP');
> +こんにちは
> +\.
> +
> +DROP TABLE test;

We have a couple of non-ASCII characters in the tests, but I suspect
that this one will not be digested correctly everywhere, even if
EUC_JP should be OK to use for the check.  How about writing an
arbitrary sequence of bytes into a temporary file that gets used for
the COPY FROM instead?  See for example how we do that with
abs_builddir in copy.sql.
--
Michael

Attachment

pgsql-hackers by date:

Previous
From: Alexander Korotkov
Date:
Subject: Re: Transaction timeout
Next
From: Michael Paquier
Date:
Subject: Re: Small fix on query_id_enabled