Re: Emitting JSON to file using COPY TO - Mailing list pgsql-hackers

From Joe Conway
Subject Re: Emitting JSON to file using COPY TO
Date
Msg-id 4162f7f7-6fd6-4720-98e2-89f80e3de2ed@joeconway.com
Whole thread Raw
In response to Re: Emitting JSON to file using COPY TO  (Joe Conway <mail@joeconway.com>)
Responses Re: Emitting JSON to file using COPY TO
Re: Emitting JSON to file using COPY TO
Re: Emitting JSON to file using COPY TO
Re: Emitting JSON to file using COPY TO
List pgsql-hackers
On 12/6/23 18:09, Joe Conway wrote:
> On 12/6/23 14:47, Joe Conway wrote:
>> On 12/6/23 13:59, Daniel Verite wrote:
>>>     Andrew Dunstan wrote:
>>> 
>>>> IMNSHO, we should produce either a single JSON 
>>>> document (the ARRAY case) or a series of JSON documents, one per row 
>>>> (the LINES case).
>>> 
>>> "COPY Operations" in the doc says:
>>> 
>>> " The backend sends a CopyOutResponse message to the frontend, followed
>>>     by zero or more CopyData messages (always one per row), followed by
>>>     CopyDone".
>>> 
>>> In the ARRAY case, the first messages with the copyjsontest
>>> regression test look like this (tshark output):
>>> 
>>> PostgreSQL
>>>      Type: CopyOut response
>>>      Length: 13
>>>      Format: Text (0)
>>>      Columns: 3
>>>     Format: Text (0)
>>> PostgreSQL
>>>      Type: Copy data
>>>      Length: 6
>>>      Copy data: 5b0a
>>> PostgreSQL
>>>      Type: Copy data
>>>      Length: 76
>>>      Copy data:
>>> 207b226964223a312c226631223a226c696e652077697468205c2220696e2069743a2031…
>>> 
>>> The first Copy data message with contents "5b0a" does not qualify
>>> as a row of data with 3 columns as advertised in the CopyOut
>>> message. Isn't that a problem?
>> 
>> 
>> Is it a real problem, or just a bit of documentation change that I missed?
>> 
>> Anything receiving this and looking for a json array should know how to
>> assemble the data correctly despite the extra CopyData messages.
> 
> Hmm, maybe the real problem here is that Columns do not equal "3" for
> the json mode case -- that should really say "1" I think, because the
> row is not represented as 3 columns but rather 1 json object.
> 
> Does that sound correct?
> 
> Assuming yes, there is still maybe an issue that there are two more
> "rows" that actual output rows (the "[" and the "]"), but maybe those
> are less likely to cause some hazard?


The attached should fix the CopyOut response to say one column. I.e. it 
ought to look something like:

PostgreSQL
      Type: CopyOut response
      Length: 13
      Format: Text (0)
      Columns: 1
      Format: Text (0)
PostgreSQL
      Type: Copy data
      Length: 6
      Copy data: 5b0a
PostgreSQL
      Type: Copy data
      Length: 76
      Copy data: [...]


-- 
Joe Conway
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

Attachment

pgsql-hackers by date:

Previous
From: "David G. Johnston"
Date:
Subject: Re: Emitting JSON to file using COPY TO
Next
From: Joe Conway
Date:
Subject: Re: Emitting JSON to file using COPY TO