Re: Re: COPY BINARY file format proposal - Mailing list pgsql-hackers

From Tom Lane
Subject Re: Re: COPY BINARY file format proposal
Date
Msg-id 29751.976260665@sss.pgh.pa.us
Whole thread Raw
In response to Re: Re: COPY BINARY file format proposal  (Tom Lane <tgl@sss.pgh.pa.us>)
Responses Re: Re: COPY BINARY file format proposal  (Philip Warner <pjw@rhyme.com.au>)
List pgsql-hackers
I wrote:
> Next 4 bytes: integer layout field.  This consists of the int32 constant
> 0x0A820D0A expressed in the source machine's endianness.  (Again, value
> chosen with malice aforethought, to catch files munged by things like
> DOS/Unix newline conversion or high-bit-stripping.)

Actually, that won't do.  A little-endian machine would write 0A 0D 82
0A which would fail to trigger newline converters that are looking for
\r followed by \n (0D 0A).  If we're going to take seriously the idea of
detecting newline transforms, then we need to incorporate the test
pattern into the fixed-byte-order signature.

How about:

Signature: 12-byte sequence "PGBCOPY\n\377\r\n\0" (detects newline
replacements, dropped nulls, dropped high bits, parity changes);

Integer layout field: int32 constant 0x01020304 in source's byte order.

The rest as before.
        regards, tom lane


pgsql-hackers by date:

Previous
From: Tom Lane
Date:
Subject: Re: Re: COPY BINARY file format proposal
Next
From: "Horst Herb"
Date:
Subject: RFC: CRC datatype