Re: Pre-proposal: unicode normalized text - Mailing list pgsql-hackers

From Nico Williams
Subject Re: Pre-proposal: unicode normalized text
Date
Msg-id ZR8UBTZcywbhK4JI@ubby21
Whole thread Raw
In response to Re: Pre-proposal: unicode normalized text  (Tom Lane <tgl@sss.pgh.pa.us>)
Responses Re: Pre-proposal: unicode normalized text
List pgsql-hackers
On Thu, Oct 05, 2023 at 03:49:37PM -0400, Tom Lane wrote:
> Nico Williams <nico@cryptonector.com> writes:
> > Text+encoding can be just like bytea with a one- or two-byte prefix
> > indicating what codeset+encoding it's in.  That'd be how to encode
> > such text values on the wire, though on disk the column's type should
> > indicate the codeset+encoding, so no need to add a prefix to the value.
> 
> The precedent of BOMs (byte order marks) suggests strongly that
> such a solution would be horrible to use.

This is just how you encode the type of the string.  You have any number
of options.  The point is that already PG can encode binary data, so if
how to encode text of disparate encodings on the wire, building on top
of the encoding of bytea is an option.



pgsql-hackers by date:

Previous
From: Tom Lane
Date:
Subject: Re: Pre-proposal: unicode normalized text
Next
From: Laurenz Albe
Date:
Subject: Re: Good News Everyone! + feature proposal