Re: [PATCHES] Charset WIN1252 - Mailing list pgsql-hackers

Hello Bruce,

thank you for reply. Please let me put my remark here on top this time, 
because otherwise complete message might get more confusing.

1. the mail you quoted below is my reply to Alvaro Herrera. on 30.11.04, 
which was also in your confirmation on 01.12.04. He missed the new files 
as patches in my original mail, addressed to you and Peter Eisentraut 
also on 30.11.04. There had only been the changed files, not the new 
ones. In the mail to Alvaro Herrera with copy to pqsql-patches I sent a 
correction to patch of  "src/backend/utils/mb/encnames.c" , and as 
attachment again the missed *new* files.

2. the patches (changes) of already existing files were in the mail, 
addressed to you and Peter Eisentraut only. Based on this mail you sent 
the first confirmation on 30.11.04, and there "include/mb/pg_wchar.h" is 
listed.

3. the complete package of modified and new files were in a ZIP attached 
to a mail on 02.11.04:

> Hello developers,
>
> please find attached a ZIP with sources, headers, and tables for 
> WIN1252-Support.
>
> Even if UTF-8 handling will be completed in the engine, there will 
> still be some applications which need WIN1252 on client side. I added 
> the new charset to pg_enc and pg_enc2name[] after WIN1250 and before 
> the entries which are for client encoding only.


Because this is the first time I put an input to a development group, I 
didn't know, that "patches" were awaited. So when Peter Eisentraut wrote 
on 29.11.04

>He should send a patch, not a source tarball, so it's possible to see 
>what actually changed.
>
I sent the mail related on point 2 above.


Okay, I think things are more clear now. After backing out these patches 
from upcoming RC1, I hope you have all neccessary files available to 
apply at a later time. Meanwhile I will continue to use my patched DLLs 
for Win1252 support  ;-)


With best regards,

Roland.


---------------------------------------------------------------------------


Bruce Momjian schrieb am 05.12.2004 01:08:

>Below is the patch I applied.
>
>  
>
>>I'm the one who originally posted the patch. But the
>>"include/mb/pg_wchar.h"  was as complete source in the ZIP attached to
>>my mail on 02.11.04  as well as patch in my mail on 30.11.04 with
>>correct contents.
>>
>>And also in the confirmation of Bruce Momjian from 30.11.04 "Your patch
>>has been added to the PostgreSQL unapplied patches list ..." there was
>>the correct file contents listed.
>>    
>>
>
>You will notice there is no mention of "include/mb/pg_wchar.h" anywhere
>in the message, and it was not in the zip file, at least in the most
>recent version I applied.
>
>FYI, this patch went through several revisions as people commented on it
>so it is very possible some files get lost during the revisions.
>
>---------------------------------------------------------------------------
>
>pgman wrote:
>  
>
>>Your patch has been added to the PostgreSQL unapplied patches list at:
>>
>>    http://momjian.postgresql.org/cgi-bin/pgpatches
>>
>>It will be applied as soon as one of the PostgreSQL committers reviews
>>and approves it.
>>
>>---------------------------------------------------------------------------
>>
>>
>>Roland Volkmann wrote:
>>    
>>
>>>Hallo Alvaron
>>>
>>>
>>>Alvaro Herrera schrieb am 30.11.2004 17:38:
>>>
>>>      
>>>
>>>>On Tue, Nov 30, 2004 at 02:05:31AM +0100, Roland Volkmann wrote:
>>>>
>>>> 
>>>>
>>>>        
>>>>
>>>>>this is the first time, I use CVS (TortoisCVS), so I hope the result is 
>>>>>what you were missing:
>>>>>   
>>>>>
>>>>>          
>>>>>
>>>>You have to include the new files separately.  And please send context
>>>>diffs, not unified diffs.
>>>> 
>>>>
>>>>        
>>>>
>>>sorry, but I can't find any option for changing patch style in 
>>>TortoiseCVS. And the patches of other developers look similar to me.
>>>
>>>The new files were attached as ZIP-file to my mail on 02.11.2004. Please 
>>>find these files again attached to this mail:
>>>
>>>Index: 
>>>src/backend/utils/mb/conversion_procs/utf8_and_win1252/utf8_and_win1252.c
>>>Index: src/backend/utils/mb/conversion_procs/utf8_and_win1252/Makefile
>>>Index: src/backend/utils/mb/Unicode/utf8_to_win1252.map
>>>Index: src/backend/utils/mb/Unicode/win1252_to_utf8.map
>>>===================================================================
>>>++ new files
>>> 
>>>
>>>      
>>>
>>>>Also there's a typo here:
>>>>
>>>> 
>>>>
>>>>        
>>>>
>>>thank you, you're right. And here is the corrected patch which shoult 
>>>replace the earlier faulty one:
>>>
>>>
>>>Index: src/backend/utils/mb/encnames.c
>>>===================================================================
>>>RCS file: 
>>>Y:\CVS_Repositories\postgres.org/postgresql-8.0.0beta5/src/backend/utils/mb/encnames.c,v
>>>retrieving revision 1.1
>>>diff -u -r1.1 encnames.c
>>>--- src/backend/utils/mb/encnames.c    30 Nov 2004 23:42:16 -0000    1.1
>>>+++ src/backend/utils/mb/encnames.c    30 Nov 2004 23:56:27 -0000
>>>@@ -194,6 +194,9 @@
>>>         "win1251", PG_WIN1251
>>>     },                            /* alias for Windows-1251 */
>>>     {
>>>+        "win1252", PG_WIN1252
>>>+    },                            /* alias for Windows-1252 */
>>>+    {
>>>         "win1256", PG_WIN1256
>>>     },                            /* alias for Windows-1256 */
>>>     {
>>>@@ -221,6 +224,9 @@
>>>         "windows1251", PG_WIN1251
>>>     },                            /* Windows-1251; Microsoft */
>>>     {
>>>+        "windows1252", PG_WIN1252
>>>+    },                            /* Windows-1251; Microsoft */
>>>+    {
>>>         "windows1256", PG_WIN1256
>>>     },                            /* Windows-1256; Microsoft */
>>>     {
>>>@@ -342,6 +348,9 @@
>>>     },
>>>     {
>>>         "WIN1250", PG_WIN1250
>>>+    },
>>>+    {
>>>+        "WIN1252", PG_WIN1252
>>>     },
>>>     {
>>>         "SJIS", PG_SJIS
>>>
>>>===================================================================
>>>
>>>
>>>With best regards,
>>>
>>>Roland
>>>
>>>      
>>>
>>>#-------------------------------------------------------------------------
>>>#
>>># $PostgreSQL: pgsql/src/backend/utils/mb/conversion_procs/utf8_and_win1252/Makefile,v 1.3 2004/10/31 22:40:41 pgsql
Exp$
 
>>>#
>>>#-------------------------------------------------------------------------
>>>subdir = src/backend/utils/mb/conversion_procs/utf8_and_win1252
>>>top_builddir = ../../../../../..
>>>include $(top_builddir)/src/Makefile.global
>>>
>>>NAME        := utf8_and_win1252
>>>
>>>include $(srcdir)/../proc.mk
>>>      
>>>
>>>/*-------------------------------------------------------------------------
>>> *
>>> *      WIN1252 and UTF-8
>>> *
>>> * Portions Copyright (c) 1996-2004, PostgreSQL Global Development Group
>>> * Portions Copyright (c) 1994, Regents of the University of California
>>> *
>>> * IDENTIFICATION
>>> *      $PostgreSQL: pgsql/src/backend/utils/mb/conversion_procs/utf8_and_win1252/utf8_and_win1252.c,v 1.8
2004/10/3104:12:59 momjian Exp $
 
>>> *
>>> *-------------------------------------------------------------------------
>>> */
>>>
>>>#include "postgres.h"
>>>#include "fmgr.h"
>>>#include "mb/pg_wchar.h"
>>>#include "../../Unicode/utf8_to_win1252.map"
>>>#include "../../Unicode/win1252_to_utf8.map"
>>>
>>>PG_FUNCTION_INFO_V1(utf_to_win1252);
>>>PG_FUNCTION_INFO_V1(win1252_to_utf);
>>>
>>>extern Datum utf_to_win1252(PG_FUNCTION_ARGS);
>>>extern Datum win1252_to_utf(PG_FUNCTION_ARGS);
>>>
>>>/* ----------
>>> * conv_proc(
>>> *        INTEGER,    -- source encoding id
>>> *        INTEGER,    -- destination encoding id
>>> *        CSTRING,    -- source string (null terminated C string)
>>> *        CSTRING,    -- destination string (null terminated C string)
>>> *        INTEGER        -- source string length
>>> * ) returns VOID;
>>> * ----------
>>> */
>>>
>>>Datum
>>>utf_to_win1252(PG_FUNCTION_ARGS)
>>>{
>>>    unsigned char *src = PG_GETARG_CSTRING(2);
>>>    unsigned char *dest = PG_GETARG_CSTRING(3);
>>>    int            len = PG_GETARG_INT32(4);
>>>
>>>    Assert(PG_GETARG_INT32(0) == PG_UTF8);
>>>    Assert(PG_GETARG_INT32(1) == PG_WIN1252);
>>>    Assert(len >= 0);
>>>
>>>    UtfToLocal(src, dest, ULmapWIN1252,
>>>               sizeof(ULmapWIN1252) / sizeof(pg_utf_to_local), len);
>>>
>>>    PG_RETURN_VOID();
>>>}
>>>
>>>Datum
>>>win1252_to_utf(PG_FUNCTION_ARGS)
>>>{
>>>    unsigned char *src = PG_GETARG_CSTRING(2);
>>>    unsigned char *dest = PG_GETARG_CSTRING(3);
>>>    int            len = PG_GETARG_INT32(4);
>>>
>>>    Assert(PG_GETARG_INT32(0) == PG_WIN1252);
>>>    Assert(PG_GETARG_INT32(1) == PG_UTF8);
>>>    Assert(len >= 0);
>>>
>>>    LocalToUtf(src, dest, LUmapWIN1252,
>>>        sizeof(LUmapWIN1252) / sizeof(pg_local_to_utf), PG_WIN1252, len);
>>>
>>>    PG_RETURN_VOID();
>>>}
>>>      
>>>
>>>static pg_utf_to_local ULmapWIN1252[ 124 ] = {
>>>  {0x0000, 0x0081},
>>>  {0xc2a0, 0x00a0},
>>>  {0xc2a1, 0x00a1},
>>>  {0xc2a2, 0x00a2},
>>>  {0xc2a3, 0x00a3},
>>>  {0xc2a4, 0x00a4},
>>>  {0xc2a5, 0x00a5},
>>>  {0xc2a6, 0x00a6},
>>>  {0xc2a7, 0x00a7},
>>>  {0xc2a8, 0x00a8},
>>>  {0xc2a9, 0x00a9},
>>>  {0xc2aa, 0x00aa},
>>>  {0xc2ab, 0x00ab},
>>>  {0xc2ac, 0x00ac},
>>>  {0xc2ad, 0x00ad},
>>>  {0xc2ae, 0x00ae},
>>>  {0xc2af, 0x00af},
>>>  {0xc2b0, 0x00b0},
>>>  {0xc2b1, 0x00b1},
>>>  {0xc2b2, 0x00b2},
>>>  {0xc2b3, 0x00b3},
>>>  {0xc2b4, 0x00b4},
>>>  {0xc2b5, 0x00b5},
>>>  {0xc2b6, 0x00b6},
>>>  {0xc2b7, 0x00b7},
>>>  {0xc2b8, 0x00b8},
>>>  {0xc2b9, 0x00b9},
>>>  {0xc2ba, 0x00ba},
>>>  {0xc2bb, 0x00bb},
>>>  {0xc2bc, 0x00bc},
>>>  {0xc2bd, 0x00bd},
>>>  {0xc2be, 0x00be},
>>>  {0xc2bf, 0x00bf},
>>>  {0xc380, 0x00c0},
>>>  {0xc381, 0x00c1},
>>>  {0xc382, 0x00c2},
>>>  {0xc383, 0x00c3},
>>>  {0xc384, 0x00c4},
>>>  {0xc385, 0x00c5},
>>>  {0xc386, 0x00c6},
>>>  {0xc387, 0x00c7},
>>>  {0xc388, 0x00c8},
>>>  {0xc389, 0x00c9},
>>>  {0xc38a, 0x00ca},
>>>  {0xc38b, 0x00cb},
>>>  {0xc38c, 0x00cc},
>>>  {0xc38d, 0x00cd},
>>>  {0xc38e, 0x00ce},
>>>  {0xc38f, 0x00cf},
>>>  {0xc390, 0x00d0},
>>>  {0xc391, 0x00d1},
>>>  {0xc392, 0x00d2},
>>>  {0xc393, 0x00d3},
>>>  {0xc394, 0x00d4},
>>>  {0xc395, 0x00d5},
>>>  {0xc396, 0x00d6},
>>>  {0xc397, 0x00d7},
>>>  {0xc398, 0x00d8},
>>>  {0xc399, 0x00d9},
>>>  {0xc39a, 0x00da},
>>>  {0xc39b, 0x00db},
>>>  {0xc39c, 0x00dc},
>>>  {0xc39d, 0x00dd},
>>>  {0xc39e, 0x00de},
>>>  {0xc39f, 0x00df},
>>>  {0xc3a0, 0x00e0},
>>>  {0xc3a1, 0x00e1},
>>>  {0xc3a2, 0x00e2},
>>>  {0xc3a3, 0x00e3},
>>>  {0xc3a4, 0x00e4},
>>>  {0xc3a5, 0x00e5},
>>>  {0xc3a6, 0x00e6},
>>>  {0xc3a7, 0x00e7},
>>>  {0xc3a8, 0x00e8},
>>>  {0xc3a9, 0x00e9},
>>>  {0xc3aa, 0x00ea},
>>>  {0xc3ab, 0x00eb},
>>>  {0xc3ac, 0x00ec},
>>>  {0xc3ad, 0x00ed},
>>>  {0xc3ae, 0x00ee},
>>>  {0xc3af, 0x00ef},
>>>  {0xc3b0, 0x00f0},
>>>  {0xc3b1, 0x00f1},
>>>  {0xc3b2, 0x00f2},
>>>  {0xc3b3, 0x00f3},
>>>  {0xc3b4, 0x00f4},
>>>  {0xc3b5, 0x00f5},
>>>  {0xc3b6, 0x00f6},
>>>  {0xc3b7, 0x00f7},
>>>  {0xc3b8, 0x00f8},
>>>  {0xc3b9, 0x00f9},
>>>  {0xc3ba, 0x00fa},
>>>  {0xc3bb, 0x00fb},
>>>  {0xc3bc, 0x00fc},
>>>  {0xc3bd, 0x00fd},
>>>  {0xc3be, 0x00fe},
>>>  {0xc3bf, 0x00ff},
>>>  {0xc592, 0x008c},
>>>  {0xc593, 0x009c},
>>>  {0xc5a0, 0x008a},
>>>  {0xc5a1, 0x009a},
>>>  {0xc5b8, 0x009f},
>>>  {0xc5bd, 0x008e},
>>>  {0xc5be, 0x009e},
>>>  {0xc692, 0x0083},
>>>  {0xcb86, 0x0088},
>>>  {0xcb9c, 0x0098},
>>>  {0xe28093, 0x0096},
>>>  {0xe28094, 0x0097},
>>>  {0xe28098, 0x0091},
>>>  {0xe28099, 0x0092},
>>>  {0xe2809a, 0x0082},
>>>  {0xe2809c, 0x0093},
>>>  {0xe2809d, 0x0094},
>>>  {0xe2809e, 0x0084},
>>>  {0xe280a0, 0x0086},
>>>  {0xe280a1, 0x0087},
>>>  {0xe280a2, 0x0095},
>>>  {0xe280a6, 0x0085},
>>>  {0xe280b0, 0x0089},
>>>  {0xe280b9, 0x008b},
>>>  {0xe280ba, 0x009b},
>>>  {0xe282ac, 0x0080},
>>>  {0xe284a2, 0x0099}
>>>};
>>>      
>>>
>>>static pg_local_to_utf LUmapWIN1250[ 128 ] = {
>>>  {0x0080, 0xe282ac},
>>>  {0x0081, 0x0000},
>>>  {0x0082, 0xe2809a},
>>>  {0x0083, 0xc692},
>>>  {0x0084, 0xe2809e},
>>>  {0x0085, 0xe280a6},
>>>  {0x0086, 0xe280a0},
>>>  {0x0087, 0xe280a1},
>>>  {0x0088, 0xcb86},
>>>  {0x0089, 0xe280b0},
>>>  {0x008a, 0xc5a0},
>>>  {0x008b, 0xe280b9},
>>>  {0x008c, 0xc592},
>>>  {0x008d, 0x0000},
>>>  {0x008e, 0xc5bd},
>>>  {0x008f, 0x0000},
>>>  {0x0090, 0x0000},
>>>  {0x0091, 0xe28098},
>>>  {0x0092, 0xe28099},
>>>  {0x0093, 0xe2809c},
>>>  {0x0094, 0xe2809d},
>>>  {0x0095, 0xe280a2},
>>>  {0x0096, 0xe28093},
>>>  {0x0097, 0xe28094},
>>>  {0x0098, 0xcb9c},
>>>  {0x0099, 0xe284a2},
>>>  {0x009a, 0xc5a1},
>>>  {0x009b, 0xe280ba},
>>>  {0x009c, 0xc593},
>>>  {0x009d, 0x0000},
>>>  {0x009e, 0xc5be},
>>>  {0x009f, 0xc5b8},
>>>  {0x00a0, 0xc2a0},
>>>  {0x00a1, 0xc2a1},
>>>  {0x00a2, 0xc2a2},
>>>  {0x00a3, 0xc2a3},
>>>  {0x00a4, 0xc2a4},
>>>  {0x00a5, 0xc2a5},
>>>  {0x00a6, 0xc2a6},
>>>  {0x00a7, 0xc2a7},
>>>  {0x00a8, 0xc2a8},
>>>  {0x00a9, 0xc2a9},
>>>  {0x00aa, 0xc2aa},
>>>  {0x00ab, 0xc2ab},
>>>  {0x00ac, 0xc2ac},
>>>  {0x00ad, 0xc2ad},
>>>  {0x00ae, 0xc2ae},
>>>  {0x00af, 0xc2af},
>>>  {0x00b0, 0xc2b0},
>>>  {0x00b1, 0xc2b1},
>>>  {0x00b2, 0xc2b2},
>>>  {0x00b3, 0xc2b3},
>>>  {0x00b4, 0xc2b4},
>>>  {0x00b5, 0xc2b5},
>>>  {0x00b6, 0xc2b6},
>>>  {0x00b7, 0xc2b7},
>>>  {0x00b8, 0xc2b8},
>>>  {0x00b9, 0xc2b9},
>>>  {0x00ba, 0xc2ba},
>>>  {0x00bb, 0xc2bb},
>>>  {0x00bc, 0xc2bc},
>>>  {0x00bd, 0xc2bd},
>>>  {0x00be, 0xc2be},
>>>  {0x00bf, 0xc2bf},
>>>  {0x00c0, 0xc380},
>>>  {0x00c1, 0xc381},
>>>  {0x00c2, 0xc382},
>>>  {0x00c3, 0xc383},
>>>  {0x00c4, 0xc384},
>>>  {0x00c5, 0xc385},
>>>  {0x00c6, 0xc386},
>>>  {0x00c7, 0xc387},
>>>  {0x00c8, 0xc388},
>>>  {0x00c9, 0xc389},
>>>  {0x00ca, 0xc38a},
>>>  {0x00cb, 0xc38b},
>>>  {0x00cc, 0xc38c},
>>>  {0x00cd, 0xc38d},
>>>  {0x00ce, 0xc38e},
>>>  {0x00cf, 0xc38f},
>>>  {0x00d0, 0xc390},
>>>  {0x00d1, 0xc391},
>>>  {0x00d2, 0xc392},
>>>  {0x00d3, 0xc393},
>>>  {0x00d4, 0xc394},
>>>  {0x00d5, 0xc395},
>>>  {0x00d6, 0xc396},
>>>  {0x00d7, 0xc397},
>>>  {0x00d8, 0xc398},
>>>  {0x00d9, 0xc399},
>>>  {0x00da, 0xc39a},
>>>  {0x00db, 0xc39b},
>>>  {0x00dc, 0xc39c},
>>>  {0x00dd, 0xc39d},
>>>  {0x00de, 0xc39e},
>>>  {0x00df, 0xc39f},
>>>  {0x00e0, 0xc3a0},
>>>  {0x00e1, 0xc3a1},
>>>  {0x00e2, 0xc3a2},
>>>  {0x00e3, 0xc3a3},
>>>  {0x00e4, 0xc3a4},
>>>  {0x00e5, 0xc3a5},
>>>  {0x00e6, 0xc3a6},
>>>  {0x00e7, 0xc3a7},
>>>  {0x00e8, 0xc3a8},
>>>  {0x00e9, 0xc3a9},
>>>  {0x00ea, 0xc3aa},
>>>  {0x00eb, 0xc3ab},
>>>  {0x00ec, 0xc3ac},
>>>  {0x00ed, 0xc3ad},
>>>  {0x00ee, 0xc3ae},
>>>  {0x00ef, 0xc3af},
>>>  {0x00f0, 0xc3b0},
>>>  {0x00f1, 0xc3b1},
>>>  {0x00f2, 0xc3b2},
>>>  {0x00f3, 0xc3b3},
>>>  {0x00f4, 0xc3b4},
>>>  {0x00f5, 0xc3b5},
>>>  {0x00f6, 0xc3b6},
>>>  {0x00f7, 0xc3b7},
>>>  {0x00f8, 0xc3b8},
>>>  {0x00f9, 0xc3b9},
>>>  {0x00fa, 0xc3ba},
>>>  {0x00fb, 0xc3bb},
>>>  {0x00fc, 0xc3bc},
>>>  {0x00fd, 0xc3bd},
>>>  {0x00fe, 0xc3be},
>>>  {0x00ff, 0xc3bf}
>>>};
>>>      
>>>
>>>---------------------------(end of broadcast)---------------------------
>>>TIP 3: if posting/reading through Usenet, please send an appropriate
>>>      subscribe-nomail command to majordomo@postgresql.org so that your
>>>      message can get through to the mailing list cleanly
>>>      
>>>
>>-- 
>>  Bruce Momjian                        |  http://candle.pha.pa.us
>>  pgman@candle.pha.pa.us               |  (610) 359-1001
>>  +  If your life is a hard drive,     |  13 Roberts Road
>>  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073
>>    
>>
>
>  
>