Re: 7.4.1 release status - Turkish Locale - Mailing list pgsql-hackers

<div class="Section1"><pre><font color="black" face="Courier New" size="2"><span style="font-size:10.0pt">> <span
class="SpellE">We</span><span class="SpellE">might</span> <span class="SpellE">think</span> <span
class="SpellE">that</span><span class="SpellE">the</span> <span class="SpellE">Turkish</span>-<span
class="SpellE">locale</span>problem Devrim <span class="SpellE">Gunduz</span> <span class="SpellE">pointed</span> <span
class="SpellE">out</span></span></font></pre><pre><fontcolor="black" face="Courier New" size="2"><span
style="font-size:10.0pt">>is a <span class="SpellE">must</span>-<span class="SpellE">fix</span>, <span
class="SpellE">too</span>.<spanstyle="mso-spacerun:yes">  </span>But <span class="SpellE">I'm</span> not <span
class="SpellE">convinced</span>yet <span class="SpellE">what</span> <span class="SpellE">to</span> do <span
class="SpellE">about</span>it.</span></font></pre><p class="MsoNormal"><font face="Arial" size="2"><span
style="font-size:10.0pt;
font-family:Arial"> </span></font><p class="MsoPlainText"><font face="Courier New" size="2"><span style="font-size:
10.0pt">Here is a <span class="SpellE">first</span> <span class="SpellE">try</span> <span class="SpellE">to</span>
<spanclass="SpellE">fix</span> <span class="SpellE">what</span> Devrim <span class="SpellE">Gunduz</span> <span
class="SpellE">talked</span><span class="SpellE">about</span>.<br /><br /><span class="SpellE">Please</span> be <span
class="SpellE">patient</span><span class="SpellE">with</span> <span class="SpellE">me</span> <span
class="SpellE">for</span>it is <span class="SpellE">the</span> <span class="SpellE">first</span> <span
class="SpellE">major</span><span class="SpellE">patch</span><br /> I <span class="SpellE">submit</span> <span
class="SpellE">and</span>I <span class="SpellE">realize</span> <span class="SpellE">that</span> I <span
class="SpellE">blatantly</span><span class="SpellE">violated</span> <span class="SpellE">many</span> <span
class="SpellE">rules</span><br/> of <span class="SpellE">good</span> <span class="SpellE">style</span> in <span
class="SpellE">PostgreSQL</span><span class="SpellE">source</span> <span class="SpellE">code</span>.<br /><br /><span
class="SpellE">First</span>,<span class="SpellE">about</span> <span class="SpellE">the</span> problem. <span
class="SpellE">Turkish</span><span class="SpellE">language</span> has <span class="SpellE">two</span> <span
class="SpellE">letters</span>"i".<br /><span class="SpellE">One</span> is <span class="SpellE">with</span> <span
class="SpellE">dot</span>on top <span class="SpellE">and</span> <span class="SpellE">the</span> <span
class="SpellE">other</span>is <span class="SpellE">without</span>. <span class="SpellE">Simply</span> as <span
class="SpellE">that</span>.<br/><span class="SpellE">The</span> <span class="SpellE">one</span> <span
class="SpellE">with</span><span class="SpellE">dot</span> has <span class="SpellE">the</span> <span
class="SpellE">dot</span><span class="SpellE">both</span> as <span class="SpellE">capital</span> <span
class="SpellE">and</span><span class="SpellE">lower</span>-<span class="SpellE">case</span> <span
class="SpellE">and</span><br/><span class="SpellE">the</span> <span class="SpellE">one</span> <span
class="SpellE">without</span><span class="SpellE">dot</span> has no <span class="SpellE">dot</span> in <span
class="SpellE">both</span><span class="SpellE">upper</span> <span class="SpellE">and</span> <span
class="SpellE">lower</span><span class="SpellE">case</span>...<br /> as <span class="SpellE">opposed</span> <span
class="SpellE">to</span><span class="SpellE">English</span> <span class="SpellE">where</span> "i" has a <span
class="SpellE">dot</span><span class="SpellE">when</span> <span class="SpellE">lower</span>-<span
class="SpellE">case</span><span class="SpellE">and</span><br /> has no <span class="SpellE">dot</span> <span
class="SpellE">when</span><span class="SpellE">upper</span>-<span class="SpellE">case</span>.<br /><br /> Problem <span
class="SpellE">arise</span><span class="SpellE">when</span> <span class="SpellE">PostgreSQL</span>, <span
class="SpellE">while</span><span class="SpellE">running</span> <span class="SpellE">with</span> "<span
class="SpellE">tr</span>_TR"<span class="SpellE">locale</span><br /><span class="SpellE">converts</span> <span
class="SpellE">to</span><span class="SpellE">lower</span>-<span class="SpellE">case</span> an <span
class="SpellE">identifier</span>as a <span class="SpellE">table</span>, an <span class="SpellE">index</span> <span
class="SpellE">or</span><br/> a <span class="SpellE">column</span> name. <span class="SpellE">If</span> it is <span
class="SpellE">written</span><span class="SpellE">with</span> <span class="SpellE">capital</span> "I", <span
class="SpellE">tolower</span>()<span class="SpellE">with</span><br /> 'I' as <span class="SpellE">argument</span> <span
class="SpellE">will</span><span class="SpellE">return</span> <span class="SpellE">Turkish</span> <span
class="SpellE">specific</span><span class="SpellE">character</span>: <br /> 'i'-<span
class="SpellE">without</span>-a-<spanclass="SpellE">dot</span> <span class="SpellE">what</span> I <span
class="SpellE">am</span><span class="SpellE">afraid</span> <span class="SpellE">will</span> not be <span
class="SpellE">shown</span><span class="SpellE">correctly</span><br /> in <span class="SpellE">your</span> e-mail <span
class="SpellE">readers</span>.<br/><br /><span class="SpellE">Let</span> <span class="SpellE">me</span> <span
class="SpellE">give</span><span class="SpellE">some</span> <span class="SpellE">examples</span>.<br /><br /><span
class="SpellE">initdb</span><span class="SpellE">script</span> <span class="SpellE">runs</span> <span
class="SpellE">apparently</span><span class="SpellE">innocent</span> <span class="SpellE">script</span> in file<br
/><spanclass="SpellE">src</span>/<span class="SpellE">backend</span>/<span class="SpellE">utils</span>/<span
class="SpellE">mb</span>/<spanclass="SpellE">conversion</span>_<span class="SpellE">procs</span>/<span
class="SpellE">conversion</span>_<spanclass="SpellE">create</span>.<span class="SpellE">sql</span><br /><span
class="SpellE">to</span><span class="SpellE">create</span> a <span class="SpellE">couple</span> of <span
class="SpellE">functions</span><span class="SpellE">whose</span> <span class="SpellE">only</span> <span
class="SpellE">fault</span><span class="SpellE">was</span><br /><span class="SpellE">to</span> <span
class="SpellE">declare</span>it <span class="SpellE">their</span> <span class="SpellE">return</span> <span
class="SpellE">parameters</span>as VOID. <span class="SpellE">Backend</span><br /><span class="SpellE">returns</span>
<spanclass="SpellE">error</span> <span class="SpellE">message</span> <span class="SpellE">that</span> <span
class="SpellE">type</span>"<span class="SpellE">vo</span> d" is not <span class="SpellE">found</span> <span
class="SpellE">and</span><br/><span class="SpellE">initdb</span> <span class="SpellE">fails</span>.<br /><br /> A <span
class="SpellE">nothing</span><span class="SpellE">suspecting</span> <span class="SpellE">novice</span> <span
class="SpellE">user</span><span class="SpellE">was</span> <span class="SpellE">excited</span> <span
class="SpellE">about</span><br/> SERIAL data <span class="SpellE">type</span> <span class="SpellE">he</span> <span
class="SpellE">was</span><span class="SpellE">tail</span> is <span class="SpellE">present</span> in <span
class="SpellE">PostgreSQL</span>.<br/><span class="SpellE">It</span> <span class="SpellE">took</span> us <span
class="SpellE">with</span>Devrim a <span class="SpellE">lot</span> of time <span class="SpellE">to</span> <span
class="SpellE">explain</span><span class="SpellE">why</span> <span class="SpellE">he</span><br /><span
class="SpellE">need</span><span class="SpellE">to</span> <span class="SpellE">type</span> SERIAL as <span
class="SpellE">SERiAL</span><span class="SpellE">for</span> <span class="SpellE">now</span> <span
class="SpellE">till</span>a <span class="SpellE">workaround</span><br /> is <span class="SpellE">developed</span>.<br
/><br/><span class="SpellE">Another</span> <span class="SpellE">case</span> <span class="SpellE">happened</span> <span
class="SpellE">with</span><span class="SpellE">me</span> <span class="SpellE">when</span> I <span
class="SpellE">wanted</span><span class="SpellE">to</span> restore<br /> a <span class="SpellE">pg</span>_<span
class="SpellE">dump</span><span class="SpellE">dump</span>. Restore <span class="SpellE">failed</span> <span
class="SpellE">because</span><span class="SpellE">script</span> <span class="SpellE">was</span> <span
class="SpellE">creating</span><br/><span class="SpellE">scripts</span> <span class="SpellE">that</span> <span
class="SpellE">belong</span><span class="SpellE">to</span> PUBLIC.<br /><br /><br /><span class="SpellE">For</span>
<spanclass="SpellE">the</span> <span class="SpellE">solution</span>, <span class="SpellE">after</span> <span
class="SpellE">some</span><span class="SpellE">research</span> <span class="SpellE">we</span> <span
class="SpellE">found</span><span class="SpellE">out</span> <span class="SpellE">that</span><br /><span
class="SpellE">offender</span>is <span class="SpellE">tolower</span>() <span class="SpellE">call</span> in <span
class="SpellE">src</span>/<spanclass="SpellE">backend</span>/<span class="SpellE">parser</span>/<span
class="SpellE">scan</span>.l<br/> in {<span class="SpellE">identifier</span>} <span class="SpellE">section</span>.
<spanclass="SpellE">tolower</span>() <span class="SpellE">works</span> <span class="SpellE">fine</span> <span
class="SpellE">with</span><span class="SpellE">any</span><br /><span class="SpellE">locale</span> <span
class="SpellE">and</span><span class="SpellE">with</span> <span class="SpellE">any</span> <span
class="SpellE">character</span><span class="SpellE">save</span> <span class="SpellE">for</span> <span
class="SpellE">the</span><span class="SpellE">Turkish</span> <span class="SpellE">locale</span><br /><span
class="SpellE">and</span><span class="SpellE">capital</span> 'I' <span class="SpellE">character</span>. <span
class="SpellE">So</span>,<span class="SpellE">the</span> <span class="SpellE">obvious</span> <span
class="SpellE">solution</span>is<br /><span class="SpellE">to</span> put a <span class="SpellE">check</span> <span
class="SpellE">for</span><span class="SpellE">Turkish</span> <span class="SpellE">locale</span> <span
class="SpellE">and</span>'I' <span class="SpellE">character</span>.<br /><span class="SpellE">Something</span> <span
class="SpellE">like</span><span class="SpellE">this</span>:<br /><br /><span class="SpellE">if</span>( <<span
class="SpellE">locale</span>is <span class="SpellE">Turkish</span>> && <span class="SpellE">ident</span>[i]
=='I' )<br /><span style="mso-tab-count:1">      </span><span class="SpellE">ident</span>[i] = 'i';<br /> else<br
/><spanstyle="mso-tab-count:1">      </span><span class="SpellE">ident</span>[i] = <span
class="SpellE">tolower</span>((<spanclass="SpellE">unsigned</span> <span class="SpellE">char</span>) <span
class="SpellE">ident</span>[i]);<br/><br /><span class="SpellE">Looks</span> <span class="SpellE">rather</span> <span
class="SpellE">simple</span>but <span class="SpellE">the</span> hard <span class="SpellE">part</span> <span
class="SpellE">was</span><span class="SpellE">to</span> <span class="SpellE">figure</span> <span
class="SpellE">out</span><br/><span class="SpellE">what</span> is <span class="SpellE">the</span> <span
class="SpellE">current</span><span class="SpellE">locale</span>. <span class="SpellE">To</span> do <span
class="SpellE">this</span>I <span class="SpellE">added</span><br /><br /><span class="SpellE">const</span> <span
class="SpellE">char</span>*<span class="SpellE">get</span>_<span class="SpellE">locale</span>_<span
class="SpellE">category</span>(<spanclass="SpellE">const</span> <span class="SpellE">char</span> *<span
class="SpellE">category</span>);<br/><br /><span class="SpellE">to</span> <span class="SpellE">src</span>/<span
class="SpellE">backend</span>/<spanclass="SpellE">utils</span>/<span class="SpellE">adt</span>/<span
class="SpellE">pg</span>_<spanclass="SpellE">locale</span>.c <span class="SpellE">that</span> <span
class="SpellE">would</span><span class="SpellE">return</span><br /><span class="SpellE">locale</span> <span
class="SpellE">identifier</span><span class="SpellE">for</span> <span class="SpellE">the</span> <span
class="SpellE">category</span><span class="SpellE">specified</span> <span class="SpellE">or</span> LC_ALL<br /><span
class="SpellE">if</span><span class="SpellE">category</span> is NULL. I <span class="SpellE">could</span> not <span
class="SpellE">find</span><span class="SpellE">any</span> <span class="SpellE">other</span> <span
class="SpellE">function</span><br/><span class="SpellE">that</span> <span class="SpellE">will</span> <span
class="SpellE">return</span><span class="SpellE">what</span> I <span class="SpellE">need</span>. <span
class="SpellE">Please</span><span class="SpellE">help</span> <span class="SpellE">me</span> <span
class="SpellE">to</span><span class="SpellE">find</span><br /><span class="SpellE">one</span> <span
class="SpellE">because</span>I <span class="SpellE">would</span> <span class="SpellE">hate</span> <span
class="SpellE">to</span><span class="SpellE">introduce</span> a <span class="SpellE">new</span> <span
class="SpellE">function</span>.<br/><br /> I <span class="SpellE">realize</span> <span class="SpellE">that</span>
{<spanclass="SpellE">identifier</span>} <span class="SpellE">section</span> is <span class="SpellE">very</span> <span
class="SpellE">performance</span><br/><span class="SpellE">critical</span> <span class="SpellE">so</span> I <span
class="SpellE">introduced</span>a global <span class="SpellE">variable</span><br /><br /><span
class="SpellE">static</span><span class="SpellE">int</span> <span class="SpellE">isturkishlocale</span> = -1;<br /><br
/>at <span class="SpellE">the</span> <span class="SpellE">beginning</span> of <span class="SpellE">src</span>/<span
class="SpellE">backend</span>/<spanclass="SpellE">parser</span>/<span class="SpellE">scan</span>.l<br /><span
class="SpellE">It</span>is set <span class="SpellE">to</span> -1 <span class="SpellE">when</span> not yet <span
class="SpellE">initialized</span>,0 <span class="SpellE">if</span><br /><span class="SpellE">locale</span> is not <span
class="SpellE">Turkish</span><span class="SpellE">and</span> 1 <span class="SpellE">if</span> <span
class="SpellE">locale</span>is <span class="SpellE">Turkish</span>.<br /><br /><span class="SpellE">It</span> <span
class="SpellE">might</span>not be <span class="SpellE">the</span> <span class="SpellE">way</span> it is <span
class="SpellE">usually</span>done in <span class="SpellE">PostgreSQL</span><br /><span class="SpellE">source</span>
<spanclass="SpellE">code</span>. <span class="SpellE">Could</span> <span class="SpellE">you</span> <span
class="SpellE">pleas</span><span class="SpellE">advise</span> <span class="SpellE">if</span> <span
class="SpellE">the</span>name I <span class="SpellE">chose</span><br /> is <span class="SpellE">appropriate</span>
<spanclass="SpellE">and</span> <span class="SpellE">whether</span> <span class="SpellE">there</span> is a <span
class="SpellE">more</span><span class="SpellE">appropriate</span><br /><span class="SpellE">place</span> <span
class="SpellE">to</span>put <span class="SpellE">declaration</span> <span class="SpellE">and</span> <span
class="SpellE">initialization</span>.<br/><br /><span class="SpellE">Best</span> <span
class="SpellE">regards</span>,<br/><span class="SpellE">Nicolai</span> <span class="SpellE">Tufar</span> & Devrim
<spanclass="SpellE">Gunduz</span></span></font><font face="Arial"><span style="font-family:Arial"></span></font><p
class="MsoNormal"><fontface="Arial" size="2"><span style="font-size:10.0pt; 
font-family:Arial"> </span></font></div>