Re: WIP patch: Collation support - Mailing list pgsql-hackers

From Heikki Linnakangas
Subject Re: WIP patch: Collation support
Date
Msg-id 48D2688C.4020408@enterprisedb.com
Whole thread Raw
In response to Re: WIP patch: Collation support  (Gregory Stark <stark@enterprisedb.com>)
Responses Re: WIP patch: Collation support  (Martijn van Oosterhout <kleptog@svana.org>)
List pgsql-hackers
Gregory Stark wrote:
> Heikki Linnakangas <heikki.linnakangas@enterprisedb.com> writes:
>
>> Well, I proposed disallowing using a different collation than the source
>> database, except for using template0 as the source. That's pretty limited, but
>> is trivial to implement and still let's you have databases with different
>> collations in the same cluster.
>
> +     if (strcmp(dbtemplate, "template0") != 0 &&
> +         (strcmp(lc_collate, src_collation) || strcmp(lc_ctype, src_ctype)))
> +         ereport(NOTICE,
> +                 (errmsg("database \"%s\" needs to be reindexed manually (REINDEX DATABASE)",
> +                         dbname)));
> +
>
> This isn't what you described but I think I prefer it this way as just a
> warning not an error.

Well, I'd prefer to make it an error, but I'm willing to listen if
others feel otherwise. I don't think the inconvenience of having to use
template0 is that big, compared to the potential of strange behavior
people would run into if they ignore the advice to reindex.

One weakness with a straight strcmp comparison is that it won't
recognize aliases of the same locale. For example, "fi_FI.UTF8" and
"fi_FI.UTF-8".

> AFAIK we can't easily connect to the new database and do some fiddling with
> it, can we? If we could we could check if there are any non-empty indexes
> which depend on the collation and only print the warning if we find any (and
> even mark them invalid).

I don't see that happening, unfortunately..

Attached is an updated version of the stripped-down patch. I've cleaned
it up a bit, and added more sanity checks. Documentation is still
missing and I haven't test it much.

--
   Heikki Linnakangas
   EnterpriseDB   http://www.enterprisedb.com
*** doc/src/sgml/ref/create_database.sgml
--- doc/src/sgml/ref/create_database.sgml
***************
*** 24,29 **** CREATE DATABASE <replaceable class="PARAMETER">name</replaceable>
--- 24,31 ----
      [ [ WITH ] [ OWNER [=] <replaceable class="parameter">dbowner</replaceable> ]
             [ TEMPLATE [=] <replaceable class="parameter">template</replaceable> ]
             [ ENCODING [=] <replaceable class="parameter">encoding</replaceable> ]
+            [ COLLATE [=] <replaceable class="parameter">collation</replaceable> ]
+            [ CTYPE [=] <replaceable class="parameter">ctype</replaceable> ]
             [ TABLESPACE [=] <replaceable class="parameter">tablespace</replaceable> ]
             [ CONNECTION LIMIT [=] <replaceable class="parameter">connlimit</replaceable> ] ]
  </synopsis>
***************
*** 113,118 **** CREATE DATABASE <replaceable class="PARAMETER">name</replaceable>
--- 115,136 ----
        </listitem>
       </varlistentry>
       <varlistentry>
+       <term><replaceable class="parameter">collation</replaceable></term>
+       <listitem>
+        <para>
+         LC_COLLATE setting to use in the new database.  XXX
+        </para>
+       </listitem>
+      </varlistentry>
+      <varlistentry>
+       <term><replaceable class="parameter">ctype</replaceable></term>
+       <listitem>
+        <para>
+         LC_CTYPE setting to use in the new database.  XXX
+        </para>
+       </listitem>
+      </varlistentry>
+      <varlistentry>
        <term><replaceable class="parameter">tablespace</replaceable></term>
        <listitem>
         <para>
*** src/backend/access/transam/xlog.c
--- src/backend/access/transam/xlog.c
***************
*** 3847,3853 **** WriteControlFile(void)
  {
      int            fd;
      char        buffer[PG_CONTROL_SIZE];        /* need not be aligned */
-     char       *localeptr;

      /*
       * Initialize version and compatibility-check fields
--- 3847,3852 ----
***************
*** 3876,3893 **** WriteControlFile(void)
      ControlFile->float4ByVal = FLOAT4PASSBYVAL;
      ControlFile->float8ByVal = FLOAT8PASSBYVAL;

-     ControlFile->localeBuflen = LOCALE_NAME_BUFLEN;
-     localeptr = setlocale(LC_COLLATE, NULL);
-     if (!localeptr)
-         ereport(PANIC,
-                 (errmsg("invalid LC_COLLATE setting")));
-     StrNCpy(ControlFile->lc_collate, localeptr, LOCALE_NAME_BUFLEN);
-     localeptr = setlocale(LC_CTYPE, NULL);
-     if (!localeptr)
-         ereport(PANIC,
-                 (errmsg("invalid LC_CTYPE setting")));
-     StrNCpy(ControlFile->lc_ctype, localeptr, LOCALE_NAME_BUFLEN);
-
      /* Contents are protected with a CRC */
      INIT_CRC32(ControlFile->crc);
      COMP_CRC32(ControlFile->crc,
--- 3875,3880 ----
***************
*** 4126,4159 **** ReadControlFile(void)
                             " but the server was compiled without USE_FLOAT8_BYVAL."),
                   errhint("It looks like you need to recompile or initdb.")));
  #endif
-
-     if (ControlFile->localeBuflen != LOCALE_NAME_BUFLEN)
-         ereport(FATAL,
-                 (errmsg("database files are incompatible with server"),
-                  errdetail("The database cluster was initialized with LOCALE_NAME_BUFLEN %d,"
-                   " but the server was compiled with LOCALE_NAME_BUFLEN %d.",
-                            ControlFile->localeBuflen, LOCALE_NAME_BUFLEN),
-                  errhint("It looks like you need to recompile or initdb.")));
-     if (pg_perm_setlocale(LC_COLLATE, ControlFile->lc_collate) == NULL)
-         ereport(FATAL,
-             (errmsg("database files are incompatible with operating system"),
-              errdetail("The database cluster was initialized with LC_COLLATE \"%s\","
-                        " which is not recognized by setlocale().",
-                        ControlFile->lc_collate),
-              errhint("It looks like you need to initdb or install locale support.")));
-     if (pg_perm_setlocale(LC_CTYPE, ControlFile->lc_ctype) == NULL)
-         ereport(FATAL,
-             (errmsg("database files are incompatible with operating system"),
-         errdetail("The database cluster was initialized with LC_CTYPE \"%s\","
-                   " which is not recognized by setlocale().",
-                   ControlFile->lc_ctype),
-              errhint("It looks like you need to initdb or install locale support.")));
-
-     /* Make the fixed locale settings visible as GUC variables, too */
-     SetConfigOption("lc_collate", ControlFile->lc_collate,
-                     PGC_INTERNAL, PGC_S_OVERRIDE);
-     SetConfigOption("lc_ctype", ControlFile->lc_ctype,
-                     PGC_INTERNAL, PGC_S_OVERRIDE);
  }

  void
--- 4113,4118 ----
*** src/backend/commands/dbcommands.c
--- src/backend/commands/dbcommands.c
***************
*** 53,58 ****
--- 53,59 ----
  #include "utils/fmgroids.h"
  #include "utils/guc.h"
  #include "utils/lsyscache.h"
+ #include "utils/pg_locale.h"
  #include "utils/syscache.h"
  #include "utils/tqual.h"

***************
*** 69,75 **** static bool get_db_info(const char *name, LOCKMODE lockmode,
              Oid *dbIdP, Oid *ownerIdP,
              int *encodingP, bool *dbIsTemplateP, bool *dbAllowConnP,
              Oid *dbLastSysOidP, TransactionId *dbFrozenXidP,
!             Oid *dbTablespace);
  static bool have_createdb_privilege(void);
  static void remove_dbtablespaces(Oid db_id);
  static bool check_db_file_conflict(Oid db_id);
--- 70,76 ----
              Oid *dbIdP, Oid *ownerIdP,
              int *encodingP, bool *dbIsTemplateP, bool *dbAllowConnP,
              Oid *dbLastSysOidP, TransactionId *dbFrozenXidP,
!             Oid *dbTablespace, char **dbCollation, char **dbCtype);
  static bool have_createdb_privilege(void);
  static void remove_dbtablespaces(Oid db_id);
  static bool check_db_file_conflict(Oid db_id);
***************
*** 87,92 **** createdb(const CreatedbStmt *stmt)
--- 88,95 ----
      Oid            src_dboid;
      Oid            src_owner;
      int            src_encoding;
+     char       *src_collation;
+     char       *src_ctype;
      bool        src_istemplate;
      bool        src_allowconn;
      Oid            src_lastsysoid;
***************
*** 104,113 **** createdb(const CreatedbStmt *stmt)
--- 107,120 ----
      DefElem    *downer = NULL;
      DefElem    *dtemplate = NULL;
      DefElem    *dencoding = NULL;
+     DefElem    *dcollation = NULL;
+     DefElem    *dctype = NULL;
      DefElem    *dconnlimit = NULL;
      char       *dbname = stmt->dbname;
      char       *dbowner = NULL;
      const char *dbtemplate = NULL;
+     char       *lc_collate = NULL;
+     char       *lc_ctype = NULL;
      int            encoding = -1;
      int            dbconnlimit = -1;
      int            ctype_encoding;
***************
*** 152,157 **** createdb(const CreatedbStmt *stmt)
--- 159,180 ----
                           errmsg("conflicting or redundant options")));
              dencoding = defel;
          }
+         else if (strcmp(defel->defname, "collate") == 0)
+         {
+             if (dcollation)
+                 ereport(ERROR,
+                         (errcode(ERRCODE_SYNTAX_ERROR),
+                          errmsg("conflicting or redundant options")));
+             dcollation = defel;
+         }
+         else if (strcmp(defel->defname, "ctype") == 0)
+         {
+             if (dctype)
+                 ereport(ERROR,
+                         (errcode(ERRCODE_SYNTAX_ERROR),
+                          errmsg("conflicting or redundant options")));
+             dctype = defel;
+         }
          else if (strcmp(defel->defname, "connectionlimit") == 0)
          {
              if (dconnlimit)
***************
*** 205,210 **** createdb(const CreatedbStmt *stmt)
--- 228,238 ----
              elog(ERROR, "unrecognized node type: %d",
                   nodeTag(dencoding->arg));
      }
+     if (dcollation && dcollation->arg)
+         lc_collate = strVal(dcollation->arg);
+     if (dctype && dctype->arg)
+         lc_ctype = strVal(dctype->arg);
+
      if (dconnlimit && dconnlimit->arg)
          dbconnlimit = intVal(dconnlimit->arg);

***************
*** 243,249 **** createdb(const CreatedbStmt *stmt)
      if (!get_db_info(dbtemplate, ShareLock,
                       &src_dboid, &src_owner, &src_encoding,
                       &src_istemplate, &src_allowconn, &src_lastsysoid,
!                      &src_frozenxid, &src_deftablespace))
          ereport(ERROR,
                  (errcode(ERRCODE_UNDEFINED_DATABASE),
                   errmsg("template database \"%s\" does not exist",
--- 271,278 ----
      if (!get_db_info(dbtemplate, ShareLock,
                       &src_dboid, &src_owner, &src_encoding,
                       &src_istemplate, &src_allowconn, &src_lastsysoid,
!                      &src_frozenxid, &src_deftablespace,
!                      &src_collation, &src_ctype))
          ereport(ERROR,
                  (errcode(ERRCODE_UNDEFINED_DATABASE),
                   errmsg("template database \"%s\" does not exist",
***************
*** 262,270 **** createdb(const CreatedbStmt *stmt)
                              dbtemplate)));
      }

!     /* If encoding is defaulted, use source's encoding */
      if (encoding < 0)
          encoding = src_encoding;

      /* Some encodings are client only */
      if (!PG_VALID_BE_ENCODING(encoding))
--- 291,303 ----
                              dbtemplate)));
      }

!     /* If encoding or locales are defaulted, use source's setting */
      if (encoding < 0)
          encoding = src_encoding;
+     if (lc_collate == NULL)
+         lc_collate = src_collation;
+     if (lc_ctype == NULL)
+         lc_ctype = src_ctype;

      /* Some encodings are client only */
      if (!PG_VALID_BE_ENCODING(encoding))
***************
*** 272,277 **** createdb(const CreatedbStmt *stmt)
--- 305,320 ----
                  (errcode(ERRCODE_WRONG_OBJECT_TYPE),
                   errmsg("invalid server encoding %d", encoding)));

+     /* Check that the chosen locales are valid */
+     if (!check_locale(LC_COLLATE, lc_collate))
+         ereport(ERROR,
+                 (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+                  errmsg("invalid locale name %s", lc_collate)));
+     if (!check_locale(LC_CTYPE, lc_ctype))
+         ereport(ERROR,
+                 (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+                  errmsg("invalid locale name %s", lc_ctype)));
+
      /*
       * Check whether encoding matches server locale settings.  We allow
       * mismatch in three cases:
***************
*** 290,296 **** createdb(const CreatedbStmt *stmt)
       *
       * Note: if you change this policy, fix initdb to match.
       */
!     ctype_encoding = pg_get_encoding_from_locale(NULL);

      if (!(ctype_encoding == encoding ||
            ctype_encoding == PG_SQL_ASCII ||
--- 333,339 ----
       *
       * Note: if you change this policy, fix initdb to match.
       */
!     ctype_encoding = pg_get_encoding_from_locale(lc_ctype);

      if (!(ctype_encoding == encoding ||
            ctype_encoding == PG_SQL_ASCII ||
***************
*** 299,310 **** createdb(const CreatedbStmt *stmt)
  #endif
            (encoding == PG_SQL_ASCII && superuser())))
          ereport(ERROR,
!                 (errmsg("encoding %s does not match server's locale %s",
                          pg_encoding_to_char(encoding),
!                         setlocale(LC_CTYPE, NULL)),
!              errdetail("The server's LC_CTYPE setting requires encoding %s.",
                         pg_encoding_to_char(ctype_encoding))));

      /* Resolve default tablespace for new database */
      if (dtablespacename && dtablespacename->arg)
      {
--- 342,373 ----
  #endif
            (encoding == PG_SQL_ASCII && superuser())))
          ereport(ERROR,
!                 (errmsg("encoding %s does not match locale %s",
                          pg_encoding_to_char(encoding),
!                         lc_ctype),
!              errdetail("The chosen LC_CTYPE setting requires encoding %s.",
                         pg_encoding_to_char(ctype_encoding))));

+     /*
+      * Check that the new locale is compatible with the source database.
+      *
+      * We know that template0 doesn't contain any indexes that depend on
+      * collation or ctype, so template0 can be used as template for
+      * any locale.
+      */
+     if (strcmp(dbtemplate, "template0") != 0)
+     {
+         if (strcmp(lc_collate, src_collation))
+             ereport(ERROR,
+                     (errmsg("new collation is incompatible with the collation of the template database (%s)",
src_collation),
+                      errhint("Use the same collation as in the template database, or use template0 as template")));
+
+         if (strcmp(lc_ctype, src_ctype))
+             ereport(ERROR,
+                     (errmsg("new ctype is incompatible with the ctype of the template database (%s)", src_ctype),
+                      errhint("Use the same ctype as in the template database, or use template0 as template")));
+     }
+
      /* Resolve default tablespace for new database */
      if (dtablespacename && dtablespacename->arg)
      {
***************
*** 421,426 **** createdb(const CreatedbStmt *stmt)
--- 484,491 ----
          DirectFunctionCall1(namein, CStringGetDatum(dbname));
      new_record[Anum_pg_database_datdba - 1] = ObjectIdGetDatum(datdba);
      new_record[Anum_pg_database_encoding - 1] = Int32GetDatum(encoding);
+     new_record[Anum_pg_database_collation - 1] = DirectFunctionCall1(namein, CStringGetDatum(lc_collate));
+     new_record[Anum_pg_database_ctype - 1] = DirectFunctionCall1(namein, CStringGetDatum(lc_ctype));
      new_record[Anum_pg_database_datistemplate - 1] = BoolGetDatum(false);
      new_record[Anum_pg_database_datallowconn - 1] = BoolGetDatum(true);
      new_record[Anum_pg_database_datconnlimit - 1] = Int32GetDatum(dbconnlimit);
***************
*** 629,635 **** dropdb(const char *dbname, bool missing_ok)
      pgdbrel = heap_open(DatabaseRelationId, RowExclusiveLock);

      if (!get_db_info(dbname, AccessExclusiveLock, &db_id, NULL, NULL,
!                      &db_istemplate, NULL, NULL, NULL, NULL))
      {
          if (!missing_ok)
          {
--- 694,700 ----
      pgdbrel = heap_open(DatabaseRelationId, RowExclusiveLock);

      if (!get_db_info(dbname, AccessExclusiveLock, &db_id, NULL, NULL,
!                      &db_istemplate, NULL, NULL, NULL, NULL, NULL, NULL))
      {
          if (!missing_ok)
          {
***************
*** 781,787 **** RenameDatabase(const char *oldname, const char *newname)
      rel = heap_open(DatabaseRelationId, RowExclusiveLock);

      if (!get_db_info(oldname, AccessExclusiveLock, &db_id, NULL, NULL,
!                      NULL, NULL, NULL, NULL, NULL))
          ereport(ERROR,
                  (errcode(ERRCODE_UNDEFINED_DATABASE),
                   errmsg("database \"%s\" does not exist", oldname)));
--- 846,852 ----
      rel = heap_open(DatabaseRelationId, RowExclusiveLock);

      if (!get_db_info(oldname, AccessExclusiveLock, &db_id, NULL, NULL,
!                      NULL, NULL, NULL, NULL, NULL, NULL, NULL))
          ereport(ERROR,
                  (errcode(ERRCODE_UNDEFINED_DATABASE),
                   errmsg("database \"%s\" does not exist", oldname)));
***************
*** 1168,1174 **** get_db_info(const char *name, LOCKMODE lockmode,
              Oid *dbIdP, Oid *ownerIdP,
              int *encodingP, bool *dbIsTemplateP, bool *dbAllowConnP,
              Oid *dbLastSysOidP, TransactionId *dbFrozenXidP,
!             Oid *dbTablespace)
  {
      bool        result = false;
      Relation    relation;
--- 1233,1239 ----
              Oid *dbIdP, Oid *ownerIdP,
              int *encodingP, bool *dbIsTemplateP, bool *dbAllowConnP,
              Oid *dbLastSysOidP, TransactionId *dbFrozenXidP,
!             Oid *dbTablespace, char **dbCollation, char **dbCtype)
  {
      bool        result = false;
      Relation    relation;
***************
*** 1259,1264 **** get_db_info(const char *name, LOCKMODE lockmode,
--- 1324,1334 ----
                  /* default tablespace for this database */
                  if (dbTablespace)
                      *dbTablespace = dbform->dattablespace;
+                  /* default locale settings for this database */
+                  if (dbCollation)
+                      *dbCollation = pstrdup(NameStr(dbform->collation));
+                  if (dbCtype)
+                      *dbCtype = pstrdup(NameStr(dbform->ctype));
                  ReleaseSysCache(tuple);
                  result = true;
                  break;
*** src/backend/parser/gram.y
--- src/backend/parser/gram.y
***************
*** 398,404 **** static TypeName *TableFuncTypeName(List *columns);
      CLUSTER COALESCE COLLATE COLUMN COMMENT COMMIT
      COMMITTED CONCURRENTLY CONFIGURATION CONNECTION CONSTRAINT CONSTRAINTS
      CONTENT_P CONTINUE_P CONVERSION_P COPY COST CREATE CREATEDB
!     CREATEROLE CREATEUSER CROSS CSV CURRENT_P CURRENT_DATE CURRENT_ROLE
      CURRENT_TIME CURRENT_TIMESTAMP CURRENT_USER CURSOR CYCLE

      DATABASE DAY_P DEALLOCATE DEC DECIMAL_P DECLARE DEFAULT DEFAULTS
--- 398,404 ----
      CLUSTER COALESCE COLLATE COLUMN COMMENT COMMIT
      COMMITTED CONCURRENTLY CONFIGURATION CONNECTION CONSTRAINT CONSTRAINTS
      CONTENT_P CONTINUE_P CONVERSION_P COPY COST CREATE CREATEDB
!     CREATEROLE CREATEUSER CROSS CSV CTYPE CURRENT_P CURRENT_DATE CURRENT_ROLE
      CURRENT_TIME CURRENT_TIMESTAMP CURRENT_USER CURSOR CYCLE

      DATABASE DAY_P DEALLOCATE DEC DECIMAL_P DECLARE DEFAULT DEFAULTS
***************
*** 5458,5463 **** createdb_opt_item:
--- 5458,5479 ----
                  {
                      $$ = makeDefElem("encoding", NULL);
                  }
+             | COLLATE opt_equal Sconst
+                 {
+                     $$ = makeDefElem("collate", (Node *)makeString($3));
+                 }
+             | COLLATE opt_equal DEFAULT
+                 {
+                     $$ = makeDefElem("collate", NULL);
+                 }
+             | CTYPE opt_equal Sconst
+                 {
+                     $$ = makeDefElem("ctype", (Node *)makeString($3));
+                 }
+             | CTYPE opt_equal DEFAULT
+                 {
+                     $$ = makeDefElem("ctype", NULL);
+                 }
              | CONNECTION LIMIT opt_equal SignedIconst
                  {
                      $$ = makeDefElem("connectionlimit", (Node *)makeInteger($4));
***************
*** 9216,9221 **** unreserved_keyword:
--- 9232,9238 ----
              | CREATEROLE
              | CREATEUSER
              | CSV
+             | CTYPE
              | CURRENT_P
              | CURSOR
              | CYCLE
*** src/backend/parser/keywords.c
--- src/backend/parser/keywords.c
***************
*** 114,119 **** const ScanKeyword ScanKeywords[] = {
--- 114,120 ----
      {"createuser", CREATEUSER, UNRESERVED_KEYWORD},
      {"cross", CROSS, TYPE_FUNC_NAME_KEYWORD},
      {"csv", CSV, UNRESERVED_KEYWORD},
+     {"ctype", CTYPE, UNRESERVED_KEYWORD},
      {"current", CURRENT_P, UNRESERVED_KEYWORD},
      {"current_date", CURRENT_DATE, RESERVED_KEYWORD},
      {"current_role", CURRENT_ROLE, RESERVED_KEYWORD},
*** src/backend/utils/adt/pg_locale.c
--- src/backend/utils/adt/pg_locale.c
***************
*** 189,194 **** pg_perm_setlocale(int category, const char *locale)
--- 189,218 ----
  }


+ /*
+  * Is the locale name valid for the locale category?
+  */
+ bool
+ check_locale(int category, const char *value)
+ {
+     char       *save;
+     bool        ret;
+
+     save = setlocale(category, NULL);
+     if (!save)
+         return false;            /* won't happen, we hope */
+
+     /* save may be pointing at a modifiable scratch variable, see above */
+     save = pstrdup(save);
+
+     ret = (setlocale(category, value) != NULL);
+
+     setlocale(category, save);    /* assume this won't fail */
+     pfree(save);
+
+     return ret;
+ }
+
  /* GUC assign hooks */

  /*
***************
*** 203,223 **** pg_perm_setlocale(int category, const char *locale)
  static const char *
  locale_xxx_assign(int category, const char *value, bool doit, GucSource source)
  {
!     char       *save;
!
!     save = setlocale(category, NULL);
!     if (!save)
!         return NULL;            /* won't happen, we hope */
!
!     /* save may be pointing at a modifiable scratch variable, see above */
!     save = pstrdup(save);
!
!     if (!setlocale(category, value))
          value = NULL;            /* set failure return marker */

-     setlocale(category, save);    /* assume this won't fail */
-     pfree(save);
-
      /* need to reload cache next time? */
      if (doit && value != NULL)
      {
--- 227,235 ----
  static const char *
  locale_xxx_assign(int category, const char *value, bool doit, GucSource source)
  {
!     if (!check_locale(category, value))
          value = NULL;            /* set failure return marker */

      /* need to reload cache next time? */
      if (doit && value != NULL)
      {
*** src/backend/utils/init/postinit.c
--- src/backend/utils/init/postinit.c
***************
*** 159,164 **** CheckMyDatabase(const char *name, bool am_superuser)
--- 159,166 ----
  {
      HeapTuple    tup;
      Form_pg_database dbform;
+     char       *collate;
+     char       *ctype;

      /* Fetch our real pg_database row */
      tup = SearchSysCache(DATABASEOID,
***************
*** 240,245 **** CheckMyDatabase(const char *name, bool am_superuser)
--- 242,269 ----
      /* If we have no other source of client_encoding, use server encoding */
      SetConfigOption("client_encoding", GetDatabaseEncodingName(),
                      PGC_BACKEND, PGC_S_DEFAULT);
+
+     /* assign locale variables */
+     collate = NameStr(dbform->collation);
+     ctype = NameStr(dbform->ctype);
+
+     if (setlocale(LC_COLLATE, collate) == NULL)
+         ereport(FATAL,
+             (errmsg("database locale is incompatible with operating system"),
+             errdetail("The database was initialized with LC_COLLATE \"%s\", "
+                         " which is not recognized by setlocale().", collate),
+             errhint("Recreate the database with another locale or install the missing locale.")));
+
+     if (setlocale(LC_CTYPE, ctype) == NULL)
+         ereport(FATAL,
+             (errmsg("database locale is incompatible with operating system"),
+             errdetail("The database was initialized with LC_CTYPE \"%s\", "
+                         " which is not recognized by setlocale().", ctype),
+             errhint("Recreate the database with another locale or install the missing locale.")));
+
+     /* Make the locale settings visible as GUC variables, too */
+     SetConfigOption("lc_collate", collate, PGC_INTERNAL, PGC_S_DATABASE);
+     SetConfigOption("lc_ctype", ctype, PGC_INTERNAL, PGC_S_DATABASE);

      /*
       * Lastly, set up any database-specific configuration variables.
*** src/bin/initdb/initdb.c
--- src/bin/initdb/initdb.c
***************
*** 1353,1358 **** bootstrap_template1(char *short_version)
--- 1353,1362 ----

      bki_lines = replace_token(bki_lines, "ENCODING", encodingid);

+     bki_lines = replace_token(bki_lines, "LC_COLLATE", lc_collate);
+
+     bki_lines = replace_token(bki_lines, "LC_CTYPE", lc_ctype);
+
      /*
       * Pass correct LC_xxx environment to bootstrap.
       *
***************
*** 2378,2389 **** usage(const char *progname)
      printf(_("\nOptions:\n"));
      printf(_(" [-D, --pgdata=]DATADIR     location for this database cluster\n"));
      printf(_("  -E, --encoding=ENCODING   set default encoding for new databases\n"));
!     printf(_("  --locale=LOCALE           initialize database cluster with given locale\n"));
      printf(_("  --lc-collate, --lc-ctype, --lc-messages=LOCALE\n"
               "  --lc-monetary, --lc-numeric, --lc-time=LOCALE\n"
!              "                            initialize database cluster with given locale\n"
!              "                            in the respective category (default taken from\n"
!              "                            environment)\n"));
      printf(_("  --no-locale               equivalent to --locale=C\n"));
      printf(_("  -T, --text-search-config=CFG\n"
           "                            default text search configuration\n"));
--- 2382,2393 ----
      printf(_("\nOptions:\n"));
      printf(_(" [-D, --pgdata=]DATADIR     location for this database cluster\n"));
      printf(_("  -E, --encoding=ENCODING   set default encoding for new databases\n"));
!     printf(_("  --locale=LOCALE           set default locale for new databases\n"));
      printf(_("  --lc-collate, --lc-ctype, --lc-messages=LOCALE\n"
               "  --lc-monetary, --lc-numeric, --lc-time=LOCALE\n"
!              "                            set default locale in the respective\n"
!              "                            category for new databases (default\n"
!              "                            taken from environment)\n"));
      printf(_("  --no-locale               equivalent to --locale=C\n"));
      printf(_("  -T, --text-search-config=CFG\n"
           "                            default text search configuration\n"));
***************
*** 2806,2815 **** main(int argc, char *argv[])
          strcmp(lc_ctype, lc_numeric) == 0 &&
          strcmp(lc_ctype, lc_monetary) == 0 &&
          strcmp(lc_ctype, lc_messages) == 0)
!         printf(_("The database cluster will be initialized with locale %s.\n"), lc_ctype);
      else
      {
!         printf(_("The database cluster will be initialized with locales\n"
                   "  COLLATE:  %s\n"
                   "  CTYPE:    %s\n"
                   "  MESSAGES: %s\n"
--- 2810,2821 ----
          strcmp(lc_ctype, lc_numeric) == 0 &&
          strcmp(lc_ctype, lc_monetary) == 0 &&
          strcmp(lc_ctype, lc_messages) == 0)
!         printf(_("The template databases will be initialized with locale %s.\n"), lc_ctype);
      else
      {
!         /* XXX only collate and ctype are actually set in stone here, others
!          * are userset gucs */
!         printf(_("The template databases will be initialized with locales\n"
                   "  COLLATE:  %s\n"
                   "  CTYPE:    %s\n"
                   "  MESSAGES: %s\n"
*** src/bin/pg_controldata/pg_controldata.c
--- src/bin/pg_controldata/pg_controldata.c
***************
*** 220,231 **** main(int argc, char *argv[])
             (ControlFile.float4ByVal ? _("by value") : _("by reference")));
      printf(_("Float8 argument passing:              %s\n"),
             (ControlFile.float8ByVal ? _("by value") : _("by reference")));
-     printf(_("Maximum length of locale name:        %u\n"),
-            ControlFile.localeBuflen);
-     printf(_("LC_COLLATE:                           %s\n"),
-            ControlFile.lc_collate);
-     printf(_("LC_CTYPE:                             %s\n"),
-            ControlFile.lc_ctype);
-
      return 0;
  }
--- 220,224 ----
*** src/bin/pg_resetxlog/pg_resetxlog.c
--- src/bin/pg_resetxlog/pg_resetxlog.c
***************
*** 493,514 **** GuessControlValues(void)
  #endif
      ControlFile.float4ByVal = FLOAT4PASSBYVAL;
      ControlFile.float8ByVal = FLOAT8PASSBYVAL;
-     ControlFile.localeBuflen = LOCALE_NAME_BUFLEN;
-
-     localeptr = setlocale(LC_COLLATE, "");
-     if (!localeptr)
-     {
-         fprintf(stderr, _("%s: invalid LC_COLLATE setting\n"), progname);
-         exit(1);
-     }
-     strlcpy(ControlFile.lc_collate, localeptr, sizeof(ControlFile.lc_collate));
-     localeptr = setlocale(LC_CTYPE, "");
-     if (!localeptr)
-     {
-         fprintf(stderr, _("%s: invalid LC_CTYPE setting\n"), progname);
-         exit(1);
-     }
-     strlcpy(ControlFile.lc_ctype, localeptr, sizeof(ControlFile.lc_ctype));

      /*
       * XXX eventually, should try to grovel through old XLOG to develop more
--- 493,498 ----
***************
*** 584,595 **** PrintControlValues(bool guessed)
             (ControlFile.float4ByVal ? _("by value") : _("by reference")));
      printf(_("Float8 argument passing:              %s\n"),
             (ControlFile.float8ByVal ? _("by value") : _("by reference")));
-     printf(_("Maximum length of locale name:        %u\n"),
-            ControlFile.localeBuflen);
-     printf(_("LC_COLLATE:                           %s\n"),
-            ControlFile.lc_collate);
-     printf(_("LC_CTYPE:                             %s\n"),
-            ControlFile.lc_ctype);
  }


--- 568,573 ----
*** src/bin/scripts/createdb.c
--- src/bin/scripts/createdb.c
***************
*** 32,37 **** main(int argc, char *argv[])
--- 32,39 ----
          {"tablespace", required_argument, NULL, 'D'},
          {"template", required_argument, NULL, 'T'},
          {"encoding", required_argument, NULL, 'E'},
+         {"lc-collate", required_argument, NULL, 1},
+         {"lc-ctype", required_argument, NULL, 2},
          {NULL, 0, NULL, 0}
      };

***************
*** 50,55 **** main(int argc, char *argv[])
--- 52,59 ----
      char       *tablespace = NULL;
      char       *template = NULL;
      char       *encoding = NULL;
+     char       *lc_collate = NULL;
+     char       *lc_ctype = NULL;

      PQExpBufferData sql;

***************
*** 95,100 **** main(int argc, char *argv[])
--- 99,110 ----
              case 'E':
                  encoding = optarg;
                  break;
+             case 1:
+                 lc_collate = optarg;
+                 break;
+             case 2:
+                 lc_ctype = optarg;
+                 break;
              default:
                  fprintf(stderr, _("Try \"%s --help\" for more information.\n"), progname);
                  exit(1);
***************
*** 152,157 **** main(int argc, char *argv[])
--- 162,172 ----
          appendPQExpBuffer(&sql, " ENCODING '%s'", encoding);
      if (template)
          appendPQExpBuffer(&sql, " TEMPLATE %s", fmtId(template));
+     if (lc_collate)
+         appendPQExpBuffer(&sql, " LCCOLLATE %s", fmtId(lc_collate));
+     if (lc_ctype)
+         appendPQExpBuffer(&sql, " LCCTYPE %s", fmtId(lc_ctype));
+
      appendPQExpBuffer(&sql, ";\n");

      conn = connectDatabase(strcmp(dbname, "postgres") == 0 ? "template1" : "postgres",
***************
*** 209,214 **** help(const char *progname)
--- 224,232 ----
      printf(_("\nOptions:\n"));
      printf(_("  -D, --tablespace=TABLESPACE  default tablespace for the database\n"));
      printf(_("  -E, --encoding=ENCODING      encoding for the database\n"));
+     printf(_("    --lc-collate=LOCALE          LC_COLLATE setting for the database\n"));
+     printf(_("    --lc-ctype=LOCALE            LC_CTYPE setting for the database\n"));
+
      printf(_("  -O, --owner=OWNER            database user to own the new database\n"));
      printf(_("  -T, --template=TEMPLATE      template database to copy\n"));
      printf(_("  -e, --echo                   show the commands being sent to the server\n"));
*** src/include/catalog/pg_control.h
--- src/include/catalog/pg_control.h
***************
*** 144,154 **** typedef struct ControlFileData
      bool        float4ByVal;    /* float4 pass-by-value? */
      bool        float8ByVal;    /* float8, int8, etc pass-by-value? */

-     /* active locales */
-     uint32        localeBuflen;
-     char        lc_collate[LOCALE_NAME_BUFLEN];
-     char        lc_ctype[LOCALE_NAME_BUFLEN];
-
      /* CRC of all above ... MUST BE LAST! */
      pg_crc32    crc;
  } ControlFileData;
--- 144,149 ----
*** src/include/catalog/pg_database.h
--- src/include/catalog/pg_database.h
***************
*** 33,38 **** CATALOG(pg_database,1262) BKI_SHARED_RELATION
--- 33,40 ----
      NameData    datname;        /* database name */
      Oid            datdba;            /* owner of database */
      int4        encoding;        /* character encoding */
+     NameData    collation;        /* LC_COLLATE of database */
+     NameData    ctype;            /* LC_CTYPE of database */
      bool        datistemplate;    /* allowed as CREATE DATABASE template? */
      bool        datallowconn;    /* new connections allowed? */
      int4        datconnlimit;    /* max connections allowed (-1=no limit) */
***************
*** 54,73 **** typedef FormData_pg_database *Form_pg_database;
   *        compiler constants for pg_database
   * ----------------
   */
! #define Natts_pg_database                11
  #define Anum_pg_database_datname        1
  #define Anum_pg_database_datdba            2
  #define Anum_pg_database_encoding        3
! #define Anum_pg_database_datistemplate    4
! #define Anum_pg_database_datallowconn    5
! #define Anum_pg_database_datconnlimit    6
! #define Anum_pg_database_datlastsysoid    7
! #define Anum_pg_database_datfrozenxid    8
! #define Anum_pg_database_dattablespace    9
! #define Anum_pg_database_datconfig        10
! #define Anum_pg_database_datacl            11

! DATA(insert OID = 1 (  template1 PGUID ENCODING t t -1 0 0 1663 _null_ _null_ ));
  SHDESCR("default template database");
  #define TemplateDbOid            1

--- 56,77 ----
   *        compiler constants for pg_database
   * ----------------
   */
! #define Natts_pg_database                13
  #define Anum_pg_database_datname        1
  #define Anum_pg_database_datdba            2
  #define Anum_pg_database_encoding        3
! #define Anum_pg_database_collation        4
! #define Anum_pg_database_ctype            5
! #define Anum_pg_database_datistemplate    6
! #define Anum_pg_database_datallowconn    7
! #define Anum_pg_database_datconnlimit    8
! #define Anum_pg_database_datlastsysoid    9
! #define Anum_pg_database_datfrozenxid    10
! #define Anum_pg_database_dattablespace    11
! #define Anum_pg_database_datconfig        12
! #define Anum_pg_database_datacl            13

! DATA(insert OID = 1 (  template1 PGUID ENCODING "LC_COLLATE" "LC_CTYPE" t t -1 0 0 1663 _null_ _null_));
  SHDESCR("default template database");
  #define TemplateDbOid            1

*** src/include/utils/pg_locale.h
--- src/include/utils/pg_locale.h
***************
*** 39,44 **** extern const char *locale_numeric_assign(const char *value,
--- 39,45 ----
  extern const char *locale_time_assign(const char *value,
                     bool doit, GucSource source);

+ extern bool check_locale(int category, const char *locale);
  extern char *pg_perm_setlocale(int category, const char *locale);

  extern bool lc_collate_is_c(void);
*** src/interfaces/ecpg/preproc/preproc.y
--- src/interfaces/ecpg/preproc/preproc.y
***************
*** 428,434 **** add_typedef(char *name, char * dimension, char * length, enum ECPGttype type_enu
      CLUSTER COALESCE COLLATE COLUMN COMMENT COMMIT
      COMMITTED CONCURRENTLY CONFIGURATION CONNECTION CONSTRAINT CONSTRAINTS
      CONTENT_P CONTINUE_P CONVERSION_P COPY COST CREATE CREATEDB
!     CREATEROLE CREATEUSER CROSS CSV CURRENT_P CURRENT_DATE CURRENT_ROLE
      CURRENT_TIME CURRENT_TIMESTAMP CURRENT_USER CURSOR CYCLE

      DATABASE DAY_P DEALLOCATE DEC DECIMAL_P DECLARE DEFAULT DEFAULTS
--- 428,434 ----
      CLUSTER COALESCE COLLATE COLUMN COMMENT COMMIT
      COMMITTED CONCURRENTLY CONFIGURATION CONNECTION CONSTRAINT CONSTRAINTS
      CONTENT_P CONTINUE_P CONVERSION_P COPY COST CREATE CREATEDB
!     CREATEROLE CREATEUSER CROSS CSV CTYPE CURRENT_P CURRENT_DATE CURRENT_ROLE
      CURRENT_TIME CURRENT_TIMESTAMP CURRENT_USER CURSOR CYCLE

      DATABASE DAY_P DEALLOCATE DEC DECIMAL_P DECLARE DEFAULT DEFAULTS

pgsql-hackers by date:

Previous
From: Tom Lane
Date:
Subject: Re: Do we really need a 7.4.22 release now?
Next
From: Magnus Hagander
Date:
Subject: Re: Do we really need a 7.4.22 release now?