Thread: database file compatibility patch

This patches checks MAXIMUM_ALIGNOF and endian to make sure that the
database file used is compatible with the server version. We use
SHORT_ALIGNOF, INT_ALIGNOF, DOUBLE_ALIGNOF and MAXIMUM_ALIGNOF (which is
just the largest of these) to align columns within a row (see att_align())
or rows within a page. On all platforms we know, SHORT_ALIGNOF is 2, and
we only support platforms with INT_ALIGOF equals to 4. What it comes down
to is that MAXIMUM_ALIGNOF is sufficient to tell the difference between
the platforms we need to deal with. Also, check the endian to make sure
that the multibytes numbers storage is compatible.

Regards, Qingqing


---

Index: backend/access/transam/xlog.c
===================================================================
RCS file: /projects/cvsroot/pgsql/src/backend/access/transam/xlog.c,v
retrieving revision 1.218
diff -c -r1.218 xlog.c
*** backend/access/transam/xlog.c    22 Aug 2005 23:59:04 -0000    1.218
--- backend/access/transam/xlog.c    2 Oct 2005 18:08:25 -0000
***************
*** 3420,3425 ****
--- 3420,3427 ----
      ControlFile->blcksz = BLCKSZ;
      ControlFile->relseg_size = RELSEG_SIZE;
      ControlFile->xlog_seg_size = XLOG_SEG_SIZE;
+     ControlFile->maximum_alignof = MAXIMUM_ALIGNOF;
+     ControlFile->endian = ENDIAN_CODE;

      ControlFile->nameDataLen = NAMEDATALEN;
      ControlFile->indexMaxKeys = INDEX_MAX_KEYS;
***************
*** 3583,3588 ****
--- 3585,3604 ----
                     " but the server was compiled with XLOG_SEG_SIZE %d.",
                             ControlFile->xlog_seg_size, XLOG_SEG_SIZE),
               errhint("It looks like you need to recompile or initdb.")));
+     if (ControlFile->maximum_alignof != MAXIMUM_ALIGNOF)
+         ereport(FATAL,
+                 (errmsg("database files are incompatible with server"),
+                  errdetail("The database cluster was initialized with MAXIMUM_ALIGNOF
+ %d,"
+                    " but the server was compiled with MAXIMUM_ALIGNOF %d.",
+                           ControlFile->maximum_alignof, MAXIMUM_ALIGNOF),
+              errhint("It looks like you need to recompile or initdb.")));
+     if (ControlFile->endian != ENDIAN_CODE)
+         ereport(FATAL,
+                 (errmsg("database files are incompatible with server"),
+                  errdetail("The database cluster was initialized with different"
+                      " endian with the server"),
+              errhint("It looks like you need to recompile or initdb.")));
      if (ControlFile->nameDataLen != NAMEDATALEN)
          ereport(FATAL,
                  (errmsg("database files are incompatible with server"),
Index: backend/storage/freespace/freespace.c
===================================================================
RCS file: /projects/cvsroot/pgsql/src/backend/storage/freespace/freespace.c,v
retrieving revision 1.48
diff -c -r1.48 freespace.c
*** backend/storage/freespace/freespace.c    20 Aug 2005 23:26:20 -0000    1.48
--- backend/storage/freespace/freespace.c    2 Oct 2005 18:08:26 -0000
***************
*** 116,122 ****
   *
   * The file format is:
   *        label            "FSM\0"
-  *        endian            constant 0x01020304 for detecting endianness problems
   *        version#
   *        numRels
   *    -- for each rel, in *reverse* usage order:
--- 116,121 ----
***************
*** 134,147 ****

  /* Fixed values in header */
  #define FSM_CACHE_LABEL        "FSM"
- #define FSM_CACHE_ENDIAN    0x01020304
  #define FSM_CACHE_VERSION    20030305

  /* File header layout */
  typedef struct FsmCacheFileHeader
  {
      char        label[4];
-     uint32        endian;
      uint32        version;
      int32        numRels;
  } FsmCacheFileHeader;
--- 133,144 ----
***************
*** 767,773 ****
      /* Write file header */
      MemSet(&header, 0, sizeof(header));
      strcpy(header.label, FSM_CACHE_LABEL);
-     header.endian = FSM_CACHE_ENDIAN;
      header.version = FSM_CACHE_VERSION;
      header.numRels = FreeSpaceMap->numRels;
      if (fwrite(&header, 1, sizeof(header), fp) != sizeof(header))
--- 764,769 ----
***************
*** 868,874 ****
      /* Read and verify file header */
      if (fread(&header, 1, sizeof(header), fp) != sizeof(header) ||
          strcmp(header.label, FSM_CACHE_LABEL) != 0 ||
-         header.endian != FSM_CACHE_ENDIAN ||
          header.version != FSM_CACHE_VERSION ||
          header.numRels < 0)
      {
--- 864,869 ----
Index: bin/pg_controldata/pg_controldata.c
===================================================================
RCS file: /projects/cvsroot/pgsql/src/bin/pg_controldata/pg_controldata.c,v
retrieving revision 1.25
diff -c -r1.25 pg_controldata.c
*** bin/pg_controldata/pg_controldata.c    8 Jun 2005 15:50:27 -0000    1.25
--- bin/pg_controldata/pg_controldata.c    2 Oct 2005 18:08:26 -0000
***************
*** 171,176 ****
--- 171,178 ----
      printf(_("Database block size:                  %u\n"), ControlFile.blcksz);
      printf(_("Blocks per segment of large relation: %u\n"), ControlFile.relseg_size);
      printf(_("Bytes per WAL segment:                %u\n"), ControlFile.xlog_seg_size);
+     printf(_("Maximum alignment:                    %u\n"), ControlFile.maximum_alignof);
+     printf(_("Endian code:                          %u\n"), ControlFile.endian);
      printf(_("Maximum length of identifiers:        %u\n"), ControlFile.nameDataLen);
      printf(_("Maximum columns in an index:          %u\n"), ControlFile.indexMaxKeys);
      printf(_("Date/time type storage:               %s\n"),
Index: bin/pg_resetxlog/pg_resetxlog.c
===================================================================
RCS file: /projects/cvsroot/pgsql/src/bin/pg_resetxlog/pg_resetxlog.c,v
retrieving revision 1.36
diff -c -r1.36 pg_resetxlog.c
*** bin/pg_resetxlog/pg_resetxlog.c    29 Sep 2005 08:34:50 -0000    1.36
--- bin/pg_resetxlog/pg_resetxlog.c    2 Oct 2005 18:08:26 -0000
***************
*** 459,464 ****
--- 459,466 ----
      ControlFile.blcksz = BLCKSZ;
      ControlFile.relseg_size = RELSEG_SIZE;
      ControlFile.xlog_seg_size = XLOG_SEG_SIZE;
+     ControlFile.maximum_alignof = MAXIMUM_ALIGNOF;
+     ControlFile.endian = ENDIAN_CODE;
      ControlFile.nameDataLen = NAMEDATALEN;
      ControlFile.indexMaxKeys = INDEX_MAX_KEYS;
  #ifdef HAVE_INT64_TIMESTAMP
***************
*** 525,530 ****
--- 527,535 ----
      printf(_("Latest checkpoint's NextMultiOffset:  %u\n"), ControlFile.checkPointCopy.nextMultiOffset);
      printf(_("Database block size:                  %u\n"), ControlFile.blcksz);
      printf(_("Blocks per segment of large relation: %u\n"), ControlFile.relseg_size);
+     printf(_("Bytes per WAL segment:                %u\n"), ControlFile.xlog_seg_size);
+     printf(_("Maximum alignment:                    %u\n"), ControlFile.maximum_alignof);
+     printf(_("Endian code:                          %u\n"), ControlFile.endian);
      printf(_("Maximum length of identifiers:        %u\n"), ControlFile.nameDataLen);
      printf(_("Maximum columns in an index:          %u\n"), ControlFile.indexMaxKeys);
      printf(_("Date/time type storage:               %s\n"),
Index: include/catalog/pg_control.h
===================================================================
RCS file: /projects/cvsroot/pgsql/src/include/catalog/pg_control.h,v
retrieving revision 1.23
diff -c -r1.23 pg_control.h
*** include/catalog/pg_control.h    8 Jun 2005 15:50:28 -0000    1.23
--- include/catalog/pg_control.h    2 Oct 2005 18:08:26 -0000
***************
*** 60,65 ****
--- 60,66 ----
      DB_IN_PRODUCTION
  } DBState;

+ #define ENDIAN_CODE        0x01020304
  #define LOCALE_NAME_BUFLEN    128

  /*
***************
*** 116,121 ****
--- 117,125 ----

      uint32        xlog_seg_size;    /* size of each WAL segment */

+     uint32        maximum_alignof; /* alignment requirement */
+     uint32        endian;         /* endian check code */
+
      uint32        nameDataLen;    /* catalog name field width */
      uint32        indexMaxKeys;    /* max number of columns in an index */

Qingqing Zhou <zhouqq@cs.toronto.edu> writes:
> This patches checks MAXIMUM_ALIGNOF and endian to make sure that the
> database file used is compatible with the server version.

I missed seeing this patch in my inbox, so wrote and applied my own
version earlier today.  Sorry for the missed communication :-(.  It's
about the same result though.

> What it comes down
> to is that MAXIMUM_ALIGNOF is sufficient to tell the difference between
> the platforms we need to deal with. Also, check the endian to make sure
> that the multibytes numbers storage is compatible.

There's not much need to check endianness explicitly, since the
pg_control_version check will surely fail if there's an endianness
discrepancy (not to mention the other checks on pg_control fields).
[ Actually, I had originally thought that alignment would be reflected
implicitly in pg_control too, but given that it's mostly int32 fields
I don't think that can be relied on. ]

What I did add, after reviewing the past discussion of these issues,
is a simple-minded test to see if the floating-point storage format
is the same.  That and endianness/alignment are the only points that
have been mentioned as likely causes of cross-platform incompatibility.

            regards, tom lane

"Tom Lane" <tgl@sss.pgh.pa.us> wrote
>
> There's not much need to check endianness explicitly, since the
> pg_control_version check will surely fail if there's an endianness
> discrepancy (not to mention the other checks on pg_control fields).
>

Oh, right. So for the same reason, is it safe to remove the endian check
in FsmCacheFileHeader?

Regards, Qingqing

Qingqing Zhou <zhouqq@cs.toronto.edu> writes:
> "Tom Lane" <tgl@sss.pgh.pa.us> wrote
>> There's not much need to check endianness explicitly, since the
>> pg_control_version check will surely fail if there's an endianness
>> discrepancy (not to mention the other checks on pg_control fields).

> Oh, right. So for the same reason, is it safe to remove the endian check
> in FsmCacheFileHeader?

[ Scratches head... ]  Yeah, I would think so.  I'm trying to remember
why I put that code in there in the first place ...

            regards, tom lane