Thread: Database file compatability
If a database is created with a 64 bit version of initdb, would a 32bit backend be able to talk to it? Likewise, would a backend compiled by a different compiler be able to? If there was some kind of incompatability, would the backend just refuse to start, or would it start and start silently trashing data? -- Jim C. Nasby, Sr. Engineering Consultant jnasby@pervasive.com Pervasive Software http://pervasive.com work: 512-231-6117 vcard: http://jim.nasby.net/pervasive.vcf cell: 512-569-9461
""Jim C. Nasby"" <jnasby@pervasive.com> wrote > If a database is created with a 64 bit version of initdb, would a 32bit > backend be able to talk to it? Likewise, would a backend compiled by a > different compiler be able to? > The key problem I believe is the serials of ALIGNOF macros. Especially for MAX_ALIGNOF. Different Hardware/OS/compiler will have different understanding of it. Compare your two versions PG, if they match, then with big chance, you can exchange their data. > If there was some kind of incompatability, would the backend just refuse > to start, or would it start and start silently trashing data? > -- Undefined. Mostly core dump. Regards, Qingqing
"Qingqing Zhou" <zhouqq@cs.toronto.edu> writes:
> ""Jim C. Nasby"" <jnasby@pervasive.com> wrote
>> If a database is created with a 64 bit version of initdb, would a 32bit
>> backend be able to talk to it? Likewise, would a backend compiled by a
>> different compiler be able to?
> The key problem I believe is the serials of ALIGNOF macros. Especially for
> MAX_ALIGNOF. Different Hardware/OS/compiler will have different
> understanding of it.
Yeah. It might be worth adding MAX_ALIGNOF to the set of configuration
data stored in pg_control, just to be sure you couldn't shoot yourself
in the foot that way.
regards, tom lane
Jim C. Nasby wrote: >If a database is created with a 64 bit version of initdb, would a 32bit >backend be able to talk to it? Likewise, would a backend compiled by a >different compiler be able to? > > Not in my experience at least from going 32 bit intel to 64bit opteron. >If there was some kind of incompatability, would the backend just refuse >to start, or would it start and start silently trashing data? > > -- Your PostgreSQL solutions company - Command Prompt, Inc. 1.800.492.2240 PostgreSQL Replication, Consulting, Custom Programming, 24x7 support Managed Services, Shared and Dedicated Hosting Co-Authors: plPHP, plPerlNG - http://www.commandprompt.com/
On Mon, 2005-09-26 at 17:27 -0500, Jim C. Nasby wrote: > If a database is created with a 64 bit version of initdb, would a 32bit > backend be able to talk to it? Likewise, would a backend compiled by a > different compiler be able to? > > If there was some kind of incompatability, would the backend just refuse > to start, or would it start and start silently trashing data? I plugged a storage array that was initialized with ia32 and attached it to an amd64 machine. The postmaster complained at startup that such-and-such magic value was incorrect and refused to start. However it was implied on the mailing list that for certain unlucky magic values the postmaster may have started anyway and eaten the database. -jwb
On Mon, Sep 26, 2005 at 07:05:28PM -0400, Tom Lane wrote: > "Qingqing Zhou" <zhouqq@cs.toronto.edu> writes: > > ""Jim C. Nasby"" <jnasby@pervasive.com> wrote > >> If a database is created with a 64 bit version of initdb, would a 32bit > >> backend be able to talk to it? Likewise, would a backend compiled by a > >> different compiler be able to? > > > The key problem I believe is the serials of ALIGNOF macros. Especially for > > MAX_ALIGNOF. Different Hardware/OS/compiler will have different > > understanding of it. > > Yeah. It might be worth adding MAX_ALIGNOF to the set of configuration > data stored in pg_control, just to be sure you couldn't shoot yourself > in the foot that way. PLEASE. :) ISTM that 64 bit is becomming much more common, so I think the odds of someone going from a 32 to 64 bit (or vice-versa) version of PostgreSQL on the same machine is much larger now than it has been in the past. I think we really need to protect this as much as possible. This isn't so much a foot-gun as a foot-nuclear-weapon. Would I be correct in assuming that doing this for 8.1 would require another initdb? :/ For something as minor as this, would it be reasonable to ship a utility to avoid the initdb? I'd very much like to see this in 8.1... -- Jim C. Nasby, Sr. Engineering Consultant jnasby@pervasive.com Pervasive Software http://pervasive.com work: 512-231-6117 vcard: http://jim.nasby.net/pervasive.vcf cell: 512-569-9461
""Jim C. Nasby"" <jnasby@pervasive.com> wrote >> Yeah. It might be worth adding MAX_ALIGNOF to the set of configuration >> data stored in pg_control, just to be sure you couldn't shoot yourself >> in the foot that way. > > PLEASE. :) > I am coming up with a patch of it. I think all ALIGNOF macros should be checked. Regards, Qingqing
"Qingqing Zhou" <zhouqq@cs.toronto.edu> writes:
> I think all ALIGNOF macros should be checked.
There are no platforms for which ALIGNOF_SHORT is different from 2.
I don't think there are any platforms we care about where ALIGNOF_INT
is different from 4. The cases of interest are ALIGNOF_DOUBLE,
ALIGNOF_LONG, ALIGNOF_LONG_LONG_INT (note that MAXIMUM_ALIGNOF is
just the largest of these). In practice "long int" is the same type
as either "int" or "long long int", so ALIGNOF_LONG isn't a distinct
case either. What it comes down to is that MAXIMUM_ALIGNOF is
sufficient to tell the difference between the platforms we need to
deal with. If you have a counterexample, tell us about it.
regards, tom lane
"Tom Lane" <tgl@sss.pgh.pa.us> wrote in message news:1223.1127878072@sss.pgh.pa.us... > > There are no platforms for which ALIGNOF_SHORT is different from 2. > I don't think there are any platforms we care about where ALIGNOF_INT > is different from 4. The cases of interest are ALIGNOF_DOUBLE, > ALIGNOF_LONG, ALIGNOF_LONG_LONG_INT (note that MAXIMUM_ALIGNOF is > just the largest of these). In practice "long int" is the same type > as either "int" or "long long int", so ALIGNOF_LONG isn't a distinct > case either. What it comes down to is that MAXIMUM_ALIGNOF is > sufficient to tell the difference between the platforms we need to > deal with. If you have a counterexample, tell us about it. > (1) Yes, ALIGNOF_SHORT is always 2. (2) There is a possible sequence like this: ALIGNOF_LONG 4 ALIGNOF_DOUBLE 8 MAXIMUM_ALIGNOF 8 vs. ALIGNOF_LONG 8 ALIGNOF_DOUBLE 8 MAXIMUM_ALIGNOF 8 Eg. http://developers.sun.com/prodtech/cc/articles/about_amd64_abi.html http://devrsrc1.external.hp.com/STK/wellbehavedrestrict.html So we should at least check ALIGNOF_LONG as well. (3) There are some machines with sizeof(int) equals to 64, if my memory saves, which might imply that ALIGNOF_INT equals to 8. So conservatively, we'd better check ALIGNOF_INT, ALIGNOF_LONG and MAXIMUM_ALIGNOF. Regards, Qingqing
"Qingqing Zhou" <zhouqq@cs.toronto.edu> writes:
> "Tom Lane" <tgl@sss.pgh.pa.us> wrote in message
> There is a possible sequence like this:
> ALIGNOF_LONG 4
> ALIGNOF_DOUBLE 8
> MAXIMUM_ALIGNOF 8
> vs.
> ALIGNOF_LONG 8
> ALIGNOF_DOUBLE 8
> MAXIMUM_ALIGNOF 8
> So we should at least check ALIGNOF_LONG as well.
No, we don't need to, because we do not really care about ALIGNOF_LONG
per se. We don't use "long" as an on-disk datatype, precisely because
we don't know what size it is. We use int32 and int64. The former has
align 4 on all machines AFAIK, and the latter has MAXIMUM_ALIGNOF.
> There are some machines with sizeof(int) equals to 64, if my memory saves,
> which might imply that ALIGNOF_INT equals to 8.
If there were such a machine, Postgres wouldn't run on it anyway, and
a lot of other software too. There'd be no way to have both int16 and
int32 types ("short" could cover only one of them).
regards, tom lane
"Qingqing Zhou" <zhouqq@cs.toronto.edu> wrote > > "Tom Lane" <tgl@sss.pgh.pa.us> wrote in message > news:1223.1127878072@sss.pgh.pa.us... > > > > There are no platforms for which ALIGNOF_SHORT is different from 2. > > I don't think there are any platforms we care about where ALIGNOF_INT > > is different from 4. The cases of interest are ALIGNOF_DOUBLE, > > ALIGNOF_LONG, ALIGNOF_LONG_LONG_INT (note that MAXIMUM_ALIGNOF is > > just the largest of these). In practice "long int" is the same type > > as either "int" or "long long int", so ALIGNOF_LONG isn't a distinct > > case either. What it comes down to is that MAXIMUM_ALIGNOF is > > sufficient to tell the difference between the platforms we need to > > deal with. If you have a counterexample, tell us about it. > > > (1) > Yes, ALIGNOF_SHORT is always 2. > > (2) > There is a possible sequence like this: > > ALIGNOF_LONG 4 > ALIGNOF_DOUBLE 8 > MAXIMUM_ALIGNOF 8 > > vs. > > ALIGNOF_LONG 8 > ALIGNOF_DOUBLE 8 > MAXIMUM_ALIGNOF 8 > > Eg. > http://developers.sun.com/prodtech/cc/articles/about_amd64_abi.html > http://devrsrc1.external.hp.com/STK/wellbehavedrestrict.html > > So we should at least check ALIGNOF_LONG as well. > > (3) > There are some machines with sizeof(int) equals to 64, if my memory saves, > which might imply that ALIGNOF_INT equals to 8. sizeof(int) maybe 8, but not 64. And the configure option `--enable-integer-datetimes' may affect the data layout. > > So conservatively, we'd better check ALIGNOF_INT, ALIGNOF_LONG and > MAXIMUM_ALIGNOF. > > Regards, > Qingqing > >
"William ZHANG" <uniware@zedware.org> wrote > > sizeof(int) maybe 8, but not 64. > And the configure option `--enable-integer-datetimes' may affect the data > layout. > Yes, typo. This has been checked by ControlFileData.enableIntTimes. Regards, Qingqing
On Wed, Sep 28, 2005 at 10:22:51AM -0400, Tom Lane wrote: > "Qingqing Zhou" <zhouqq@cs.toronto.edu> writes: > > "Tom Lane" <tgl@sss.pgh.pa.us> wrote in message > > There is a possible sequence like this: > > > ALIGNOF_LONG 4 > > ALIGNOF_DOUBLE 8 > > MAXIMUM_ALIGNOF 8 > > > vs. > > > ALIGNOF_LONG 8 > > ALIGNOF_DOUBLE 8 > > MAXIMUM_ALIGNOF 8 > > > So we should at least check ALIGNOF_LONG as well. > > No, we don't need to, because we do not really care about ALIGNOF_LONG > per se. We don't use "long" as an on-disk datatype, precisely because > we don't know what size it is. We use int32 and int64. The former has > align 4 on all machines AFAIK, and the latter has MAXIMUM_ALIGNOF. Is there a serious penalty associated with just checking them all? Seems like better safe than sorry... On a related note, are checks for endianness made as well? -- Jim C. Nasby, Sr. Engineering Consultant jnasby@pervasive.com Pervasive Software http://pervasive.com work: 512-231-6117 vcard: http://jim.nasby.net/pervasive.vcf cell: 512-569-9461