Thread: Replication between 64/32bit systems?
Hi folks, I'm planning to set up streaming replication from one master to one slave. I've read at http://www.postgresql.org/docs/9.1/static/warm-standby.html that the "hardware architecture" of both systems must be the same. Sure enough, what I'd really like to do is replicate from a Windows (or Linux) 64bit master to a Linux 32bit slave -- that's what I currently have easily available. I tried just following the instructions and copied the data directory to the slaves, but postgres does not start: FATAL: falsche Prüfsumme in Kontrolldatei (FATAL: wrong checksum in control file) So I'd like to ask if there is anything I could do by e.g. changing compile-time options at the slave to get things going? Thank you for any advice!, best regards -hannes
On September 20, 2011 01:26:06 PM Hannes Erven wrote: > So I'd like to ask if there is anything I could do by e.g. changing > compile-time options at the slave to get things going? > No.
On Tue, 2011-09-20 at 22:26 +0200, Hannes Erven wrote: > [...] > I'm planning to set up streaming replication from one master to one > slave. I've read at > http://www.postgresql.org/docs/9.1/static/warm-standby.html that the > "hardware architecture" of both systems must be the same. > > Sure enough, what I'd really like to do is replicate from a Windows (or > Linux) 64bit master to a Linux 32bit slave -- that's what I currently > have easily available. > > I tried just following the instructions and copied the data directory to > the slaves, but postgres does not start: > FATAL: falsche Prüfsumme in Kontrolldatei > (FATAL: wrong checksum in control file) > > > So I'd like to ask if there is anything I could do by e.g. changing > compile-time options at the slave to get things going? > There's nothing you can do. It won't work between a 32bit and a 64bit machines. -- Guillaume http://blog.guillaume.lelarge.info http://www.dalibo.com
From the manual, "In any case the hardware architecture must be the same shipping from, say, a 32-bit to a 64-bit system will not work."... I don't even believe you can copy a 64bit database into 32bit machine... what makes warm standby impossible... I've tried this in the past, and give up - without even contacting the list. I was sure I was doing wrong. What I did was moving all my database servers (I've 7: 2 production, 2 slaves, 2 development, 1 test release) into 64 bit platform. Other well known and equally respected databases poses same limitation (not able to replicate from 32 to 64 or vice versa, or to other operating systems due the "big endian"-"little endian" issue). My2c, Edson. -----Mensagem original----- De: pgsql-general-owner@postgresql.org [mailto:pgsql-general-owner@postgresql.org] Em nome de Hannes Erven Enviada em: terça-feira, 20 de setembro de 2011 17:26 Para: pgsql Assunto: [GENERAL] Replication between 64/32bit systems? Hi folks, I'm planning to set up streaming replication from one master to one slave. I've read at http://www.postgresql.org/docs/9.1/static/warm-standby.html that the "hardware architecture" of both systems must be the same. Sure enough, what I'd really like to do is replicate from a Windows (or Linux) 64bit master to a Linux 32bit slave -- that's what I currently have easily available. I tried just following the instructions and copied the data directory to the slaves, but postgres does not start: FATAL: falsche Prüfsumme in Kontrolldatei (FATAL: wrong checksum in control file) So I'd like to ask if there is anything I could do by e.g. changing compile-time options at the slave to get things going? Thank you for any advice!, best regards -hannes -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general
On Tue, 2011-09-20 at 22:37 +0200, Guillaume Lelarge wrote: > On Tue, 2011-09-20 at 22:26 +0200, Hannes Erven wrote: > > [...] > > I'm planning to set up streaming replication from one master to one > > slave. I've read at > > http://www.postgresql.org/docs/9.1/static/warm-standby.html that the > > "hardware architecture" of both systems must be the same. > > > > Sure enough, what I'd really like to do is replicate from a Windows (or > > Linux) 64bit master to a Linux 32bit slave -- that's what I currently > > have easily available. > > > > I tried just following the instructions and copied the data directory to > > the slaves, but postgres does not start: > > FATAL: falsche Prüfsumme in Kontrolldatei > > (FATAL: wrong checksum in control file) > > > > > > So I'd like to ask if there is anything I could do by e.g. changing > > compile-time options at the slave to get things going? > > > > There's nothing you can do. It won't work between a 32bit and a 64bit > machines. > Just when I hit the Send button, I wondered if that would be possible if he installed a 32bit PostgreSQL on his 64bit Windows. It could be OK with the same OS, but with different OS, there is less chance, I guess. -- Guillaume http://blog.guillaume.lelarge.info http://www.dalibo.com
Hi Hannes,
You can't use streaming replication, but you can use slony to replicate
between different architectures.
- Chris
On 09/20/2011 02:26 PM, Hannes Erven wrote:
> Hi folks,
>
>
> I'm planning to set up streaming replication from one master to one
> slave. I've read at
> http://www.postgresql.org/docs/9.1/static/warm-standby.html that the
> "hardware architecture" of both systems must be the same.
>
> Sure enough, what I'd really like to do is replicate from a Windows (or
> Linux) 64bit master to a Linux 32bit slave -- that's what I currently
> have easily available.
>
> I tried just following the instructions and copied the data directory to
> the slaves, but postgres does not start:
> FATAL: falsche Prüfsumme in Kontrolldatei
> (FATAL: wrong checksum in control file)
>
>
> So I'd like to ask if there is anything I could do by e.g. changing
> compile-time options at the slave to get things going?
>
>
> Thank you for any advice!,
> best regards
>
> -hannes
>
On Tue, Sep 20, 2011 at 2:49 PM, Chris Ernst <cernst@zvelo.com> wrote: > Hi Hannes, > > You can't use streaming replication, but you can use slony to replicate > between different architectures. But that's not all! You can also use slony to migrate from one major version to another! This is one of the things it was built for.
Folks,
>> Sure enough, what I'd really like to do is replicate from a Windows
>> (or Linux) 64bit master to a Linux 32bit slave -- that's what I
>> currently have easily available.
thank you for your replies -- unfortunately, I'm not very content with
"it simply won't work" without understanding what the reasons are. ;-)
Endianness, OK, but that's the same on i32/64 platforms. So is it just
the length of the int & friends data types? I don't (yet) believe
there's no way around that.
So I tried compiling the postgres sources (identical version as on
server) with the following gcc options in CFLAGS:
-m128bit-long-double
-malign-double
This leads me to a pg_controldata executable that does no more complain
about invalid CRCs and it also prints out valid-looking data.
But, the postmaster process does not start up -- no error, but it just
doesn't come up.
When starting with --single, I get the following output:
LOG: database system was shut down at 2011-09-22 17:29:34 CEST
DEBUG: primary_conninfo = 'host=192.168.xx.yy port=5432 user=yyy
password=zzz'
DEBUG: standby_mode = 'on'
LOG: entering standby mode
DEBUG: checkpoint record is at 3/EB000378
DEBUG: redo record is at 3/EB000378; shutdown TRUE
DEBUG: next transaction ID: 0/14549; next OID: 483488
DEBUG: next MultiXactId: 7; next MultiXactOffset: 13
DEBUG: oldest unfrozen transaction ID: 654, in database 1
DEBUG: transaction ID wrap limit is 2147484301, limited by database
with OID 1
LOG: consistent recovery state reached at 3/EB0003D0
LOG: record with zero length at 3/EB0003D0
DEBUG: could not open file "pg_xlog/0000000100000003000000EB" (log file
3, segment 235): File or directory not found
So... it doesn't look to be that bad, is it?
I'm still hoping someone would give me a clue why 32/64 bit platform
xlogs are (and always will?) be absolutely incompatible... ???
Thanks again, and sorry for my "insisting"...
-hannes
On 09/22/11 1:35 PM, Hannes Erven wrote: > I'm still hoping someone would give me a clue why 32/64 bit platform > xlogs are (and always will?) be absolutely incompatible... ??? because the tuple representations are different, plain and simple. you can run a 32bit Postgres on your 64bit platform. -- john r pierce N 37, W 122 santa cruz ca mid-left coast
On Thu, Sep 22, 2011 at 2:45 PM, John R Pierce <pierce@hogranch.com> wrote: > On 09/22/11 1:35 PM, Hannes Erven wrote: >> >> I'm still hoping someone would give me a clue why 32/64 bit platform >> xlogs are (and always will?) be absolutely incompatible... ??? > > because the tuple representations are different, plain and simple. > > you can run a 32bit Postgres on your 64bit platform. > Also even if you somehow manage to tweak it to the point where it works it is: 1) Untested/unsupported (you are on your own!) and 2) Not guaranteed to continue to work. Really, IMO if replication is important enough to do, it's important enough to stick with supported environments. Best Wishes, Chris Travers
John R Pierce <pierce@hogranch.com> writes:
> On 09/22/11 1:35 PM, Hannes Erven wrote:
>> I'm still hoping someone would give me a clue why 32/64 bit platform
>> xlogs are (and always will?) be absolutely incompatible... ???
> because the tuple representations are different, plain and simple.
Specifically, the data alignment rules are different, so a 64-bit tuple
can have padding bytes in it in places where a 32-bit tuple doesn't.
This spills over into different numbers of tuples fitting on a page,
etc etc.
> you can run a 32bit Postgres on your 64bit platform.
Yeah, that would be the thing to do if you're stuck with replicating to
nonidentical hardware.
regards, tom lane
On 09/23/2011 04:35 AM, Hannes Erven wrote: > thank you for your replies -- unfortunately, I'm not very content with > "it simply won't work" without understanding what the reasons are. ;-) > > Endianness, OK, but that's the same on i32/64 platforms. So is it just > the length of the int& friends data types? I don't (yet) believe > there's no way around that. Theoretically, code could be written that understood the 32-bit form and translated it to the 64-bit form or vice versa. AFAIK no such code has been written and I don't think anyone's particularly interested in doing it. Similarly, it'd be possible to come up with an arch-independent WAL and datadir format that fixed the sizes and endianness of all representations. There'd be a performance cost, but it probably wouldn't be a huge one, and it'd probably be possible to make a "portable format database" an option that could be chosen at initdb time. Again, though, I doubt anyone's particularly interested in doing the required work, and it's not clear a patch would be accepted because of the increased maintenance and testing burden it'd impose. -- Craig Ringer
On Thu, Sep 22, 2011 at 5:46 PM, Craig Ringer <ringerc@ringerc.id.au> wrote: > On 09/23/2011 04:35 AM, Hannes Erven wrote: >> >> thank you for your replies -- unfortunately, I'm not very content with >> "it simply won't work" without understanding what the reasons are. ;-) >> >> Endianness, OK, but that's the same on i32/64 platforms. So is it just >> the length of the int& friends data types? I don't (yet) believe >> there's no way around that. > > Theoretically, code could be written that understood the 32-bit form and > translated it to the 64-bit form or vice versa. AFAIK no such code has been > written and I don't think anyone's particularly interested in doing it. Not if the number of tuples per page is different, unless one wants to arbitrarily limit that, with attendant performance costs..... Best Wishes, Chris Travers
OK,
thank you for clarification, I'll "resign" ;-) and accept that mixing
32/64 bits is not possible.
I continued by experiments, and would like to share my results. First of
all, I'm an idiot because I had logging_collector set to on and wondered
why the terminals did not give any meaningful output when postgres was
started with -d4 .
Having fixed that, what actually seems to work with 9.1.0 is:
On the Windows host (64 bit):
- just use the 32bit binaries
- initdb --locale C
(or any other locale that is also available on the slave)
Linux replication slave (32 bit):
- use CFLAGS="m128bit-long-double -malign-double" ./configure
(otherwise you'll hit "checksum error in control file")
When you try to connect to a 64bit postgres host, the connection
succeeds and everything looks fine so far, until the first transaction
needs to be transferred: then, weird messages appear on the slave.
Obviously, this is the result of binary protocol incompatibility...
So far, I have only tried to replicate a small, newly created test
database. Next, I'll transfer a bigger database to a locale=C cluster
and see if replication works there as well.
Thanks again for your support,
-hannes