Thread: Linux: PAE or x64

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hello,

I'll use PG on a dedicated machine with more than 4GB of memory.

The problem is: what would be better to use: PAE ("bigmem" kernels) or
64-bit kernel?

PAE pro: half the memory per pointer, int, etc.

PAE risk: is PG able to take advantage of all the memory on PAE, esp. re
things like shared buffers?

64-bit pro: it's more mainstream now, no worries about shared bufs etc.


PAE overhead according to wikipedia should be small. (?)

So, which one should be better?


- --

Regards,
mk

- --
Premature optimization is the root of all fun.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQEcBAEBAgAGBQJNCJi2AAoJEFMgHzhQQ7hO05AH/jEr5hHlbqOh5MMdRR2GI3fM
EyJhA1dCH9hMgOEn0k07NzFLCZcwZRCVTTIPvgZPz/4pLjSG/37N/oacy8KX6tDo
8i9r8YM41wwlX2xeaowHqXybTFkRloRLN1Lb5cOzm5AadJn+F9+QjVf2r9MSrpGy
F/0pfWzR0Ams8HTbeW3GeB+vezEduRH8XUmuHs8DYx0q7KoeTAqyzFJwelU5R4vo
PCz8s/4A2t0A1wkRy4J+nVteORYjjdY3Yz44Df/3Tt1oVCbn9+fV979XjWwWiXXk
g9wq42/RI++gH8DBAx9iokBW8EKFb86GR4jlP4BxxO/glPhQv+bI1To9aPJwRm8=
=1gzy
-----END PGP SIGNATURE-----

On Wed, 15 Dec 2010 11:30:14 +0100
Marcin Krol <mrkafk@gmail.com> wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Hello,
>
> I'll use PG on a dedicated machine with more than 4GB of memory.
>
> The problem is: what would be better to use: PAE ("bigmem" kernels) or
> 64-bit kernel?
>
> PAE pro: half the memory per pointer, int, etc.
>
> PAE risk: is PG able to take advantage of all the memory on PAE, esp. re
> things like shared buffers?
>
> 64-bit pro: it's more mainstream now, no worries about shared bufs etc.
>
>
> PAE overhead according to wikipedia should be small. (?)
>
> So, which one should be better?
Don't even bother with PAE. Let it rot.
memory consumption due to 64 bits pointers is negligible.
Laurent

On 15 December 2010 10:37, Laurent Wandrebeck <l.wandrebeck@gmail.com> wrote:
> Don't even bother with PAE. Let it rot.
> memory consumption due to 64 bits pointers is negligible.

And yet, that has been used by authoritative people as a partial
justification for pg lacking a 64-bit version on Windows in the past
on more than one occasion. Perhaps it wasn't stressed too much, but
certainly it was treated as a greater than negligible issue:

http://blog.hagander.net/archives/73-PostgreSQL-vs-64-bit-windows.html


--
Regards,
Peter Geoghegan

* Peter Geoghegan (peter.geoghegan86@gmail.com) wrote:
> And yet, that has been used by authoritative people as a partial
> justification for pg lacking a 64-bit version on Windows in the past
> on more than one occasion.

You're misreading poor Magnus.  He didn't offer any 'justification'
regarding why there isn't a Win64 port.  He simply was pointing out, for
those who assume every 'real' tool must be 64bit, that a 32bit PG is
still a very viable and useful tool.  The reason there isn't a Win64
port has everything to do with no one being interested enough to work on
it, *because* PG runs decently as a 32bit app.  If you'd like to work on
it, or pay someone to, I'm sure you'd find many in the community who
would love to see it happen and might even be able to help.

> Perhaps it wasn't stressed too much, but
> certainly it was treated as a greater than negligible issue:

Compared to the costs of PAE?  The memory overhead is *well* worth it.
Let's try to keep this in context here.

    Thanks,

        Stephen

On 15 December 2010 14:13, Stephen Frost <sfrost@snowman.net> wrote:
> You're misreading poor Magnus.  He didn't offer any 'justification'
> regarding why there isn't a Win64 port.  He simply was pointing out, for
> those who assume every 'real' tool must be 64bit, that a 32bit PG is
> still a very viable and useful tool.  The reason there isn't a Win64
> port has everything to do with no one being interested enough to work on
> it, *because* PG runs decently as a 32bit app.  If you'd like to work on
> it, or pay someone to, I'm sure you'd find many in the community who
> would love to see it happen and might even be able to help.

Actually, there is a 64-bit port for windows now. I don't think I
misrepresented Magnus - the post suggested that the then-lack of a
64-bit windows port wasn't a pressing issue, and that various
technical considerations *partially* justified there not being one at
the time (the word size of binaries, and more importantly PG's
architecture). It's an assessment that I agreed with.

>> Perhaps it wasn't stressed too much, but
>> certainly it was treated as a greater than negligible issue:
>
> Compared to the costs of PAE?  The memory overhead is *well* worth it.
> Let's try to keep this in context here.

I wasn't making any comparison. Magnus's post didn't mention PAE, and
had nothing to do with it, but I just thought I'd draw attention to
where the idea probably originally came from. However, it's worth
noting that PAE still limits each individual process to a 32-bit
address space, so PG disproportionately benefits from it (at least for
high-concurrency use cases, which are the use cases where PG excels
generally).

What Magnus said is supported (albeit sort of weakly) by this:
http://msdn.microsoft.com/en-us/library/ee418798(v=vs.85).aspx

"When you compile applications as 64-bit, the calculations get more
complicated. A 64-bit program uses 64-bit pointers, and its
instructions are slightly larger, so the memory requirement is
slightly increased. This can cause a slight drop in performance. On
the other hand, having twice as many registers and having the ability
to do 64-bit integer calculations in a single instruction will often
more than compensate. The net result is that a 64-bit application
might run slightly slower than the same application compiled as
32-bit, but it will often run slightly faster."


--
Regards,
Peter Geoghegan

On Wed, Dec 15, 2010 at 8:20 AM, Peter Geoghegan
<peter.geoghegan86@gmail.com> wrote:
> Actually, there is a 64-bit port for windows now. I don't think I
> misrepresented Magnus - the post suggested that the then-lack of a
> 64-bit windows port wasn't a pressing issue, and that various
> technical considerations *partially* justified there not being one at
> the time (the word size of binaries, and more importantly PG's
> architecture). It's an assessment that I agreed with.

Also there was (is?) the issue that the pg / shared memory system used
on windows is apparently quite inefficient at using large amounts of
memory, so there was no pressing need there for 64 bitness either.

On Wed, Dec 15, 2010 at 17:34, Scott Marlowe <scott.marlowe@gmail.com> wrote:
> On Wed, Dec 15, 2010 at 8:20 AM, Peter Geoghegan
> <peter.geoghegan86@gmail.com> wrote:
>> Actually, there is a 64-bit port for windows now. I don't think I
>> misrepresented Magnus - the post suggested that the then-lack of a
>> 64-bit windows port wasn't a pressing issue, and that various
>> technical considerations *partially* justified there not being one at
>> the time (the word size of binaries, and more importantly PG's
>> architecture). It's an assessment that I agreed with.
>
> Also there was (is?) the issue that the pg / shared memory system used
> on windows is apparently quite inefficient at using large amounts of
> memory, so there was no pressing need there for 64 bitness either.

I think there still is, but I don't think anybody has run any proper
benchmarks on different sizes of shared buffers on Win64. That would
be some useful information to have, if anybody's listening.. (all my
64-bit windows boxes are in VMs on "cloud infrastructure" and those
not very suitable for benchmarking..)

Another case was DW-style loads where you actually want to be able to
use gigabytes of memory in a single backend for sort/hash operations
and such. For that, there was no workarond...

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/