Thread: [HACKERS] DSA on 32 bit systems
Hi hackers, Commit e8fdbd58fe564a29977f4331cd26f9697d76fc40 introduced fallback atomic u64 support. PG_HAVE_ATOMIC_U64_SUPPORT now always finishes up defined so it is pointless to test for it in dsa.h. Here's a patch to remove the obsolete test and comment. That raises the question of whether we should go further and use a 64 bit dsa_pointer even on 32 bit systems. In a 32 bit dsa_pointer build, a single dsa_area is limited to 32 segments of up to 128MB in size (actually the largest power of 2 size it can allocate at once is 64MB due to free page management overhead). The 64 bit version can handle 1024 segments of up to 1TB, which ought to be enough for anybody. Having just one build variant without any limits you're likely to hit is appealing, but using oversized pointers will presumably slow 32 bit systems down, perhaps especially for client code that uses dsa_pointer_atomic (though I haven't tested that). Thought experiment: To allow allocations of 256MB without reducing the total size available with a pattern of small allocations (~1GB), we'd have to start larger (4MB) and grow faster (double every time) due to lack of bits to represent the segment number. That doesn't seem very friendly to small systems, so maybe it's not a great idea to change it. Thoughts? If we keep 32 bit dsa_pointer, I now see that parallel hash should probably know about this limit so it can abandon its load factor goal rather than making a doomed allocation request. -- Thomas Munro http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Attachment
On Wed, Jul 12, 2017 at 7:34 AM, Thomas Munro <thomas.munro@enterprisedb.com> wrote: > That raises the question of whether we should go further and use a 64 > bit dsa_pointer even on 32 bit systems. In a 32 bit dsa_pointer > build, a single dsa_area is limited to 32 segments of up to 128MB in > size (actually the largest power of 2 size it can allocate at once is > 64MB due to free page management overhead). The 64 bit version can > handle 1024 segments of up to 1TB, which ought to be enough for > anybody. Having just one build variant without any limits you're > likely to hit is appealing, but using oversized pointers will > presumably slow 32 bit systems down, perhaps especially for client > code that uses dsa_pointer_atomic (though I haven't tested that). Hmm. What we do we think the largest single allocation that has a chance of working on a 32-bit system is in general? My guess is that 1GB is not likely to work unless things break very well for you, but 256MB and 512MB might, absent implementation limitations. > If we keep 32 bit dsa_pointer, I now see that parallel hash should > probably know about this limit so it can abandon its load factor goal > rather than making a doomed allocation request. My overall feeling here is that this *probably* doesn't matter much one way or the other, and we don't really have enough information right now to know whether wider dsa_pointers are a bigger problem than not being able to make large allocations, or visca versa. But count me as +0 for widening them to 64-bits. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company