On Thu, 12 Dec 2024 at 22:28, Andreas 'ads' Scherbaum <ads@pgug.de> wrote: > > > Hello, > > I'm running a couple of large tests, and in this particular test I have > a few million tables more. > > At some point it fails, and I gathered the following trace: > > > 2024-12-12 22:22:55.307 CET [1496210] ERROR: invalid DSA memory alloc > request size 1073741824 > 2024-12-12 22:22:55.307 CET [1496210] BACKTRACE: > postgres: ads tabletest [local] CREATE TABLE(+0x15e570) > [0x6309c379c570] > postgres: ads tabletest [local] CREATE > TABLE(dshash_find_or_insert+0x1a4) [0x6309c39882d4] > postgres: ads tabletest [local] CREATE > TABLE(pgstat_get_entry_ref+0x440) [0x6309c3b0a530]
It looks like the dshash table used in the pgstats system uses resize(), which only specifies DSA_ALLOC_ZERO, not DSA_ALLOC_HUGE, causing issues when the table grows larger than 1 GB.
I expect that error to disappear when you replace the dsa_allocate0(...) call in dshash.c's resize function with dsa_allocate_extended(..., DSA_ALLOC_HUGE | DSA_ALLOC_ZERO) as attached, but haven't tested it due to a lack of database with millions of relations.
Can confirm that the crash no longer happens when applying your patch.
Was able to both continue the old and crashed test, as well as run a new test:
tabletest=# select count(*) from information_schema.tables; count ---------- 20000211 (1 row)
Thanks,
--
Andreas 'ads' Scherbaum German PostgreSQL User Group European PostgreSQL User Group - Board of Directors Volunteer Regional Contact, Germany - PostgreSQL Project
