Re: Support allocating memory for large strings - Mailing list pgsql-hackers

From Tom Lane
Subject Re: Support allocating memory for large strings
Date
Msg-id 1357347.1762569165@sss.pgh.pa.us
Whole thread Raw
In response to Support allocating memory for large strings  (Maxim Zibitsker <max.zibitsker@gmail.com>)
List pgsql-hackers
Maxim Zibitsker <max.zibitsker@gmail.com> writes:
> PostgreSQL's MaxAllocSize limit prevents storing individual variable-length character strings exceeding ~1GB, causing
"invalidmemory alloc request size" errors during INSERT operations on tables with large text columns. 

This is news to no one.  We are not especially interested in trying to
relax that limit, because doing so would bleed over into approximately
everything in the backend, and create opportunities for
integer-overflow bugs in many places that are perfectly okay today.
The cost-benefit ratio for changing this decision is horrible.

> The attached patches demonstrates a proof of concept using
> palloc_extended with MCXT_ALLOC_HUGE in the write path.

"Proof of concept"?  This can't possibly fix your problem, because it
does nothing for the fact that tuple size fields are still limited
to 1GB, as are varlena headers for individual fields.  A serious
attack on this limitation, at a guess, would require a patch on the
order of 100K lines, and that might be an underestimate.

            regards, tom lane



pgsql-hackers by date:

Previous
From: Maxim Zibitsker
Date:
Subject: Support allocating memory for large strings
Next
From: Bruce Momjian
Date:
Subject: Re: Extra blank line in StrategyGetBuffer