The following bug has been logged on the website:
Bug reference: 14051
Logged by: David Gould
Email address: daveg@sonic.net
PostgreSQL version: 9.4.6
Operating system: Linux
Description:
Creating a GIN index that contains a large number of duplicate keys fails.
create table ginfail as
select i::int, array[1::int] as manydups
from generate_series(1, 180000000) x(i);
SELECT 180000000
create index ginfail_gin on ginfail using gin(manydups);
ERROR: invalid memory alloc request size 2013265920
(gdb) bt
#0 errstart (elevel=20, filename=0x129d55e "mcxt.c", lineno=724,
funcname=0x129d5b7 "repalloc", domain=0x0)
at elog.c:233
#1 0x000000000085f120 in elog_finish (elevel=20, fmt=0x129d568 "invalid
memory alloc request size %zu") at elog.c:1359
#2 0x0000000000879bbd in repalloc (pointer=Unhandled dwarf expression
opcode 0xf3
) at mcxt.c:724
#3 0x0000000000594340 in ginCombineData (existing=0x33cd230,
newdata=0x7fff895cb7b0, arg=0x7fff895ce320)
at ginbulk.c:42
#4 0x0000000000877316 in rb_insert (rb=0x3578da0, data=0x7fff895cb7b0,
isNew=0x7fff895cb7af "") at rbtree.c:425
#5 0x000000000059452d in ginInsertBAEntry (accum=0x7fff895ce320,
heapptr=0x35794f4, attnum=1, entries=0x3396da0,
categories=0x3396db8 "\003\362Se\302wTe0\360\070\003", nentries=1) at
ginbulk.c:157
#6 ginInsertBAEntries (accum=0x7fff895ce320, heapptr=0x35794f4, attnum=1,
entries=0x3396da0,
categories=0x3396db8 "\003\362Se\302wTe0\360\070\003", nentries=1) at
ginbulk.c:229
#7 0x000000000058873c in ginHeapTupleBulkInsert (index=Unhandled dwarf
expression opcode 0xf3
) at gininsert.c:260
src/backend/access/gin/ginbulk.c:
/* Combiner function for rbtree.c */
static void
ginCombineData(RBNode *existing, const RBNode *newdata, void *arg)
{
GinEntryAccumulator *eo = (GinEntryAccumulator *) existing;
const GinEntryAccumulator *en = (const GinEntryAccumulator *) newdata;
BuildAccumulator *accum = (BuildAccumulator *) arg;
/*
* Note this code assumes that newdata contains only one itempointer.
*/
if (eo->count >= eo->maxcount)
{
accum->allocatedMemory -= GetMemoryChunkSpace(eo->list);
eo->maxcount *= 2;
eo->list = (ItemPointerData *)
repalloc(eo->list, sizeof(ItemPointerData) * eo->maxcount);
accum->allocatedMemory += GetMemoryChunkSpace(eo->list);
}
The problem appears to be that the repalloc() of eo->list to double the size
of the allocation runs over MaxAllocSize. Perhaps it could use
repalloc_huge() instead?
-dg
