Re: [HACKERS] Postgres Speed or lack thereof - Mailing list pgsql-hackers

> 
> Tom Lane wrote:
> > 
> > > Note that our mmgr adds 16 bytes to each allocation
> > > (+ some bytes in malloc) - a great overhead, yes?
> > 
> > Youch ... for a list-node-sized request, that's a lot of overhead.
> 
> And lists are very often used by planner and - never pfreed.
> 
> > Getting this right is probably going to take some thought and work,
> > but it looks worthwhile from a performance standpoint.  Maybe for 6.6?
> 
> Yes - I have to return to MVCC stuff...
> So, I consider my exercizes with mmgr as vacation from MVCC -:)
> 
> > > It shows that we should get rid of system malloc/free and do
> > > all things in mmgr itself - this would allow us much faster
> > > free memory contexts at statement/transaction end.
> > 
> > I don't think we can or should stop using malloc(), but we can
> > ask it for large blocks and do our own allocations inside those
> > blocks --- was that what you meant?
> 
> No. We could ask brk() for large blocks.
> The problem is where to handle dynamic allocations.
> As long as they are handled by malloc we can't put
> them in proper blocks of current memory context.
> But having our own handling malloc would become useless.
> 
> Vadim

We could use 4 methods for dynamic allocation:

1)    malloc/free - for persistent storage allocation

2)    palloc/pfree - for storage belonging to some context andwhich we can keep track of and free explicitly

3)    fast_palloc - for storage which impossible, too difficult or tooexpensive to keep track of. This storage should
beallocated withfast and simple inline code from bigger chunks allocated with palloc.This storage would never freed
explicitly,so that code could besimple and fast, but the big chunks would be freed automatically atthe end of the
transaction.

4)    fast_talloc - we could introduce a `tuple' context handled likefast_palloc for storage used only while processing
onetuple.This storage could be fast allocated from few big chunks allocatedwith palloc and freed explicitly after the
tuplehas been processed.This could avoid the overhead of many malloc/palloc while reducingthe overall memory usage for
transactionwhich process many rows.The total cost per tuple could be one palloc and one pfree.We could also simply
reusethe chunks for every tuple and pfree them only at the end of the transaction. This would cost one palloc/pfreeper
transaction.

This would require revising the code and changing palloc/pfree with the new
functions where appropriate, but this could be done gradually because the
old palloc/pfree are always safe.

-- 
Massimo Dal Zotto

+----------------------------------------------------------------------+
|  Massimo Dal Zotto               email: dz@cs.unitn.it               |
|  Via Marconi, 141                phone: ++39-0461534251              |
|  38057 Pergine Valsugana (TN)      www: http://www.cs.unitn.it/~dz/  |
|  Italy                             pgp: finger dz@tango.cs.unitn.it  |
+----------------------------------------------------------------------+