Thread: malloc() in Dllist

Hi,
I'm total confuse. I continue in query cache implementation and I want use
Dllist routines, but what I see --- it hardly use malloc/free. Why? With
this is a Dllist usage _very_ limited... (the catcache works with malloc? 
--- hmm interesting)
For my current situation I resolve it via some simple alloc function
changer for Dllist. But for future will probably good create specific
memory context that will based on malloc/free and in pg's sources use
_always_ and _only_ palloc in all routines. And if anyone want use
system malloc/free he can call MemoryContextSwitchTo( malloc_based_context );
Tom, assume you with some 'malloc_based_context' in your memory-management
proposal?
                    Karel

Karel Zak <zakkr@zf.jcu.cz> writes:
>  I'm total confuse. I continue in query cache implementation and I want use
> Dllist routines, but what I see --- it hardly use malloc/free. Why? With
> this is a Dllist usage _very_ limited... (the catcache works with malloc? 
> --- hmm interesting)

I think the reason Dllist still uses malloc directly is that the
selfsame code is used by libpq, and there's no palloc in the frontend
environment.  This is pretty ugly of course.  It would probably be
better to give libpq its own copy of the routines (I don't think it
needs 'em all anyway, nor ought to be polluting application namespace
with global symbols that don't start with "pq") and then the backend's
version could use palloc.  But we'd have to look at all the callers to
be sure they have current memory context set to an appropriate thing.

>  Tom, assume you with some 'malloc_based_context' in your memory-management
> proposal?

It can't be *directly* malloc based if we want to ensure that pfree()
works regardless of context type.  We need that header with the
back-link to the context to be there always.  We could have a context
type where each chunk is always a separate malloc request, and palloc
is just a thin wrapper that adds the chunk header ... but I'm not sure
what for ...
        regards, tom lane

On Fri, 19 May 2000, Tom Lane wrote:

> Karel Zak <zakkr@zf.jcu.cz> writes:
> >  I'm total confuse. I continue in query cache implementation and I want use
> > Dllist routines, but what I see --- it hardly use malloc/free. Why? With
> > this is a Dllist usage _very_ limited... (the catcache works with malloc? 
> > --- hmm interesting)
> 
> I think the reason Dllist still uses malloc directly is that the
> selfsame code is used by libpq, and there's no palloc in the frontend
> environment.  This is pretty ugly of course.  It would probably be
Yes, I know about Dllist in the frontend, but more nice is use '#if' if
anyone want share some code between backend-frontend.

> better to give libpq its own copy of the routines (I don't think it

But keep up two copy is more exigent.

> needs 'em all anyway, nor ought to be polluting application namespace
> with global symbols that don't start with "pq") and then the backend's
> version could use palloc.  But we'd have to look at all the callers to
> be sure they have current memory context set to an appropriate thing.
> 
> >  Tom, assume you with some 'malloc_based_context' in your memory-management
> > proposal?
> 
> It can't be *directly* malloc based if we want to ensure that pfree()
> works regardless of context type.  We need that header with the
> back-link to the context to be there always.  We could have a context

I said it bad. I also mean one-chunk-one-malloc with correct chunk header.
We already discussed about back-link in chunk header. I understand you.

> type where each chunk is always a separate malloc request, and palloc
> is just a thin wrapper that adds the chunk header ... but I'm not sure
> what for ...
IMHO this is correct and rightdown solution. We have system malloc in:
       bootstrap/bootstrap.c, catalog/heap.c, commands/sequence.c, executor/spi.c, lib/dllist.c,gram.c /
scan.c,port/dynloader/aix.c,port/dynloader/nextstep.c,regex/engine.c,regex/regcomp.c,storage/buffer/localbuf.c,storage/file/fd.c,tcop/postgres.c,tioga/Varray.c,tioga/tgRecipe.c,utils/fmgr/dfmgr.c,utils/cache/inval.c,utils/error/elog.c,utils/init/findbe.c,utils/init/miscinit.c
... it is long list :-(  Spit it to more memory contexts will better. I not
sure if all these routines really need directly malloc. IMHO it probably need 
some persistent contexts, and a one-chunk-one-malloc context very probably
not will need often (or never). All these are possible implement via standard 
block-based contexts, but with more separete contexts. 
It is good described in your proposal. A context for elog, parser, ... etc.    In will in 7.2 ? :-)
                        Karel