Thread: MemoryContextCheck()
Here is the patch with memory leak checker. This checker allow detect
in-chunk leaks, overwrite-next-chunk leaks and overwrite block-freeptr leaks.
A in-chunk leak --- if something overwrite space after wanted (via palloc()
size, but it is still inside chunk. For example
x = palloc(12); /* create 16b chunk */
memset(x, '#', 13);
this leak is in the current source total invisible, because chunk is 16b and
leak is in the "align space".
For this feature I add data_size to StandardChunk, and all memory which go
from AllocSetAlloc() is marked as 0x7F.
The MemoryContextCheck() is compiled '#ifdef USE_ASSERT_CHECKING'.
I add this checking to 'tcop/postgres.c' and is active after each backend
query, but it is probably not sufficient, because some MemoryContext exist
only during memory processing --- will good if someone who known where
it is needful (Tom:-) add it for others contexts;
A problem in the current source is that we have still some malloc()
allocation that is not needful and this allocation is total invisible for
all context routines. For example Dllist in backend (pretty dirty it is in
catcache where values in Dllist are palloc-ed, but list is malloc-ed).
--- and BTW. this Dllist design stand in the way for query cache :-)
Tom, if you agree I start replace some mallocs.
BTW. --- Tom, have you idea for across transaction presistent allocation for
SQL functions? (like regex - now it is via malloc)
I almost forget. I add one if() to AllocSetAlloc(), for 'size' that are
greater than ALLOC_BIGCHUNK_LIMIT is not needful check AllocSetFreeIndex(),
because 'fidx' is always 'ALLOCSET_NUM_FREELISTS - 1'. It a little brisk up
allocation for very large chunks. Right?
Karel
Attachment
> A problem in the current source is that we have still some malloc() > allocation that is not needful and this allocation is total invisible for > all context routines. For example Dllist in backend (pretty dirty it is in > catcache where values in Dllist are palloc-ed, but list is malloc-ed). > --- and BTW. this Dllist design stand in the way for query cache :-) That Dllist really has to be removed and made to use the existing routines, if possible. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 853-3000 + If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup. | Drexel Hill, Pennsylvania 19026
Bruce Momjian <pgman@candle.pha.pa.us> writes:
>> --- and BTW. this Dllist design stand in the way for query cache :-)
> That Dllist really has to be removed and made to use the existing
> routines, if possible.
Yeah, that dllist stuff is on my hit list for memory rewrite. It should
be using palloc().
There are some other places that use malloc() directly, but I'm less
concerned about fixing them.
regards, tom lane
On Mon, 10 Jul 2000, Tom Lane wrote:
And my patch ? :-)
> Bruce Momjian <pgman@candle.pha.pa.us> writes:
> >> --- and BTW. this Dllist design stand in the way for query cache :-)
>
> > That Dllist really has to be removed and made to use the existing
> > routines, if possible.
>
> Yeah, that dllist stuff is on my hit list for memory rewrite. It should
> be using palloc().
Yes. It is a long "malloc-list". IMHO if we have standard chunks and
pfree() is not a problem, we not need malloc. We can *always* use some
context. Now is malloc() very often used for persistent allocation, but it
is posible via some persistent context too. Or exist some real matter for
malloc()?
Karel
...malloc list:
bootstrap/bootstrap.c
bootstrap/bootparse.c
bootstrap/bootscanner.c
commands/async.c
commands/sequence.c
executor/spi.c
lib/dllist.c
parser/gram.c
parser/scan.c
port/dynloader/aix.c
port/dynloader/nextstep.c
port/nextstep/port.c
postmaster/postmaster.c
regex/engine.c
regex/regcomp.c
regex/regexec.c
storage/buffer/localbuf.c
storage/file/fd.c
storage/ipc/ipc.c
storage/ipc/sinval.c
tioga/Varray.c
tioga/tgRecipe.c
utils/fmgr/dfmgr.c
utils/cache/inval.c
utils/error/elog.c
utils/init/findbe.c
utils/init/globals.c
utils/init/miscinit.c
utils/misc/guc-file.l
utils/misc/guc.c
utils/misc/ps_status.c
utils/misc/guc-file.c
Applied. > > Here is the patch with memory leak checker. This checker allow detect > in-chunk leaks, overwrite-next-chunk leaks and overwrite block-freeptr leaks. > > A in-chunk leak --- if something overwrite space after wanted (via palloc() > size, but it is still inside chunk. For example > > x = palloc(12); /* create 16b chunk */ > memset(x, '#', 13); > > this leak is in the current source total invisible, because chunk is 16b and > leak is in the "align space". > > For this feature I add data_size to StandardChunk, and all memory which go > from AllocSetAlloc() is marked as 0x7F. > > The MemoryContextCheck() is compiled '#ifdef USE_ASSERT_CHECKING'. > > I add this checking to 'tcop/postgres.c' and is active after each backend > query, but it is probably not sufficient, because some MemoryContext exist > only during memory processing --- will good if someone who known where > it is needful (Tom:-) add it for others contexts; > > A problem in the current source is that we have still some malloc() > allocation that is not needful and this allocation is total invisible for > all context routines. For example Dllist in backend (pretty dirty it is in > catcache where values in Dllist are palloc-ed, but list is malloc-ed). > --- and BTW. this Dllist design stand in the way for query cache :-) > > Tom, if you agree I start replace some mallocs. > > BTW. --- Tom, have you idea for across transaction presistent allocation for > SQL functions? (like regex - now it is via malloc) > > > I almost forget. I add one if() to AllocSetAlloc(), for 'size' that are > greater than ALLOC_BIGCHUNK_LIMIT is not needful check AllocSetFreeIndex(), > because 'fidx' is always 'ALLOCSET_NUM_FREELISTS - 1'. It a little brisk up > allocation for very large chunks. Right? > > Karel Content-Description: [ application/x-gzip is not supported, skipping... ] -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 853-3000 + If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup. | Drexel Hill, Pennsylvania 19026
> > On Mon, 10 Jul 2000, Tom Lane wrote: > > And my patch ? :-) Applied. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 853-3000 + If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup. | Drexel Hill, Pennsylvania 19026
On Tue, 11 Jul 2000, Bruce Momjian wrote:
> Applied.
Thanks.
Karel