Thread: patch for memory overrun on Linux(i386)
Hi, It turns of that one of the bugs I detected with Electric Fence is caused by an assumption that memory return by palloc and malloc is aligned on 8 bytes boundaries (double alignment). Adjusting this to correspond with the reality of the malloc implementation on my pentium based linux box fixes the problem. The following simple patch to include/utils/memutils.h will fix the problem. 85c85 < #if ! defined(sco) --- > #if ! defined(sco) && !defined(linux) There is still another buffer overrun which I will try to find. The remaining one is triggered by creating a sequence. I hope this patch will be applied if it makes sense to you guys. With regards from Maurice.
Some things just require confirmation...can someone else (Thomas?) running
Linux comment on this before I apply it? I just find it kinda unexpected
to see something that doesn't work for SCO doesn't work for Linux too :)
On Sat, 21 Mar 1998, Maurice Gittens wrote:
> Hi,
>
> It turns of that one of the bugs I detected with Electric Fence
> is caused by an assumption that memory return by palloc and malloc
> is aligned on 8 bytes boundaries (double alignment).
> Adjusting this to correspond with the reality of the malloc
> implementation on my pentium based linux box fixes the problem.
>
> The following simple patch to include/utils/memutils.h
> will fix the problem.
>
> 85c85
> < #if ! defined(sco)
> ---
> > #if ! defined(sco) && !defined(linux)
>
> There is still another buffer overrun which I will try to find.
> The remaining one is triggered by creating a sequence.
>
> I hope this patch will be applied if it makes sense to
> you guys.
>
> With regards from Maurice.
>
>
Marc G. Fournier
Systems Administrator @ hub.org
primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org
Maurice Gittens wrote: > > Hi, > > It turns of that one of the bugs I detected with Electric Fence > is caused by an assumption that memory return by palloc and malloc > is aligned on 8 bytes boundaries (double alignment). > Adjusting this to correspond with the reality of the malloc > implementation on my pentium based linux box fixes the problem. > > The following simple patch to include/utils/memutils.h > will fix the problem. > > 85c85 > < #if ! defined(sco) > --- > > #if ! defined(sco) && !defined(linux) I may be off on this one, but I thought that memory alignment was a cpu, and not necessarily an OS issue. I.E. Solaris x86 might show the "misalignment", while Linux Alpha would not. Ocie Mitchell
On Sat, 21 Mar 1998 ocie@paracel.com wrote:
> Maurice Gittens wrote:
> >
> > Hi,
> >
> > It turns of that one of the bugs I detected with Electric Fence
> > is caused by an assumption that memory return by palloc and malloc
> > is aligned on 8 bytes boundaries (double alignment).
> > Adjusting this to correspond with the reality of the malloc
> > implementation on my pentium based linux box fixes the problem.
> >
> > The following simple patch to include/utils/memutils.h
> > will fix the problem.
> >
> > 85c85
> > < #if ! defined(sco)
> > ---
> > > #if ! defined(sco) && !defined(linux)
>
> I may be off on this one, but I thought that memory alignment was a
> cpu, and not necessarily an OS issue. I.E. Solaris x86 might show the
> "misalignment", while Linux Alpha would not.
Ummm...you are comparing both two different CPU and two different
OSs here...more appropriately to your argument would be Linux/x86 vs
Linux/Alpha, wouldn't it? If it was a CPU issue...?
Marc G. Fournier
Systems Administrator @ hub.org
primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org
>
>
> Some things just require confirmation...can someone else (Thomas?) running
> Linux comment on this before I apply it? I just find it kinda unexpected
> to see something that doesn't work for SCO doesn't work for Linux too :)
On BSDI, malloc man pages says:
The allocated space is suitably aligned (after possible pointer coercion)
for storage of any type of object.
I don't believe this is a standard, and I think Linux may not follow it.
I think I heard somewhere they don't. We certainly need to have this
checked before use in a patch.
--
Bruce Momjian | 830 Blythe Avenue
maillist@candle.pha.pa.us | Drexel Hill, Pennsylvania 19026
+ If your life is a hard drive, | (610) 353-9879(w)
+ Christ can be your backup. | (610) 853-3000(h)
On Sat, 21 Mar 1998, Bruce Momjian wrote:
> >
> >
> > Some things just require confirmation...can someone else (Thomas?) running
> > Linux comment on this before I apply it? I just find it kinda unexpected
> > to see something that doesn't work for SCO doesn't work for Linux too :)
>
> On BSDI, malloc man pages says:
>
> The allocated space is suitably aligned (after possible pointer coercion)
> for storage of any type of object.
>
> I don't believe this is a standard, and I think Linux may not follow it.
> I think I heard somewhere they don't. We certainly need to have this
> checked before use in a patch.
>
>
> --
> Bruce Momjian | 830 Blythe Avenue
> maillist@candle.pha.pa.us | Drexel Hill, Pennsylvania 19026
> + If your life is a hard drive, | (610) 353-9879(w)
> + Christ can be your backup. | (610) 853-3000(h)
>
From the linux man page:
For calloc() and malloc(), the value returned is a pointer
to the allocated memory, which is suitably aligned for any
kind of variable, or NULL if the request fails.
realloc() returns a pointer to the newly allocated memory,
which is suitably aligned for any kind of variable and may
be different from ptr, or NULL if the request fails or if
size was equal to 0. If realloc() fails the original
block is left untouched - it is not freed or moved.
Marc Zuckman
marc@fallon.classyad.com
_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_
_ Visit The Home and Condo MarketPlace _
_ http://www.ClassyAd.com _
_ _
_ FREE basic property listings/advertisements and searches. _
_ _
_ Try our premium, yet inexpensive services for a real _
_ selling or buying edge! _
_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_
> >From the linux man page:
> For calloc() and malloc(), the value returned is a pointer
> to the allocated memory, which is suitably aligned for any
> kind of variable, or NULL if the request fails.
OK, then why is electric fence telling him to make DOUBLEALIGN match
INTALIGN? You would think that if this change made sense, we would be
having some Linux problems, but we don't.
---------------------------------------------------------------------------
#if ! defined(sco)
#define DOUBLEALIGN(LEN)\
(((long)(LEN) + (sizeof (double) - 1)) & ~(sizeof (double) -1))
#define MAXALIGN(LEN)\
(((long)(LEN) + (sizeof (double) - 1)) & ~(sizeof (double) -1))
#else
#define DOUBLEALIGN(LEN) INTALIGN(LEN)
#define MAXALIGN(LEN) INTALIGN(LEN)
#endif
--
Bruce Momjian | 830 Blythe Avenue
maillist@candle.pha.pa.us | Drexel Hill, Pennsylvania 19026
+ If your life is a hard drive, | (610) 353-9879(w)
+ Christ can be your backup. | (610) 853-3000(h)
> > Some things just require confirmation...can someone else (Thomas?) running > > Linux comment on this before I apply it? I just find it kinda unexpected > > to see something that doesn't work for SCO doesn't work for Linux too :) > > On BSDI, malloc man pages says: > > The allocated space is suitably aligned (after possible pointer coercion) > for storage of any type of object. > > I don't believe this is a standard, and I think Linux may not follow it. > I think I heard somewhere they don't. We certainly need to have this > checked before use in a patch. The alignment of a pointer returned from malloc() is platform specific. The BSDI man page above gets the sense of it right, and as far as I am aware all mallocs conform to this restriction. In particular x86 Linux malloc behaves exactly that way. Of course on x86 there is no special requirement any type to be aligned at all. Double do not need to be 8 byte aligned. Ints do not need to be on an even address etc. So malloc can return a pointer to any address it wants on an x86 and still conform to the restriction. On a Sparc or an Alpha, objects must to be aligned to the sizeof the type. So shorts are 2 byte aligned, ints 4 byte, and doubles 8 byte. Since malloc has no idea what you are allocating it has to return an 8 byte aligned pointer. But all this is a bit of a red herring. The original bug if I recall was caused by the caller assuming that the return from palloc was 8 byte aligned. Since the caller "knew" that the pointer was 8 byte aligned it then went on to do some "fancy" pointer arithmetic to arrive at an offset. This is what is broken, not the Linux malloc(). That said, I have a plan in mind to get a nice performance boost for palloc(). As a side effect, I will guarantee 8 byte alignment of pointers returned from palloc(). -dg David Gould dg@illustra.com 510.628.3783 or 510.305.9468 Informix Software (No, really) 300 Lakeside Drive Oakland, CA 94612 - I realize now that irony has no place in business communications.
> That said, I have a plan in mind to get a nice performance boost for palloc(). > As a side effect, I will guarantee 8 byte alignment of pointers returned > from palloc(). Oooh, sounds nice. -- Bruce Momjian | 830 Blythe Avenue maillist@candle.pha.pa.us | Drexel Hill, Pennsylvania 19026 + If your life is a hard drive, | (610) 353-9879(w) + Christ can be your backup. | (610) 853-3000(h)
> > > >From the linux man page: > > For calloc() and malloc(), the value returned is a pointer > > to the allocated memory, which is suitably aligned for any > > kind of variable, or NULL if the request fails. > > OK, then why is electric fence telling him to make DOUBLEALIGN match > INTALIGN? You would think that if this change made sense, we would be > having some Linux problems, but we don't. > > --------------------------------------------------------------------------- > > > #if ! defined(sco) > #define DOUBLEALIGN(LEN)\ > (((long)(LEN) + (sizeof (double) - 1)) & ~(sizeof (double) -1)) > > #define MAXALIGN(LEN)\ > (((long)(LEN) + (sizeof (double) - 1)) & ~(sizeof (double) -1)) > #else > #define DOUBLEALIGN(LEN) INTALIGN(LEN) > #define MAXALIGN(LEN) INTALIGN(LEN) > #endif > Hi Hackers, it seems that Linux malloc, or better libc-5.4.23 malloc, is doing what is declared in the man page, i.e. returning "memory which is suitably aligned for any kind of variable". I have some malloc traces and all malloc results are aligned to double size (8 bytes): op caller ptr size -------------------------------------------- malloc 400c32b2 080a06c8 25 malloc 400c32b2 080a06e8 13 malloc 400c32b2 080a0700 26 malloc 400c32b2 080a0720 11 malloc 400c32b2 080a0730 50 malloc 400c32b2 080a0768 14 malloc 400c32b2 080a0780 17 malloc 400c32b2 080a0798 24 malloc 400c32b2 080a07b8 27 malloc 400c32b2 080a07d8 817 malloc 400c32b2 080a0b10 11 malloc 400c32b2 080a0b20 14 malloc 400c32b2 080a0b38 164 malloc 400c32b2 080a0be0 11 malloc 400c32b2 080a0bf0 14 malloc 400c32b2 080a0c08 25 malloc 400c32b2 080a0c28 16 malloc 400c32b2 080a0c40 35 malloc 400c32b2 080a0c68 9 malloc 400c32b2 080a0c78 7 So I don't see where is the problem. Massimo Dal Zotto +----------------------------------------------------------------------+ | Massimo Dal Zotto e-mail: dz@cs.unitn.it | | Via Marconi, 141 phone: ++39-461-534251 | | 38057 Pergine Valsugana (TN) www: http://www.cs.unitn.it/~dz/ | | Italy pgp: finger dz@tango.cs.unitn.it | +----------------------------------------------------------------------+