Thread: patch for memory overrun on Linux(i386)

patch for memory overrun on Linux(i386)

From
Maurice Gittens
Date:
Hi,

It turns of that one of the bugs I detected with Electric Fence
is caused by an assumption that memory return by palloc and malloc
is aligned on 8 bytes boundaries (double alignment).
Adjusting this to correspond with the reality of the malloc
implementation on my pentium based linux box fixes the problem.

The following simple patch to include/utils/memutils.h
will fix the problem.

85c85
< #if ! defined(sco)
---
> #if ! defined(sco) && !defined(linux)

There is still another buffer overrun which I will try to find.
The remaining one is triggered by creating a sequence.

I hope this patch will be applied if it makes sense to
you guys.

With regards from Maurice.



Re: [HACKERS] patch for memory overrun on Linux(i386)

From
The Hermit Hacker
Date:
Some things just require confirmation...can someone else (Thomas?) running
Linux comment on this before I apply it?  I just find it kinda unexpected
to see something that doesn't work for SCO doesn't work for Linux too :)

On Sat, 21 Mar 1998, Maurice Gittens wrote:

> Hi,
>
> It turns of that one of the bugs I detected with Electric Fence
> is caused by an assumption that memory return by palloc and malloc
> is aligned on 8 bytes boundaries (double alignment).
> Adjusting this to correspond with the reality of the malloc
> implementation on my pentium based linux box fixes the problem.
>
> The following simple patch to include/utils/memutils.h
> will fix the problem.
>
> 85c85
> < #if ! defined(sco)
> ---
> > #if ! defined(sco) && !defined(linux)
>
> There is still another buffer overrun which I will try to find.
> The remaining one is triggered by creating a sequence.
>
> I hope this patch will be applied if it makes sense to
> you guys.
>
> With regards from Maurice.
>
>

Marc G. Fournier
Systems Administrator @ hub.org
primary: scrappy@hub.org           secondary: scrappy@{freebsd|postgresql}.org


Re: [HACKERS] patch for memory overrun on Linux(i386)

From
ocie@paracel.com
Date:
Maurice Gittens wrote:
>
> Hi,
>
> It turns of that one of the bugs I detected with Electric Fence
> is caused by an assumption that memory return by palloc and malloc
> is aligned on 8 bytes boundaries (double alignment).
> Adjusting this to correspond with the reality of the malloc
> implementation on my pentium based linux box fixes the problem.
>
> The following simple patch to include/utils/memutils.h
> will fix the problem.
>
> 85c85
> < #if ! defined(sco)
> ---
> > #if ! defined(sco) && !defined(linux)

I may be off on this one, but I thought that memory alignment was a
cpu, and not necessarily an OS issue.  I.E. Solaris x86 might show the
"misalignment", while Linux Alpha would not.

Ocie Mitchell

Re: [HACKERS] patch for memory overrun on Linux(i386)

From
The Hermit Hacker
Date:
On Sat, 21 Mar 1998 ocie@paracel.com wrote:

> Maurice Gittens wrote:
> >
> > Hi,
> >
> > It turns of that one of the bugs I detected with Electric Fence
> > is caused by an assumption that memory return by palloc and malloc
> > is aligned on 8 bytes boundaries (double alignment).
> > Adjusting this to correspond with the reality of the malloc
> > implementation on my pentium based linux box fixes the problem.
> >
> > The following simple patch to include/utils/memutils.h
> > will fix the problem.
> >
> > 85c85
> > < #if ! defined(sco)
> > ---
> > > #if ! defined(sco) && !defined(linux)
>
> I may be off on this one, but I thought that memory alignment was a
> cpu, and not necessarily an OS issue.  I.E. Solaris x86 might show the
> "misalignment", while Linux Alpha would not.

    Ummm...you are comparing both two different CPU and two different
OSs here...more appropriately to your argument would be Linux/x86 vs
Linux/Alpha, wouldn't it?  If it was a CPU issue...?

Marc G. Fournier
Systems Administrator @ hub.org
primary: scrappy@hub.org           secondary: scrappy@{freebsd|postgresql}.org


Re: [HACKERS] patch for memory overrun on Linux(i386)

From
Bruce Momjian
Date:
>
>
> Some things just require confirmation...can someone else (Thomas?) running
> Linux comment on this before I apply it?  I just find it kinda unexpected
> to see something that doesn't work for SCO doesn't work for Linux too :)

On BSDI, malloc man pages says:

     The allocated space is suitably aligned (after possible pointer coercion)
     for storage of any type of object.

I don't believe this is a standard, and I think Linux may not follow it.
I think I heard somewhere they don't.  We certainly need to have this
checked before use in a patch.


--
Bruce Momjian                          |  830 Blythe Avenue
maillist@candle.pha.pa.us              |  Drexel Hill, Pennsylvania 19026
  +  If your life is a hard drive,     |  (610) 353-9879(w)
  +  Christ can be your backup.        |  (610) 853-3000(h)

Re: [HACKERS] patch for memory overrun on Linux(i386)

From
Marc Howard Zuckman
Date:
On Sat, 21 Mar 1998, Bruce Momjian wrote:

> >
> >
> > Some things just require confirmation...can someone else (Thomas?) running
> > Linux comment on this before I apply it?  I just find it kinda unexpected
> > to see something that doesn't work for SCO doesn't work for Linux too :)
>
> On BSDI, malloc man pages says:
>
>      The allocated space is suitably aligned (after possible pointer coercion)
>      for storage of any type of object.
>
> I don't believe this is a standard, and I think Linux may not follow it.
> I think I heard somewhere they don't.  We certainly need to have this
> checked before use in a patch.
>
>
> --
> Bruce Momjian                          |  830 Blythe Avenue
> maillist@candle.pha.pa.us              |  Drexel Hill, Pennsylvania 19026
>   +  If your life is a hard drive,     |  (610) 353-9879(w)
>   +  Christ can be your backup.        |  (610) 853-3000(h)
>

From the linux man page:
 For calloc() and malloc(), the value returned is a pointer
       to the allocated memory, which is suitably aligned for any
       kind of variable, or NULL if the request fails.

       realloc() returns a pointer to the newly allocated memory,
       which is suitably aligned for any kind of variable and may
       be  different from ptr, or NULL if the request fails or if
       size was equal to 0.   If  realloc()  fails  the  original
       block is left untouched - it is not freed or moved.




Marc Zuckman
marc@fallon.classyad.com

_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_
_     Visit The Home and Condo MarketPlace              _
_          http://www.ClassyAd.com                  _
_                                  _
_  FREE basic property listings/advertisements and searches.  _
_                                  _
_  Try our premium, yet inexpensive services for a real          _
_   selling or buying edge!                      _
_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_


Re: [HACKERS] patch for memory overrun on Linux(i386)

From
Bruce Momjian
Date:
> >From the linux man page:
>  For calloc() and malloc(), the value returned is a pointer
>        to the allocated memory, which is suitably aligned for any
>        kind of variable, or NULL if the request fails.

OK, then why is electric fence telling him to make DOUBLEALIGN match
INTALIGN?  You would think that if this change made sense, we would be
having some Linux problems, but we don't.

---------------------------------------------------------------------------


#if ! defined(sco)
#define DOUBLEALIGN(LEN)\
        (((long)(LEN) + (sizeof (double) - 1)) & ~(sizeof (double) -1))

#define MAXALIGN(LEN)\
        (((long)(LEN) + (sizeof (double) - 1)) & ~(sizeof (double) -1))
#else
#define DOUBLEALIGN(LEN) INTALIGN(LEN)
#define MAXALIGN(LEN)    INTALIGN(LEN)
#endif



--
Bruce Momjian                          |  830 Blythe Avenue
maillist@candle.pha.pa.us              |  Drexel Hill, Pennsylvania 19026
  +  If your life is a hard drive,     |  (610) 353-9879(w)
  +  Christ can be your backup.        |  (610) 853-3000(h)

Re: [HACKERS] patch for memory overrun on Linux(i386)

From
dg@illustra.com (David Gould)
Date:
> > Some things just require confirmation...can someone else (Thomas?) running
> > Linux comment on this before I apply it?  I just find it kinda unexpected
> > to see something that doesn't work for SCO doesn't work for Linux too :)
>
> On BSDI, malloc man pages says:
>
>      The allocated space is suitably aligned (after possible pointer coercion)
>      for storage of any type of object.
>
> I don't believe this is a standard, and I think Linux may not follow it.
> I think I heard somewhere they don't.  We certainly need to have this
> checked before use in a patch.

The alignment of a pointer returned from malloc() is platform specific. The
BSDI man page above gets the sense of it right, and as far as I am aware
all mallocs conform to this restriction. In particular x86 Linux malloc
behaves exactly that way.

Of course on x86 there is no special requirement any type to be aligned at all.
Double do not need to be 8 byte aligned. Ints do not need to be on an even
address etc. So malloc can return a pointer to any address it wants on an x86
and still conform to the restriction.

On a Sparc or an Alpha, objects must to be aligned to the sizeof the type.
So shorts are 2 byte aligned, ints 4 byte, and doubles 8 byte. Since malloc
has no idea what you are allocating it has to return an 8 byte aligned pointer.

But all this is a bit of a red herring. The original bug if I recall was
caused by the caller assuming that the return from palloc was 8 byte aligned.
Since the caller "knew" that the pointer was 8 byte aligned it then went on
to do some "fancy" pointer arithmetic to arrive at an offset. This is what
is broken, not the Linux malloc().

That said, I have a plan in mind to get a nice performance boost for palloc().
As a side effect, I will guarantee 8 byte alignment of pointers returned
from palloc().

-dg

David Gould            dg@illustra.com           510.628.3783 or 510.305.9468
Informix Software  (No, really)         300 Lakeside Drive  Oakland, CA 94612
 - I realize now that irony has no place in business communications.



Re: [HACKERS] patch for memory overrun on Linux(i386)

From
Bruce Momjian
Date:
> That said, I have a plan in mind to get a nice performance boost for palloc().
> As a side effect, I will guarantee 8 byte alignment of pointers returned
> from palloc().

Oooh, sounds nice.

--
Bruce Momjian                          |  830 Blythe Avenue
maillist@candle.pha.pa.us              |  Drexel Hill, Pennsylvania 19026
  +  If your life is a hard drive,     |  (610) 353-9879(w)
  +  Christ can be your backup.        |  (610) 853-3000(h)

Re: [HACKERS] patch for memory overrun on Linux(i386)

From
Massimo Dal Zotto
Date:
>
> > >From the linux man page:
> >  For calloc() and malloc(), the value returned is a pointer
> >        to the allocated memory, which is suitably aligned for any
> >        kind of variable, or NULL if the request fails.
>
> OK, then why is electric fence telling him to make DOUBLEALIGN match
> INTALIGN?  You would think that if this change made sense, we would be
> having some Linux problems, but we don't.
>
> ---------------------------------------------------------------------------
>
>
> #if ! defined(sco)
> #define DOUBLEALIGN(LEN)\
>         (((long)(LEN) + (sizeof (double) - 1)) & ~(sizeof (double) -1))
>
> #define MAXALIGN(LEN)\
>         (((long)(LEN) + (sizeof (double) - 1)) & ~(sizeof (double) -1))
> #else
> #define DOUBLEALIGN(LEN) INTALIGN(LEN)
> #define MAXALIGN(LEN)    INTALIGN(LEN)
> #endif
>

Hi Hackers,

it seems that Linux malloc, or better libc-5.4.23 malloc, is doing what is
declared in the man page, i.e. returning "memory which is suitably aligned
for any kind of variable". I have some malloc traces and all malloc results
are aligned to double size (8 bytes):

op    caller        ptr        size
--------------------------------------------
malloc    400c32b2    080a06c8    25
malloc    400c32b2    080a06e8    13
malloc    400c32b2    080a0700    26
malloc    400c32b2    080a0720    11
malloc    400c32b2    080a0730    50
malloc    400c32b2    080a0768    14
malloc    400c32b2    080a0780    17
malloc    400c32b2    080a0798    24
malloc    400c32b2    080a07b8    27
malloc    400c32b2    080a07d8    817
malloc    400c32b2    080a0b10    11
malloc    400c32b2    080a0b20    14
malloc    400c32b2    080a0b38    164
malloc    400c32b2    080a0be0    11
malloc    400c32b2    080a0bf0    14
malloc    400c32b2    080a0c08    25
malloc    400c32b2    080a0c28    16
malloc    400c32b2    080a0c40    35
malloc    400c32b2    080a0c68    9
malloc    400c32b2    080a0c78    7

So I don't see where is the problem.

Massimo Dal Zotto

+----------------------------------------------------------------------+
|  Massimo Dal Zotto                e-mail:  dz@cs.unitn.it            |
|  Via Marconi, 141                 phone:  ++39-461-534251            |
|  38057 Pergine Valsugana (TN)     www:  http://www.cs.unitn.it/~dz/  |
|  Italy                            pgp:  finger dz@tango.cs.unitn.it  |
+----------------------------------------------------------------------+