Thread: 64-bit hashjoins

64-bit hashjoins

From
Erik Riedel
Date:
Platform:  Alpha, Digital UNIX 4.0D 
Software:  PostgreSQL 6.5 snaphot (11 March 1999)

I have two tables as follows:

Table    = orderfoo
+-------------------------+----------------------------+-------+
|              Field      |              Type          | Length|
+-------------------------+----------------------------+-------+
| o_orderkey              | int4 not null              |     4 |
| o_custkey               | int4 not null              |     4 |
| o_orderstatus           | char() not null            |     1 |
| o_totalprice            | float8 not null            |     8 |
| o_orderdate             | date                       |     4 |
| o_orderpriority         | char() not null            |    15 |
| o_clerk                 | char() not null            |    15 |
| o_shippriority          | int4 not null              |     4 |
| o_comment               | char() not null            |    79 |
+-------------------------+----------------------------+-------+
Index:    orderfoo_index_


Table    = customer
+-------------------------+----------------------------+-------+
|              Field      |              Type          | Length|
+-------------------------+----------------------------+-------+
| c_custkey               | int4 not null              |     4 |
| c_name                  | char() not null            |    25 |
| c_address               | char() not null            |    40 |
| c_nationkey             | int4 not null              |     4 |
| c_phone                 | char() not null            |    15 |
| c_acctbal               | float8 not null            |     8 |
| c_mktsegment            | char() not null            |    10 |
| c_comment               | char() not null            |   117 |
+-------------------------+----------------------------+-------+
Index:    customer_index_

and a query:

--
-- Query 3
--
select l_orderkey, sum(l_extendedprice*(1-l_discount)) as revenue,
o_orderdate, o_shippriority
from customer, orderfoo, lineitem
where c_mktsegment = 'BUILDING'
and c_custkey = o_custkey
and l_orderkey = o_orderkey
and o_orderdate < '1995-03-15'
and l_shipdate > '1995-03-15'
group by l_orderkey, o_orderdate, o_shippriority
order by revenue desc, o_orderdate;

whose plan includes the segment that hash joins the two tables listed
above:
   ->  Hash Join  (cost=12268.29 size=5284 width=20)     ->  Seq Scan on orderfoo  (cost=8609.00 size=72911 width=16)
 ->  Hash  (cost=0.00 size=0 width=0)       ->  Seq Scan on customer  (cost=1031.00 size=1087 width=4)
 

and which crashes the backend during the Hash Join with:
 Unaligned access pid=28933 <postgres> va=0x14027e934    pc=0x120099430 ra=0x14027e93c inst=0xb74e0010

followed swiftly by:
 Bus error (core dumped)

(note that the unaligned access and the bus error may be unrelated
events, but I suspect not - more below)

we then have a dbx backtrace that shows:
 0 ExecScanHashBucket(hjstate = 0x140238478, bucket = 0x14027e8d8,
curtuple = 0x11fffd000, hjclauses = 0x140234118, econtext = 0x140238588)
["nodeHash.c":706, 0x120099434]  1 ExecHashJoin(node = 0x140234258) ["nodeHashjoin.c":288, 0x120099f00]  2
ExecProcNode(node= 0x140234258, parent = 0x1402347d8)
 
["execProcnode.c":315, 0x120090ffc]  3 ExecNestLoop(node = 0x1402347d8, parent = 0x140236108)
["nodeNestloop.c":160, 0x12009d104]  4 ExecProcNode(node = 0x1402347d8, parent = 0x140236108)
["execProcnode.c":279, 0x120090e7c]  5 createfirstrun(node = 0x140236108) ["psort.c":409, 0x1201762fc]  6
initialrun(node= 0x140236108) ["psort.c":291, 0x120176024]  7 psort_begin(node = 0x140236108, nkeys = 5159, key =
0x14023c560)
["psort.c":150, 0x120175e64]  8 ExecSort(node = 0x140236108) ["nodeSort.c":156, 0x12009e1c4]  9 ExecProcNode(node =
0x140236108,parent = 0x1402366d8)
 
["execProcnode.c":295, 0x120090f0c] 10 ExecGroupEveryTuple(node = 0x1402366d8) ["nodeGroup.c":104, 0x12009fab4] 11
ExecGroup(node= (nil)) ["nodeGroup.c":56, 0x12009fa0c] 12 ExecProcNode(node = 0x1402366d8, parent = 0x140237020)
 
["execProcnode.c":303, 0x120090f6c] 13 ExecAgg(node = 0x140237020) ["nodeAgg.c":243, 0x120097064] 14 ExecProcNode(node
=0x140237020, parent = 0x140237478)
 
["execProcnode.c":307, 0x120090f9c] 15 createfirstrun(node = 0x140237478) ["psort.c":409, 0x1201762fc]
More (n if no)? 16 initialrun(node = 0x140237478) ["psort.c":291, 0x120176024] 17 psort_begin(node = 0x140237478, nkeys
=5159, key = 0x14023cd80)
 
["psort.c":150, 0x120175e64] 18 ExecSort(node = 0x140237478) ["nodeSort.c":156, 0x12009e1c4] 19 ExecProcNode(node =
0x140237478,parent = 0x140237478)
 
["execProcnode.c":295, 0x120090f0c] 20 ExecutePlan(estate = 0x140237ea8, plan = 0x140237478, direction =
ForwardScanDirection, destfunc = 0x14001b118) ["execMain.c":985,
0x12008f2bc] 21 ExecutorRun(queryDesc = (nil), estate = 0x140237ea8, limoffset =
0x1, limcount = 0x14001b118) ["execMain.c":360, 0x12008e780] 22 ProcessQueryDesc(queryDesc = 0x140237e78, limoffset =
(nil),
limcount = (nil)) ["pquery.c":334, 0x120123a1c]  23
ProcessQuery(parsetree = 0x1401f3a88, plan = 0x140237478, dest =
536858624) ["pquery.c":377, 0x120123b1c] 24 pg_exec_query_dest(query_string = 0x11fffd7c0 = "select l_orderkey,
sum(l_extendedprice*(1-l_discount)) as revenue, o_orderdate,
o_shippriority from customer, orderfoo, lineitem where c\...",
aclOverride = '^@') ["postgres.c":805, 0x1201217bc] 25 PostgresMain(argv = 0x11ffff808, real_argv = 0x11ffff808)
["postgres.c":703, 0x12012302c] 26 main(argv = 0x11ffff808) ["main.c":103, 0x1200ae28c]
(dbx) 

the offending code in nodeHash.c is near line 700 as:

...
 if (curtuple == NULL)   heapTuple = (HeapTuple)      LONGALIGN(ABSADDR(bucket->top)); else   heapTuple = (HeapTuple)
 LONGALIGN(((char *) curtuple + curtuple->t_len + HEAPTUPLESIZE));
 
 while (heapTuple < (HeapTuple) ABSADDR(bucket->bottom)) {
     heapTuple->t_data = (HeapTupleHeader)        ((char *) heapTuple + HEAPTUPLESIZE);
     inntuple = ExecStoreTuple(heapTuple,      /* tuple to store */                  hjstate->hj_HashTupleSlot,
/*slot */                  InvalidBuffer,        /* tuple has no buffer */                  false);       /* do not
pfreethis tuple */
 

...

it crashes at the ExecStoreTuple().

Since it gives that unaligned access error beforehand, I suspect the
LONGALIGN() macros in the lines above are the culprits somehow.  This
takes me to include/utils/memutils.h which has:


#if (defined(sun) && ! defined(sparc)) || defined(m68k)
#define LONGALIGN(LEN)  SHORTALIGN(LEN)
#elif defined (__alpha)
/* * even though "long alignment" should really be on 8-byte boundaries for * linuxalpha, we want the strictest
alignmentto be on 4-byte (int) * boundaries, because otherwise things break when they try to use the * FormData_pg_*
structures. --djm 12/12/96 */
 
#define LONGALIGN(LEN)\               (((long)(LEN) + (sizeof (int) - 1)) & ~(sizeof (int) -1))
#else
#define LONGALIGN(LEN)\               (((long)(LEN) + (sizeof (long) - 1)) & ~(sizeof (long) -1))
#endif


since I am __alpha (but __osf__, not linux), I get the version with
the sizeof(int) instead of the sizeof(long).

Can someone explain the comment from djm to me (or is djm still
listening somewhere?).  At first blush, I suspect that I actually
_want_ it to do the latter version of LONGALIGN(), since my longs
really are 8 bytes.  But when I try to do that instead, I am unable to
even run "initdb" - dies with an error like "attribute not
found/invalid" (sorry, scrolled away the window with the actual error
- I can re-create if the exact message would help anyone).

Anyone have suggestions on how I might proceed?  Are there known
problems (or known-workings) of HashJoin on 64-bit platforms?

Erik Riedel
Carnegie Mellon University
www.cs.cmu.edu/~riedel



Re: [HACKERS] 64-bit hashjoins

From
Bruce Momjian
Date:
> since I am __alpha (but __osf__, not linux), I get the version with
> the sizeof(int) instead of the sizeof(long).
> 
> Can someone explain the comment from djm to me (or is djm still
> listening somewhere?).  At first blush, I suspect that I actually
> _want_ it to do the latter version of LONGALIGN(), since my longs
> really are 8 bytes.  But when I try to do that instead, I am unable to
> even run "initdb" - dies with an error like "attribute not
> found/invalid" (sorry, scrolled away the window with the actual error
> - I can re-create if the exact message would help anyone).
> 
> Anyone have suggestions on how I might proceed?  Are there known
> problems (or known-workings) of HashJoin on 64-bit platforms?
> 

Good analysis.  I never have understood the alpha issues.  I realize
that initdb does not work in those cases, but never understood why.

Not sure who djm is.

--  Bruce Momjian                        |  http://www.op.net/~candle maillist@candle.pha.pa.us            |  (610)
853-3000+  If your life is a hard drive,     |  830 Blythe Avenue +  Christ can be your backup.        |  Drexel Hill,
Pennsylvania19026
 


Re: [HACKERS] 64-bit hashjoins

From
Tom Lane
Date:
Erik Riedel <riedel+@CMU.EDU> writes:
> Platform:  Alpha, Digital UNIX 4.0D 
> [ memutils.h says ]
>  /*
>   * even though "long alignment" should really be on 8-byte boundaries for
>   * linuxalpha, we want the strictest alignment to be on 4-byte (int)
>   * boundaries, because otherwise things break when they try to use the
>   * FormData_pg_* structures.  --djm 12/12/96
>   */

I remember looking at that code and saying "Huh?  You can't do that!".
I kept my fingers off it because I didn't have direct proof that it
was broken ... but it sounds like you do.

> Can someone explain the comment from djm to me (or is djm still
> listening somewhere?).  At first blush, I suspect that I actually
> _want_ it to do the latter version of LONGALIGN(), since my longs
> really are 8 bytes.  But when I try to do that instead, I am unable to
> even run "initdb" - dies with an error like "attribute not
> found/invalid"

Yeah, that's about what I'd expect.  The point is that the struct
layouts found in include/catalog/pg_*.h for system table records
have to match the actual physical layout of tuples on disk.  What
you are probably running into is that the attribute size/alignment
calculations done by the heaptuple code using the declared column data
types fail to match up with the struct field alignment done by the
compiler.

My guess is that either a struct field is being declared "long" when
it really oughta be "int", or some part of the tuple storage routines
is applying LONGALIGN() when it only oughta apply INTALIGN().  This
is something that would be difficult to track down or verify without
a box on which sizeof(int) != sizeof(long), so I haven't gone after it.
If you have time, please leave memutils.h with the more reasonable
looking definition of LONGALIGN() and go looking to find out which
system table has the sizing conflict.

BTW, we'd run into this same problem if any of the system tables had
a float8 column, since the alignment of those is platform-dependent.
Memo to hackers: stay away from float8 in sys tables.
        regards, tom lane


Re: [HACKERS] 64-bit hashjoins

From
Bruce Momjian
Date:
> My guess is that either a struct field is being declared "long" when
> it really oughta be "int", or some part of the tuple storage routines
> is applying LONGALIGN() when it only oughta apply INTALIGN().  This
> is something that would be difficult to track down or verify without
> a box on which sizeof(int) != sizeof(long), so I haven't gone after it.
> If you have time, please leave memutils.h with the more reasonable
> looking definition of LONGALIGN() and go looking to find out which
> system table has the sizing conflict.

Yes.  If you can tell us the column, by running initdb in debug mode
(somehow), I think we can figure out the problem.

--  Bruce Momjian                        |  http://www.op.net/~candle maillist@candle.pha.pa.us            |  (610)
853-3000+  If your life is a hard drive,     |  830 Blythe Avenue +  Christ can be your backup.        |  Drexel Hill,
Pennsylvania19026
 


Re: [HACKERS] 64-bit hashjoins

From
Erik Riedel
Date:
> Yes.  If you can tell us the column, by running initdb in debug mode
> (somehow), I think we can figure out the problem.
>  
I changed LONGALIGN to be the "more correct" version, and got the
following trace tidbit from initdb:

<...startup elided...>

+ mkdir /mnt/pgsql/data/base 
+ [ 0 -ne 0 ] 
+ rm -rf /mnt/pgsql/data/base/template1 
+ mkdir /mnt/pgsql/data/base/template1 
+ [ 0 -eq 1 ] 
BACKEND_TALK_ARG=-Q
BACKENDARGS=-boot -C -F -D/mnt/pgsql/data -Q
+ echo Creating template database in /mnt/pgsql/data/base/template1 
Creating template database in /mnt/pgsql/data/base/template1
+ [ 0 -ne 0 ] 
+ postgres -boot -C -F -D/mnt/pgsql/data -Q template1 
+ cat /usr/pdl/lib/pgsql/lib/local1_template1.bki.source 
+ sed -e s/postgres PGUID/er1p 5555/ -e s/PGUID/5555/ 
ERROR:  create index: type for attribute 'attrelid' undefined
ERROR:  create index: type for attribute 'attrelid' undefined
/usr/pdl/lib/pgsql/bin/initdb: 2300 Quit - core dumped
+ [ 131 -ne 0 ] 
+ echo initdb: could not create template database 
initdb: could not create template database
+ [ 0 -eq 0 ] 
+ echo initdb: cleaning up by wiping out /mnt/pgsql/data/base/template1 
initdb: cleaning up by wiping out /mnt/pgsql/data/base/template1
+ rm -rf /mnt/pgsql/data/base/template1 
+ exit 1 

the crash is near the ERROR statements, core dumped.  The dbx
backtrace then shows:

signal Quit at >*[__kill, 0x120185aa8]     beq    r19, 0x120185ac0
(dbx)
(dbx) where
>  0 __kill(0x0, 0x140035d50, 0x12016e508, 0xffffffffffffffff,
0x1400545b8) [0x120185aa8]  1 elog(fmt = 0x1400067b0 = "create index: type for attribute '%s'
undefined") ["elog.c":224, 0x12016e530]  2 NormIndexAttrs(attList = 0x8fc, attNumP = 0x1401e91e2, classOidP =
0x1401e91f8) ["indexcmds.c":500, 0x12007ae5c]  3 DefineIndex(heapRelationName = 0x1401991b0 = "pg_attribute",
indexRelationName = 0x1401c6e00 = "pg_attribute_relid_attnam_index",
accessMethodName = 0x1401e91f8 = "^B", attributeList = 0x1400a48d0,
parameterList = (nil), primary = '^@', predicate = (nil), rangetable =
(nil)) ["indexcmds.c":198, 0x12007a2b4]  4 Int_yyparse()
["/usr0/kosak/tmp/bisontestinstall/share/bison.simple":700, 0x120062c14]  5 BootstrapMain(argv = 0x14007da20)
["bootstrap.c":430,0x1200654c4]  6 main(argv = 0x11ffff5c8) ["main.c":100, 0x1200ae190]
 
(dbx) 

it seems to be ok until it goes to build the indices.

Does this help identify where to look?

Erik





Re: [HACKERS] 64-bit hashjoins

From
Tom Lane
Date:
Erik Riedel <riedel+@CMU.EDU> writes:
>    1 elog(fmt = 0x1400067b0 = "create index: type for attribute '%s'
> undefined") ["elog.c":224, 0x12016e530]
>    2 NormIndexAttrs(attList = 0x8fc, attNumP = 0x1401e91e2, classOidP =
> 0x1401e91f8) ["indexcmds.c":500, 0x12007ae5c]

> Does this help identify where to look?

Just from looking at that chunk of the source, it seems that the problem
is in either pg_type or pg_attribute, since it is using a field from a
pg_attribute tuple to look for a pg_type tuple... probably
pg_attribute... which starts out with   Oid         attrelid;   NameData    attname;   Oid         atttypid;
Could it be NameData?  A few minutes later: YUP!

>From pg_type, type "name" is declared as having fixed length 32 and
int-alignment (typalign = 'i').  This is dubious enough, since a
compiler is likely to treat a char array as having only byte alignment;
typalign = 'c' seems more correct.  But it's been working so far and
prolly wouldn't break on an Alpha.

But in src/include/access/tupmacs.h we find the code that actually
implements attribute alignment calculations, and it reads:

#define att_align(cur_offset, attlen, attalign) \
( \   ((attlen) < sizeof(int32)) ? \   ( \       ((attlen) == -1) ? \       ( \           ((attalign) == 'd') ?
DOUBLEALIGN(cur_offset): \                                   INTALIGN(cur_offset) \       ) \       : \       ( \
   ((attlen) == sizeof(char)) ? \           ( \               (long)(cur_offset) \           ) \           : \
( \               AssertMacro((attlen) == sizeof(short)), \               SHORTALIGN(cur_offset) \           ) \
)\   ) \   : \   ( \       ((attlen) == sizeof(int32)) ? \       ( \           INTALIGN(cur_offset) \       ) \       :
\      ( \           AssertMacro((attlen) > sizeof(int32)), \           ((attalign) == 'd') ?   DOUBLEALIGN(cur_offset)
:\                                   LONGALIGN(cur_offset) \       ) \   ) \
 
)

Walk through that with attlen = 32, attalign = 'i', and guess what:
it applies LONGALIGN().  Which is different on Alpha than everywhere
else, not to mention flat-out wrong for the given arguments.

Erik, try changing that last LONGALIGN to INTALIGN and see if it
works any better on the Alpha.

We probably really ought to bag this entire logic and replace it
with something like    attalign 'c' -> no alignment    attalign 's' -> SHORTALIGN    attalign 'i' -> INTALIGN
attalign'd' -> DOUBLEALIGN
 
with possibly some AssertMacro cross-checks that the given attlen
makes sense for the attalign.  But driving the logic primarily off
attlen rather than attalign makes little sense to me.
        regards, tom lane


Re: [HACKERS] 64-bit hashjoins

From
Tom Lane
Date:
I have committed changes that I believe eliminate the need for the
bogus LONGALIGN() definition Erik was complaining of.

The modified sources pass regression (as well as before anyway ;-))
on my machine, but I have not got an Alpha to test with.  Erik,
would you pull the current CVS sources and see if it works for you?
NOTE: please run a full install including initdb, in order to make sure
I didn't break bootstrap...
        regards, tom lane


Re: [HACKERS] 64-bit hashjoins

From
Bruce Momjian
Date:
> Walk through that with attlen = 32, attalign = 'i', and guess what:
> it applies LONGALIGN().  Which is different on Alpha than everywhere
> else, not to mention flat-out wrong for the given arguments.
> 
> Erik, try changing that last LONGALIGN to INTALIGN and see if it
> works any better on the Alpha.
> 
> We probably really ought to bag this entire logic and replace it
> with something like
>         attalign 'c' -> no alignment
>         attalign 's' -> SHORTALIGN
>         attalign 'i' -> INTALIGN
>         attalign 'd' -> DOUBLEALIGN
> with possibly some AssertMacro cross-checks that the given attlen
> makes sense for the attalign.  But driving the logic primarily off
> attlen rather than attalign makes little sense to me.

I can agree that the type alignment is clearly wrong.  I did some
cleanup of the pg_* files to they at least match, but that was obviously
not the whole picture, and this flakeyness has given us problems on
alpha.


--  Bruce Momjian                        |  http://www.op.net/~candle maillist@candle.pha.pa.us            |  (610)
853-3000+  If your life is a hard drive,     |  830 Blythe Avenue +  Christ can be your backup.        |  Drexel Hill,
Pennsylvania19026
 


Re: [HACKERS] 64-bit hashjoins

From
Erik Riedel
Date:
> The modified sources pass regression (as well as before anyway ;-))
> on my machine, but I have not got an Alpha to test with.  Erik,
> would you pull the current CVS sources and see if it works for you?
> NOTE: please run a full install including initdb, in order to make sure
> I didn't break bootstrap...
>  
Done.  Looks like it works.

It compiles fine, initdb runs fine, and my query runs correctly with
HashJoin (more than 100x improvement over the Nested Loops version!).

Thanks!

Erik



Re: [HACKERS] 64-bit hashjoins

From
Tom Lane
Date:
Now that the XXXALIGN() macros are supposed to reflect reality instead
of arbitrary decisions ;-), I have revised memutils.h to eliminate all
that "#if defined(platform)" cruft.  Instead, the actual alignment values
being used by the compiler are discovered by the configure script.
This should make things considerably more robust on machines where the
alignment requirement of the basic C datatypes is not the same as their
size.  In particular, we should no longer see any problems with the
struct declarations in include/catalog/pg_*.h not matching the way that
the tuple access code lays out the tuples.

WARNING: if you are on a machine where this actually makes a difference,
you may have to do an initdb after your next CVS update, because the
padding in your tables may change.  I think this would be most likely
to affect tables containing float8 or int8 data --- some machines
require 8-byte alignment of doubles, but some don't, and the padding of
float data will now reflect that.

Right now the system is still making an assumption that I consider
crufty: it uses typalign = 'd' (ie, DOUBLE alignment) for int8 data
(long long int).  As things stand, this would only cause problems on
machines where long long actually has stronger alignment requirements
than double.  I've never heard of such a platform, but maybe they are
out there --- has anyone heard of one?  A more likely cause of trouble
is that if any int8 columns are ever added to system tables, the code
will risk failure unless int8 and double have exactly the same alignment
requirement (because the catalog structs could get laid out differently
than the tuple code would expect).

Is it worth adding a new typalign value specifically for int8, in order
to make the world safe for int8 columns in system tables?
        regards, tom lane


Re: [HACKERS] 64-bit hashjoins

From
Thomas Lockhart
Date:
> Right now the system is still making an assumption that I consider
> crufty: it uses typalign = 'd' (ie, DOUBLE alignment) for int8 data
> (long long int).  As things stand, this would only cause problems on
> machines where long long actually has stronger alignment requirements
> than double.  I've never heard of such a platform, but maybe they are
> out there --- has anyone heard of one?

No. At least not in our list of supported platforms; don't know what
Cray mainframes require, since machines like that are not necessarily
8-bit-bytes, 4-byte-longword machines.

> Is it worth adding a new typalign value specifically for int8, in 
> order to make the world safe for int8 columns in system tables?

I would be comfortable making the same assumptions about int8 as for
double.
                       - Thomas


Re: [HACKERS] 64-bit hashjoins

From
Bruce Momjian
Date:
Tom, you fixed this, right?


> Erik Riedel <riedel+@CMU.EDU> writes:
> > Platform:  Alpha, Digital UNIX 4.0D 
> > [ memutils.h says ]
> >  /*
> >   * even though "long alignment" should really be on 8-byte boundaries for
> >   * linuxalpha, we want the strictest alignment to be on 4-byte (int)
> >   * boundaries, because otherwise things break when they try to use the
> >   * FormData_pg_* structures.  --djm 12/12/96
> >   */
> 
> I remember looking at that code and saying "Huh?  You can't do that!".
> I kept my fingers off it because I didn't have direct proof that it
> was broken ... but it sounds like you do.
> 
> > Can someone explain the comment from djm to me (or is djm still
> > listening somewhere?).  At first blush, I suspect that I actually
> > _want_ it to do the latter version of LONGALIGN(), since my longs
> > really are 8 bytes.  But when I try to do that instead, I am unable to
> > even run "initdb" - dies with an error like "attribute not
> > found/invalid"
> 
> Yeah, that's about what I'd expect.  The point is that the struct
> layouts found in include/catalog/pg_*.h for system table records
> have to match the actual physical layout of tuples on disk.  What
> you are probably running into is that the attribute size/alignment
> calculations done by the heaptuple code using the declared column data
> types fail to match up with the struct field alignment done by the
> compiler.
> 
> My guess is that either a struct field is being declared "long" when
> it really oughta be "int", or some part of the tuple storage routines
> is applying LONGALIGN() when it only oughta apply INTALIGN().  This
> is something that would be difficult to track down or verify without
> a box on which sizeof(int) != sizeof(long), so I haven't gone after it.
> If you have time, please leave memutils.h with the more reasonable
> looking definition of LONGALIGN() and go looking to find out which
> system table has the sizing conflict.
> 
> BTW, we'd run into this same problem if any of the system tables had
> a float8 column, since the alignment of those is platform-dependent.
> Memo to hackers: stay away from float8 in sys tables.
> 
>             regards, tom lane
> 
> 


--  Bruce Momjian                        |  http://www.op.net/~candle maillist@candle.pha.pa.us            |  (610)
853-3000+  If your life is a hard drive,     |  830 Blythe Avenue +  Christ can be your backup.        |  Drexel Hill,
Pennsylvania19026
 


Re: [HACKERS] 64-bit hashjoins

From
Tom Lane
Date:
Bruce Momjian <maillist@candle.pha.pa.us> writes:
> Tom, you fixed this, right?

I believe this is fixed, but it'd be nice to have some confirmation from
someone with a platform where long != int ...  Erik, have you tried it
recently?
        regards, tom lane


>> Erik Riedel <riedel+@CMU.EDU> writes:
>>>> Platform:  Alpha, Digital UNIX 4.0D 
>>>> [ memutils.h says ]
>>>> /*
>>>> * even though "long alignment" should really be on 8-byte boundaries for
>>>> * linuxalpha, we want the strictest alignment to be on 4-byte (int)
>>>> * boundaries, because otherwise things break when they try to use the
>>>> * FormData_pg_* structures.  --djm 12/12/96
>>>> */
>> 
>> I remember looking at that code and saying "Huh?  You can't do that!".
>> I kept my fingers off it because I didn't have direct proof that it
>> was broken ... but it sounds like you do.
>> 
>>>> Can someone explain the comment from djm to me (or is djm still
>>>> listening somewhere?).  At first blush, I suspect that I actually
>>>> _want_ it to do the latter version of LONGALIGN(), since my longs
>>>> really are 8 bytes.  But when I try to do that instead, I am unable to
>>>> even run "initdb" - dies with an error like "attribute not
>>>> found/invalid"
>> 
>> Yeah, that's about what I'd expect.  The point is that the struct
>> layouts found in include/catalog/pg_*.h for system table records
>> have to match the actual physical layout of tuples on disk.  What
>> you are probably running into is that the attribute size/alignment
>> calculations done by the heaptuple code using the declared column data
>> types fail to match up with the struct field alignment done by the
>> compiler.
>> 
>> My guess is that either a struct field is being declared "long" when
>> it really oughta be "int", or some part of the tuple storage routines
>> is applying LONGALIGN() when it only oughta apply INTALIGN().  This
>> is something that would be difficult to track down or verify without
>> a box on which sizeof(int) != sizeof(long), so I haven't gone after it.
>> If you have time, please leave memutils.h with the more reasonable
>> looking definition of LONGALIGN() and go looking to find out which
>> system table has the sizing conflict.
>> 
>> BTW, we'd run into this same problem if any of the system tables had
>> a float8 column, since the alignment of those is platform-dependent.
>> Memo to hackers: stay away from float8 in sys tables.
>> 
>> regards, tom lane
>> 
>> 


> -- 
>   Bruce Momjian                        |  http://www.op.net/~candle
>   maillist@candle.pha.pa.us            |  (610) 853-3000
>   +  If your life is a hard drive,     |  830 Blythe Avenue
>   +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026


Re: [HACKERS] 64-bit hashjoins

From
Erik Riedel
Date:
Excerpts from mail: 10-May-99 Re: [HACKERS] 64-bit hashjo.. by Tom
Lane@sss.pgh.pa.us 
> I believe this is fixed, but it'd be nice to have some confirmation from
> someone with a platform where long != int ...  Erik, have you tried it
> recently?
>  
Sorry for the slow response.

I tried this when the fix was first done, and I thought I reported to
the list that it worked fine.

I actually have not updated my tree since then, so I don't know about
changes after 25 March.  My logs say:

630     snapshot from postgresql CVS (25 March 1999)
631     fix for 64-bit LONGALIGN (works!)

I have been using that version since March without problems (well, at
least no problems with 64-bit ints and hashjoins...).

Erik



Re: [HACKERS] 64-bit hashjoins

From
Tom Lane
Date:
Erik Riedel <riedel+@CMU.EDU> writes:
> I actually have not updated my tree since then, so I don't know about
> changes after 25 March.  My logs say:
> 630     snapshot from postgresql CVS (25 March 1999)
> 631     fix for 64-bit LONGALIGN (works!)
> I have been using that version since March without problems (well, at
> least no problems with 64-bit ints and hashjoins...).

OK, but I've done some considerable hacking on the hashjoin code since
then.  I don't *think* I broke anything ... but ... if you have the
time to pull current sources and check again, it'd be appreciated.
(There have been a lot of other bugs fixed since March, too.)
        regards, tom lane