Thread: libpq 7.4 and binary cursor

Hi,

It seems some changes occured in network protocol between 7.3 and 7.4.


In my simple libray above libpq, I used this call to extract float8 with
a binary cursor (on x86) :

-----------------------------------------------
memcpy(&float8,      PQgetvalue(pg_res, tup_num, field_num),      PQfsize(pg_res, field_num));
-----------------------------------------------

Now with PostgreSQL 7.4, I replaced this memcpy by theses lines :

-----------------------------------------------
union {double d; int64_t i64;
} swap;
uint32_t tab_uint32[2];

/* Read two uint32 */
memcpy(tab_uint32, PQgetvalue(pg_res, tup_num, field_num), 8);

/* Swap MSB -> LSB */
tab_uint32[0] = ntohl(tab_uint32[0]);
tab_uint32[1] = ntohl(tab_uint32[1]);

/* Fusion */
swap.i64 = tab_uint32[0];
swap.i64 <<= 32;
swap.i64 |= tab_uint32[1];

/* Cast */
return swap.d;
------------------------------------------------

Is it the right method to extract binary data ? Don't exist a easiest
method ?

Thank you.

Stephane

On Thu, Jun 10, 2004 at 04:30:27PM +0200, Stephane Raimbault wrote:

> union {
>     double d; 
>     int64_t i64;
> } swap;
> uint32_t tab_uint32[2];
[...]

> /* Fusion */
> swap.i64 = tab_uint32[0];
> swap.i64 <<= 32;
> swap.i64 |= tab_uint32[1];
> 
> /* Cast */
> return swap.d;

This trick may work on some compilers and/or platforms, but it's not
correct C.  The language does not guarantee that the members of a union
will be allocated in the exact same address, or even that they will
overlap.  What if it registerizes swap.d and/or swap.i64?  It may even
recognize one of the two as uninitialized, and the other as a dead value.
That will get you very compact code, but not the effect you want.

To see how it may go wrong, imagine your compiler implemented "union"
as "#define union struct".  I think this is all described in the C FAQ,
BTW.


Jeroen

"Jeroen T. Vermeulen" <jtv@xs4all.nl> writes:
> This trick may work on some compilers and/or platforms, but it's not
> correct C.  The language does not guarantee that the members of a union
> will be allocated in the exact same address, or even that they will
> overlap.

Nonsense.  C99 6.7.2.1:
      A pointer  to  a  union  object, suitably converted, points to      each of its members (or if a member is a
bit-field,then  to      the unit in which it resides), and vice versa.
 

It does say that the results of fetching a union member other than
the one last stored to are implementation-dependent, but not that
the implementation can choose to put them in different places.
        regards, tom lane

On Thu, Jun 10, 2004 at 11:39:25AM -0400, Tom Lane wrote:
> Nonsense.  C99 6.7.2.1:
> 
>        A pointer  to  a  union  object, suitably converted, points to
>        each of its members (or if a member is a bit-field, then  to
>        the unit in which it resides), and vice versa.
> 
> It does say that the results of fetching a union member other than
> the one last stored to are implementation-dependent, but not that
> the implementation can choose to put them in different places.

I stand corrected.  Perhaps this is something that changed in C99 because
too many programs used this trick, or because enough platforms supported 
it.

Note, however, that this is about pointers to unions (and ``suitably
converted,'' whatever that means; there are cases where pointer type
conversions may involve the addition or subtraction of an offset, so there
are situations where you need to be careful about how exactly you convert
your pointers).  The code we're talking about had a union object, but no
pointers.  So the part about registerization still holds.


Jeroen

Stephane Raimbault wrote:
> 
> It seems some changes occured in network protocol between 7.3 and 7.4.

Big understatement! It was completely redesigned.

> In my simple libray above libpq, I used this call to extract float8 with
> a binary cursor (on x86) :
> ...

Pre-7.4, binary data came back in the server's byte order. Starting 7.4,
data always comes back in "network data order", which is big-endian,
regardless of server or client architecture. This is a big improvement, so
you can now portably deal with binary data. The proper way to do so in the
client is to use the ntohs() and ntohl() functions to turn network data
order into native order.