Thread: Array elements (urgent help needed)
I am desparate to find a way to access array elements from my c/c++ program. I have stored int and double data as arrays in my tables and need to retrieve them from the database. I use a binary cursor to store the query results and PQgetvalue gives me a char* pointer to the storage area where the data is stored (this is according to the documentation). I have tried to understand the internal structure of the storage area, but it changes with data type. I suspect there is a more reliable/elegant way to get access to the elements. What is the correct way of doing this? Thanks in advance. Akbar Mokhtarani
I really haven't work with arrays that much at the C
level, but I thought that the result from PQgetvalue
was just a char string. If it is an array, I thought
that the string would look like: {elem1, elem2,
elem3, ...}.
I don't think there are any helper functions for this,
so you need to manually walk that char string to peel
out the elements...at least as far as I am aware. You
might want to look at the how the operators for arrays
access the elements, e.g. the <> operator for arrays.
Again, I'm not expert, but this is what I had thought
is the case. I hope I understood your question.
HTH,
--brett
--- Akbar Mokhtarani <akbarm@slac.stanford.edu> wrote:
> I am desparate to find a way to access array
> elements from my c/c++
> program. I have stored int and double data as arrays
> in my tables and need
> to retrieve them from the database. I use a binary
> cursor to store the
> query results and PQgetvalue gives me a char*
> pointer to the storage area
> where the data is stored (this is according to the
> documentation). I have
> tried to understand the internal structure of the
> storage area, but it
> changes with data type. I suspect there is a more
> reliable/elegant way to
> get access to the elements. What is the correct way
> of doing this?
>
> Thanks in advance.
>
> Akbar Mokhtarani
>
>
> ---------------------------(end of
> broadcast)---------------------------
> TIP 4: Don't 'kill -9' the postmaster
>
Yes the result from PQgetvalue is a char*. I looked at the array.h and
used the ArrayType structure defined there to walk through the memory
area. The first 4 bytes gives you the dimension of the array, the second 8
bytes are not used and the third 6 bytes gives the number of elements, and the
rest are 4 bytes pointers to the data type stored in the array.
Using this I was
able to parse arrays of int, double, and char BUT not string. This worked
fine on a cluster of intel based machines. However, when I moved the code
to another intel based machine, the program crashed. I have not been able
to parse the memory on this machine. It is an AMD processor which should
be the same as intel processor.
This made me believe that there must be a better way of doing this without
knowing the intenal structure of the memory.
Akbar Mokhtarani
On Sun, 3 Oct 2004, Brett Schwarz wrote:
> I really haven't work with arrays that much at the C
> level, but I thought that the result from PQgetvalue
> was just a char string. If it is an array, I thought
> that the string would look like: {elem1, elem2,
> elem3, ...}.
>
> I don't think there are any helper functions for this,
> so you need to manually walk that char string to peel
> out the elements...at least as far as I am aware. You
> might want to look at the how the operators for arrays
> access the elements, e.g. the <> operator for arrays.
>
> Again, I'm not expert, but this is what I had thought
> is the case. I hope I understood your question.
>
> HTH,
>
> --brett
>
> --- Akbar Mokhtarani <akbarm@slac.stanford.edu> wrote:
>
> > I am desparate to find a way to access array
> > elements from my c/c++
> > program. I have stored int and double data as arrays
> > in my tables and need
> > to retrieve them from the database. I use a binary
> > cursor to store the
> > query results and PQgetvalue gives me a char*
> > pointer to the storage area
> > where the data is stored (this is according to the
> > documentation). I have
> > tried to understand the internal structure of the
> > storage area, but it
> > changes with data type. I suspect there is a more
> > reliable/elegant way to
> > get access to the elements. What is the correct way
> > of doing this?
> >
> > Thanks in advance.
> >
> > Akbar Mokhtarani
> >
> >
> > ---------------------------(end of
> > broadcast)---------------------------
> > TIP 4: Don't 'kill -9' the postmaster
> >
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 6: Have you searched our list archives?
>
> http://archives.postgresql.org
>
Akbar Mokhtarani <akbarm@slac.stanford.edu> writes:
> I am desparate to find a way to access array elements from my c/c++
> program. I have stored int and double data as arrays in my tables and need
> to retrieve them from the database. I use a binary cursor to store the
> query results and PQgetvalue gives me a char* pointer to the storage area
> where the data is stored (this is according to the documentation).
What PG version are you using?
In 7.3 and earlier you have to deal with the server's internal
representation, warts and all. (I think what you're getting burnt by is
failing to account for MAXALIGN alignment, but it's hard to be sure
without more detail.)
7.4 and up have a documented, more or less platform-independent
representation, but it's not necessarily any easier to work with since
you do need to be able to convert to whatever internal representation
you want to use.
In any case, you need to read the server-side source code to see what
you're dealing with. None of these details have made it into the SGML
documentation, but they're covered fairly well in the source comments.
regards, tom lane
Hi Tom, I am using PG 7.4.5. I searched the source code and found few pointers. I was able to solve my problem, sort of. I am including my observation and a test program for experts to review and correct. Hope people find it useful. Thanks, Akbar Mokhtarani On Mon, 4 Oct 2004, Tom Lane wrote: > > What PG version are you using? > > In 7.3 and earlier you have to deal with the server's internal > representation, warts and all. (I think what you're getting burnt by is > failing to account for MAXALIGN alignment, but it's hard to be sure > without more detail.) > > 7.4 and up have a documented, more or less platform-independent > representation, but it's not necessarily any easier to work with since > you do need to be able to convert to whatever internal representation > you want to use. > > In any case, you need to read the server-side source code to see what > you're dealing with. None of these details have made it into the SGML > documentation, but they're covered fairly well in the source comments. > > regards, tom lane >
Akbar Mokhtarani <akbarm@slac.stanford.edu> writes:
* A standard varlena array has the following internal structure:
This didn't quite agree with what I found but it gave me a starting point.
That's because it's describing the *internal* structure. Since you're
on 7.4, what you are actually seeing is the on-the-wire representation
emitted by array_send(), which looks like:
/* Send the array header information */pq_sendint(&buf, ndim, 4);pq_sendint(&buf, v->flags, 4);pq_sendint(&buf,
element_type,sizeof(Oid));for (i = 0; i < ndim; i++){ pq_sendint(&buf, ARR_DIMS(v)[i], 4); pq_sendint(&buf,
ARR_LBOUND(v)[i],4);}
/* Send the array elements using the element's own sendproc *//* (so N element values in whatever the datatype format
is)*/
regards, tom lane