Thread: Async processing of rows

Async processing of rows

From
Nat!
Date:
Hi

I will be writing an EOF (http://en.wikipedia.org/wiki/Enterprise_Objects_Framework 
) adaptor for Postgres. Due to the way these are structured, I want to  
process the result data row by row and not in one big tuple array. I  
looked into the pg-library and it seems that this is possible, albeit  
not without adding something to the API.

PQgetResult seems to loop as long as PGASYNC_BUSY is set, and that  
appears to be set as long as there are rows being sent from the  
server. Correct ?

So I what I think I need to do is write a function PQgetNextResult  
that only blocks if there is not enough data available for reading in  
one row.

A cursory glance at pqParseInput3 shows, that I can't call it with  
incomplete input, as data is discarded even if the parse is  
incomplete, mainly, this piece of code discards 'id' if msgLength can  
not be completely read, which makes me wary:
    conn->inCursor = conn->inStart;    if (pqGetc(&id, conn))        return;    if (pqGetInt(&msgLength, 4, conn))    {
      /* (nat) expected to see: pqUngetc( id, conn); */                return;    }
 

So am I missing something or is this basically correct ?

Ciao   Nat!
----------------------------------------------
I'd like to fly
But my wings have been so denied   -- Cantrell



Re: Async processing of rows

From
Tom Lane
Date:
Nat! <nat@mulle-kybernetik.com> writes:
> I will be writing an EOF (http://en.wikipedia.org/wiki/Enterprise_Objects_Framework 
> ) adaptor for Postgres. Due to the way these are structured, I want to  
> process the result data row by row and not in one big tuple array.

Use a cursor, perhaps.
        regards, tom lane


Re: Async processing of rows

From
Nat!
Date:
Am 15.09.2008 um 14:43 schrieb Tom Lane:

> Nat! <nat@mulle-kybernetik.com> writes:
>>

>> Use a cursor, perhaps.
>
>
>


Perhaps yes, perhaps no. Not saying no, but the overhead (mostly in  
SQL statement roundtrips) seems a little daunting.

Ciao   Nat!
------------------------------------------------------
People are quite at home with evil. ... It's fighting
it that raises the dreadful specter of inconvenience.
-- McEldowney