[HACKERS] libpq Alternate Row Processor - Mailing list pgsql-hackers

From Kyle Gearhart
Subject [HACKERS] libpq Alternate Row Processor
Date
Msg-id BLUPR14MB0162DA841005B027F4CA1465FA4F0@BLUPR14MB0162.namprd14.prod.outlook.com
Whole thread Raw
Responses Re: [HACKERS] libpq Alternate Row Processor  (Jim Nasby <Jim.Nasby@BlueTreble.com>)
Re: [HACKERS] libpq Alternate Row Processor  (Tom Lane <tgl@sss.pgh.pa.us>)
List pgsql-hackers
The guts of pqRowProcessor in libpq does a good bit of work to maintain the internal data structure of a PGresult.
Thereare a few use cases where the caller doesn't need the ability to access the result set row by row, column by
columnusing PQgetvalue.  Think of an ORM that is just going to copy the data from PGresult for each row into its own
structures.

I've got a working proof of concept that allows the caller to attach a callback that pqRowProcessor will call instead
ofgoing thru its own routine.  This eliminates all the copying of data from the PGconn buffer to a PGresult buffer and
thenultimately a series of PQgetvalue calls by the client.  The callback allows the caller to receive each row's data
directlyfrom the PGconn buffer. 

It would require exposing struct pgDataValue in libpq-fe.h.  The prototype for the callback pointer would be:
int (*PQrowProcessorCB)(PGresult*, const PGdataValue*, int col_count, void *user_data);

My initial testing shows a significant performance improvement.  I'd like some opinions on this before wiring up a
performanceproof and updating the documentation for a formal patch submission. 


Kyle Gearhart




pgsql-hackers by date:

Previous
From: Robert Haas
Date:
Subject: Re: [HACKERS] [PATCH] Rename pg_switch_xlog to pg_switch_wal
Next
From: Robert Haas
Date:
Subject: Re: [HACKERS] pageinspect: Hash index support