Re: [HACKERS] libpq Alternate Row Processor - Mailing list pgsql-hackers

From Kyle Gearhart
Subject Re: [HACKERS] libpq Alternate Row Processor
Date
Msg-id BLUPR14MB0162071912F5872BA84757DBFA420@BLUPR14MB0162.namprd14.prod.outlook.com
Whole thread Raw
In response to Re: [HACKERS] libpq Alternate Row Processor  (Kyle Gearhart <kyle.gearhart@indigohill.io>)
Responses Re: [HACKERS] libpq Alternate Row Processor  (Jim Nasby <Jim.Nasby@BlueTreble.com>)
List pgsql-hackers
> From: Tom Lane [mailto:tgl@sss.pgh.pa.us]:
>> Kyle Gearhart <kyle.gearhart@indigohill.io> writes:
>>> The guts of pqRowProcessor in libpq does a good bit of work to maintain the internal data structure of a PGresult.
Thereare a few use cases where the caller doesn't need the ability to access the result set row by row, column by
columnusing PQgetvalue.  Think of an ORM that is just going to copy the data from PGresult for each row into its own
structures.

>> It seems like you're sort of reinventing "single row mode":
https://www.postgresql.org/docs/devel/static/libpq-single-row-mode.html

>> Do we really need yet another way of breaking the unitary-query-result abstraction?


> If it's four times faster...then the option should be available in libpq.  I'm traveling tomorrow but will try to get
apatch and proof with pgbench dataset up by the middle of the week.   


Attached is a proof, test program and test results.  No documentation changes have been included at this time.

It was tested against a pgbench_accounts record set with 100,000 records.  Overall, wall clock improves 24%.  User time
elapsedis a 430% improvement.  About half the time is spent waiting on the IO with the callback.  With the regular
pqRowProcessoronly about 16% of the time is spent waiting on IO. 

The test program follows the pgbench program's command line options, with an added parameter called "m", short for
mode. Set the option to "row" for single row processing and "cb" for callback processing. 

I did not provision for the test program to accept a password from a prompt, you'll have to pass that in the arguments.

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Attachment

pgsql-hackers by date:

Previous
From: Jim Nasby
Date:
Subject: Re: [HACKERS] Press Release Draft - 2016-02-09 Cumulative Update
Next
From: Tom Lane
Date:
Subject: Re: [HACKERS] Improve OR conditions on joined columns.