Re: libpq - lack of support to set the fetch size - Mailing list pgsql-general

From matshyeq
Subject Re: libpq - lack of support to set the fetch size
Date
Msg-id CAONr5=um+dGKc=jzmpdi2rZG=+0J4E8MDLawDumU4ERUh-Ucdw@mail.gmail.com
Whole thread Raw
In response to Re: libpq - lack of support to set the fetch size  (matshyeq <matshyeq@gmail.com>)
Responses Re: libpq - lack of support to set the fetch size  (Adrian Klaver <adrian.klaver@aklaver.com>)
List pgsql-general
> Well, I don't know what "streamed" exactly means here.
> If server pushes sequentially all the data not asking client if ready to
> receive then that's what the issue is about.

This problem is handled in kernel's TCP stack - it will slow down
the connection if userspace does not read fast enough.

IOW, don't worry about it.

Again, this is something 'at discretion' of the pqlib library therefore not sure how 'slowing down' really works in different scenarios, like ie. server and app client on the same machine?
Still see quite a value in specifying this explicitly (like JDBC does), which BTW I did by using CURSOR as a workaround.

> If client asks server for another chunk each time it has received previous
> one then to me it's implicit 'fetching' scenario where user/developer
> doesn't have an option to define fetch size.

This is how it is usually implemented and configurable fetch size
is indeed useful in such situation.  But it requires separate round-trip
for each chunk so single-row-mode is superior method for processing
large queries without huge buffers.

I wouldn't worry about this (round trips) myself - at the end of the day client decides when and how much to pull.
Also, in the datawarehouse area, we consciously tune this parameter for each DB source individually which is considered very (and good!) common practice.

> >  > - is it/would it be possible to add corresponding option to pgAdmin to
> >
>  > limit initially (and each subsequently) returned rows in Query Tool by
> >
>  > custom defined max value?
> >
>
> > It could close connection in the middle of resultset but that seems like
> > bad idea.  LIMIT N or FETCH N are better for such task.
>
> I don't see why? I can't think of any single SQL tool I've been working
> with that didn't have this functionality, really.

Yeah, I see no reason pgAdmin cannot implement it.
Implementing
such feature with transparently adding LIMIT or FETCH is better.
 
Don't fully agree with last statement. FETCH is WAY better and more functional than LIMIT.

PQsetSingleRowMode() is not meant for partial resultsets, but it can
be used if you accept the downsides.
Given the 'discretion mode' I'm still not sure if that's a proper and elegant way to meet the requirement.
Maybe it is? Anyway, This is just me, an end user giving you, the PostgreSQL maintaining developers a chance to hear 'the voice of the customer'

Regards,
~Msciwoj

pgsql-general by date:

Previous
From: matshyeq
Date:
Subject: Re: libpq - lack of support to set the fetch size
Next
From: Ian Lawrence Barwick
Date:
Subject: Re: Materialized view from PG to Oracle?