libpq - lack of support to set the fetch size - Mailing list pgsql-general

From matshyeq
Subject libpq - lack of support to set the fetch size
Date
Msg-id CAONr5=tK=omSo7B9jAMngED-9gvcemRRHe7_K7J=UoGjgCKxJw@mail.gmail.com
Whole thread Raw
Responses Re: libpq - lack of support to set the fetch size  (Adrian Klaver <adrian.klaver@aklaver.com>)
Re: libpq - lack of support to set the fetch size  ("Daniel Verite" <daniel@manitou-mail.org>)
Re: libpq - lack of support to set the fetch size  (John R Pierce <pierce@hogranch.com>)
Re: libpq - lack of support to set the fetch size  (Jeff Janes <jeff.janes@gmail.com>)
List pgsql-general
Hello,

I've found an issue when tried to implement fetching rows from big table (2mln rows) in my app.
Basically I don't find an elegant and easy way (other than always use cursors) to limit the number of rows returned.
This causes my application to break due to the excessive memory consumption.

I'm using Perl and DBD::Pg library but contacted maintainer who actually pointed out this is an issue that goes much deeper (libpq):

"Unfortunately, this is a limitation in the underlying driver (libpq) rather than DBD::Pg itself. There have been talks over the years of supporting this, but nothing concrete yet. Your best bet would be to ask about this on the Postgres lists"

Would you consider putting this on the roadmap, so one day it gets improved?

Re, the details of the issue, I believe this has been well described at:

Kind Regards
~Msciwoj

pgsql-general by date:

Previous
From: Jeff Janes
Date:
Subject: Re: log_statement per table
Next
From: Adrian Klaver
Date:
Subject: Re: libpq - lack of support to set the fetch size