Thread: Selecting large tables gets killed

2014-02-20 16:16 GMT+09:00 Ashutosh Bapat <ashutosh.bapat@enterprisedb.com>:
> Hi All,
> Here is a strange behaviour with master branch with head at
(...)
> Looks like a bug in psql to me. Does anybody see that behaviour?

It's not a bug, it's your VM's OS killing off a process which is using
up too much memory.

Check /var/log/messages to see what the kernel has to say about it.

Regards

Ian Barwick

--On 20. Februar 2014 14:49:28 +0530 Ashutosh Bapat 
<ashutosh.bapat@enterprisedb.com> wrote:

> If I set some positive value for this variable, psql runs smoothly with
> any size of data. But unset that variable, and it gets killed. But it's
> nowhere written explicitly that psql can run out of memory while
> collecting the result set. Either the documentation or the behaviour
> should be modified.

Maybe somewhere in the future we should consider single row mode for psql, 
see

<http://www.postgresql.org/docs/9.3/static/libpq-single-row-mode.html>

However, i think nobody has tackled this yet, afair.

-- 
Thanks
Bernd

On Thu, Feb 20, 2014 at 12:07 PM, Ashutosh Bapat
<ashutosh.bapat@enterprisedb.com> wrote:
> That seems a good idea. We will get rid of FETCH_COUNT then, wouldn't we?

No, I don't think we want to do that. FETCH_COUNT values greater than
1 are still useful to get reasonably tabulated output without hogging
too much memory. For example:

db=# \set FETCH_COUNT 3
db=# select repeat('a', i) a, 'x'x from generate_series(1,9)i; a  | x
-----+---a   | xaa  | xaaa | xaaaa   | xaaaaa  | xaaaaaa | xaaaaaaa   | xaaaaaaaa  | xaaaaaaaaa | x


Regards,
Marti

Marti Raudsepp <marti@juffo.org> writes:
> On Thu, Feb 20, 2014 at 12:07 PM, Ashutosh Bapat
> <ashutosh.bapat@enterprisedb.com> wrote:
>> That seems a good idea. We will get rid of FETCH_COUNT then, wouldn't we?

> No, I don't think we want to do that. FETCH_COUNT values greater than
> 1 are still useful to get reasonably tabulated output without hogging
> too much memory.

Yeah.  The other reason that you can't just transparently change the
behavior is error handling: people are used to seeing either all or
none of the output of a query.  In single-row mode that guarantee
fails, since some rows might get output before the server detects
an error.
        regards, tom lane

--On 20. Februar 2014 09:51:47 -0500 Tom Lane <tgl@sss.pgh.pa.us> wrote:

> Yeah.  The other reason that you can't just transparently change the
> behavior is error handling: people are used to seeing either all or
> none of the output of a query.  In single-row mode that guarantee
> fails, since some rows might get output before the server detects
> an error.

That's true. I'd never envisioned to this transparently either, exactly of 
this reason. However, i find to have single row mode somewhere has some 
attractiveness, be it only to have some code around that shows how to do it 
right. But i fear we might complicate things in psql beyond what we really 
want.

-- 
Thanks
Bernd