Re: Problem: libpq, network traffic, memory usage - Mailing list pgsql-general

From A. Kretschmer
Subject Re: Problem: libpq, network traffic, memory usage
Date
Msg-id 20051207171356.GA30415@webserv.wug-glas.de
Whole thread Raw
In response to Problem: libpq, network traffic, memory usage  (Alexander Scholz <alexander.scholz1@freenet.de>)
Responses Re: Problem: libpq, network traffic, memory usage
List pgsql-general
am  07.12.2005, um 18:01:51 +0100 mailte Alexander Scholz folgendes:
> Dear NG,
>
> I have a serious performance flaw, when using postgresql 8.1 (other
> versions haven't been tested) with libpq. When executing a
>
>     select * from "xyz"   1)
>
> and "xyz" contains 300'000 records, it takes more than 60 seconds for the

The same question was yesterday on [pgsql-de-allgemein] ;-)


> query just to complete. First I thought it is a performance problem of the
> server, but then we noticed an enormous amount of TCP/IP packets that are
> beeing received in that (waiting)time. The memory usage of my process,
> while waiting for the query to complete, bumps up to more than 300MB, the
> CPU usage goes up to 100%! It seems as if libpq would be receiving the
> complete resultset from the server _before_ returning from the query
> execution! (Even then 300MB seems for me to be too much, but that's not the
> point!) Just to ensure: I didn't perform ANY _fetch_ so far! It's just
> about executing the query.
>
> Does anybody have any idea, what's going on here?

You have selected _ALL_ records, you got all records. Thats the point.

> When doing the query in pgAdmin III it also takes that awful long time and
> pgAdmin III memory usage also goes up to over 300 MB. We specified in
> pgAdmin III to fetch only 1 row, thus _filling_ the grid cannot be the
> reason there. The query there took 115 seconds just to complete. :-(
>
> Can anybody help me out?!? I'd appreciate even just a hint or reference to
> somewhere else. We already googled "memory and libpq" but the first 200
> hits (sigh) didn't give any useful hint in that respect...

You can use a CURSOR to walk through the result.


HTH, Andreas
--
Andreas Kretschmer    (Kontakt: siehe Header)
Heynitz:  035242/47212,      D1: 0160/7141639
GnuPG-ID 0x3FFF606C http://wwwkeys.de.pgp.net
 ===    Schollglas Unternehmensgruppe    ===

pgsql-general by date:

Previous
From: Scott Marlowe
Date:
Subject: Re: Problem: libpq, network traffic, memory usage
Next
From: "Andrew J. Kopciuch"
Date:
Subject: Re: fts, compond words?