DBI/DBD::Pg mem. use goes exponential - Mailing list pgsql-general

From Bob Parkinson
Subject DBI/DBD::Pg mem. use goes exponential
Date
Msg-id s33bee80.047@ccw0m1.nottingham.ac.uk
Whole thread Raw
Responses Re: DBI/DBD::Pg mem. use goes exponential
Re: DBI/DBD::Pg mem. use goes exponential
Re: DBI/DBD::Pg mem. use goes exponential
List pgsql-general
Hi,

I'd really like to aviod another list (ie. interfaces) if at all possible, so...

Got a script (trimmed version below) that starts to have exponential memory use after a number of iterations around the
while((....$sth->fetch())){ } struct. 

Table is quite small (520 odd rows) and the row size is v. small.

Mem. use starts off at 64M (using top SIZE), after 20 iterations it's bloated to 74M then it goes on each fetch to 82,
99,129, 193, 257, 321M, ...kerbang... err. msg below. 

Out of memory during "large" request for 134221824 bytes, total sbrk() is 419323904 bytes at export.pl line 205

I know this is the script trying to (m|p|?)alloc more mem. for the process and failing.

Upgraded PG to 8.0.3, DBI to 1.48 and DBD::Pg to 1.43 this morning, but made no difference. FreeBSD 5.4 (they'll get it
rightRSN :- )), perl 5.8.2 

Any ideas please? This seems well weird to me, but...

Cheers,

Bob


##!/usr/local/bin/perl

### SNIP set up %gwc ###

use DBI;

$dbh=DBI->connect('dbi:Pg:dbname=......................................') || die;

$stmt=qq(create temp table tt as select key,acronym,start_date,end_date,duration,charging_policy from template_event);

$dbh->prepare($stmt)->execute () || die;

foreach my $key (keys %gwc) {

    my $gatewaycode=$gwc{$key};

    # TEMPLATE_EVENT

    if($gatewaycode eq 'B' || gatewaycode eq 'Q'){

    my $stmt=qq(select acronym from tt where key=$key order by start_date asc);

    my $sth=$dbh->prepare($stmt) || die;

    my $rv=$sth->execute() || die;

    while((my $rref=$sth->fetch())){

        my ($acro,$start_date,$end_date,$duration,$charging)=@$rref;

                    sleep(4);

                   ### eyeball top here and watch it grow ###


    }

    }

}

Out of memory during "large" request for 134221824 bytes, total sbrk() is 419323904 bytes at export.pl line 205




This message has been checked for viruses but the contents of an attachment
may still contain software viruses, which could damage your computer system:
you are advised to perform your own checks. Email communications with the
University of Nottingham may be monitored as permitted by UK legislation.



This message has been checked for viruses but the contents of an attachment
may still contain software viruses, which could damage your computer system:
you are advised to perform your own checks. Email communications with the
University of Nottingham may be monitored as permitted by UK legislation.


pgsql-general by date:

Previous
From: Douglas McNaught
Date:
Subject: Re: Setting up a fine-grained permission system
Next
From: ellis@no.spam ()
Date:
Subject: Perl regular expressions