performance issues with DBI module when data too big - Mailing list pgsql-general

From Nicolas Nolst
Subject performance issues with DBI module when data too big
Date
Msg-id F141dJRh2GcClDcNMAP00011eb3@hotmail.com
Whole thread Raw
Responses Re: performance issues with DBI module when data too big  (Jan Wieck <janwieck@yahoo.com>)
Re: [ADMIN] performance issues with DBI module when data too big  (Andrew Perrin <clists@perrin.socsci.unc.edu>)
List pgsql-general
<div style="background-color:"><div><div><div></div><div></div><p>Hi all,<p>I have developped a perl script to populate
adatabase with two tables: sessions and actions.<p>the table actions contains the following columns: session_id, url,
timestamp.The column session_id references to the table sessions. <p>the table sessions contains the following columns:
session_id,remote_ip, phone_type, phone_number. The column session_id is serial.<p>The lines of the table actions which
arepart of the same session have the same session_id.<p>There are then more lines in the table actions than in the
tablesession.<p><br />To fill the two tables, I first need to know if the session already exists for a certain
phone_type,a certain remote_ip and a certain phone_number:<p>SELECT session_id FROM sessions WHERE (phone_number =
?) AND(remote_ip  = ?) AND (phone_type = ?) ORDER BY session_id;<p>I also need to apply a criteria to know if I have
to adda new entry in the table sessions or not:<p>SELECT (max(timestamp) + ?)<? FROM actions WHERE (session_id = ?);
 <p> <p>If the session already exists I add a line in the table actions with a INSERT<p>If the session doesn't exist or
ifthe criteria is true, I add a line in the table sessions with an INSERT, retrieve the session_id of the line just
addedwith the following request<p>SELECT session_id FROM sessions where (msisdn=?) AND (remote_ip=?) AND
(user_agent=?) ORDER BYsession_id DESC LIMIT 1<p>and the add with a INSERT a line in the table actions.<p>I have put
indexeson sessions(session_id), sessions(msisdn), actions(session_id).<p>My problem is that populating my database is
slower when the data gets bigger and the performance falls dramatically. I thought that is would be improve with my
indexesbut the problem still persists.<p>Could you please give me some clues that could solve this
issue.<p>Thanks.<p><br/> <br /><br />Nicolas Nolst <div></div><div></div><img
src="http://www.3dflags/World/Gif/belgium_gs.gif"/><div></div></div></div></div><br clear="all" /><hr />Join the
world�slargest e-mail service with MSN Hotmail. <a href="http://g.msn.com/1HM300901/158">Click Here</a><br /> 

pgsql-general by date:

Previous
From: Jan Wieck
Date:
Subject: Re: What popular, large commercial websites run
Next
From: Curt Sampson
Date:
Subject: View vs. Statement Query Plan