Thread: Updating session id based on accesstimeout
Hi,
I want to set a session id based on a session timeout per user. I have a very large logfile with more than 1.3 Mio records and look for a very efficient solution - maybe with PL/pgSQL .
The important columns in the table are: access timestamp and userid sorted by userid and access time.
The logic for the session id is: when ever the userid changes or the time between two following records is greater than 60 sec create a new session id and update the record with this id.
I tried to solve this problem with Perl and the DBI interface - but it took about 12 hours to update all records.
The fastes solution I found until now was not to use any data base, sort the data with the unix command sort and process the logfile text directly with perl.
Thanks in advance!
Andreas
On Sat, 29 Nov 2003, Andreas Heissenberger wrote: > Hi, > > I want to set a session id based on a session timeout per user. I have a very large logfile with more than 1.3 Mio recordsand look for a very efficient solution - maybe with PL/pgSQL . > > The important columns in the table are: access timestamp and userid sorted by userid and access time. > The logic for the session id is: when ever the userid changes or the time between two following records is greater than60 sec create a new session id and update the record with this id. > > I tried to solve this problem with Perl and the DBI interface - but it took about 12 hours to update all records. > The fastes solution I found until now was not to use any data base, sort the data with the unix command sort and processthe logfile text directly with perl. You won't find a method faster than updating text files, until you stop brute forcing the data. Databases don't provide speed, at least in this case. They provide ACID compliance, and other features. Perhaps, if you revisit your assumptions, you might find that your need is solved by a different notion of the problem. -j -- Jamie Lawrence jal@jal.org "God created the integers, all else is the work of man." - Kronecker
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 > I want to set a session id based on a session timeout per user. I > have a very large logfile with more than 1.3 Mio records and look > for a very efficient solution - maybe with PL/pgSQL . Can you post a clearer example with the exact columns in the table, and what you wish to do? - -- Greg Sabino Mullane greg@turnstep.com PGP Key: 0x14964AC8 200312022156 -----BEGIN PGP SIGNATURE----- iD8DBQE/zVDsvJuQZxSWSsgRAnXaAJ9ndHTI7ha9ZyWS+Bnybgbve09jVQCffWTa H/csLQmY29QTReOH7XYMtHs= =7xo3 -----END PGP SIGNATURE-----