RE: Slowdown problem when writing 1.7million records - Mailing list pgsql-general

From Stephen Livesey
Subject RE: Slowdown problem when writing 1.7million records
Date
Msg-id GDENKIKOJLMLLEIOOCCICELLCDAA.ste@exact3ex.co.uk
Whole thread Raw
In response to Re: Slowdown problem when writing 1.7million records  (Tom Lane <tgl@sss.pgh.pa.us>)
List pgsql-general

> No, it's not.  Do you have any triggers or rules on this table that
> you haven't shown us?  How about other tables referencing this one
> as foreign keys?  (Probably not, if you're running an identical test
> on MySQL, but I just want to be sure that I'm not missing something.)

I have no triggers or rules.  I have only created 1 table and their are no
foreign keys.

>
> How exactly are you writing the records?

First I read the data from a 'Powerflex' file and hold this in a record set.
     pfxstmt = pfxconn.createStatement();
     pfxrs = pfxstmt.executeQuery("SELECT * from expafh");

I then perform a loop which writes the data to my 'Postgresql' file as
follows:
      stmt = conn.createStatement();
    while (pfxrs.next()) {
        cmd = "INSERT INTO expafh VALUES ";
        cmd = cmd +
"('"+pfxrs.getString(2)+"',"+pfxrs.getString(3)+",'"+pfxrs.getString(4)+"','
"+pfxrs.getString(5)+"')";
        stmt.executeUpdate(cmd);
        }

>
> I have a suspicion that the slowdown must be on the client side (perhaps
> some inefficiency in the JDBC code?) but that's only a guess at this
> point.
>

I have used identical code for all of my testing, the only changes being
which drivers I use to access the data.


Thanks
Stephen Livesey

Legal Disclaimer:
Internet communications are not secure and therefore Exact Abacus does
not accept legal responsibility for the contents of this message.  Any views
or opinions presented are solely those of the author and do not necessarily
represent those of Exact Abacus unless otherwise specifically stated.

pgsql-general by date:

Previous
From: "Gordon A. Runkle"
Date:
Subject: Re: Migrate from MS SQL 6.5 to postgres??
Next
From: Fabrice Scemama
Date:
Subject: Re: postgresql upgrades -> where's the dump go?