Loading data into Postgres - Mailing list pgsql-general

From Przemyslaw Bak
Subject Loading data into Postgres
Date
Msg-id Pine.SV4.3.96.980531164247.23497A-100000@STORE3
Whole thread Raw
List pgsql-general
Hi,

every night my script (using perl) will load about 50.000 rows of
data into Postgres. I know two ways:

In first version of my script I used method by inserting, eg:
---------- cut ----------
#!/bin/perl

use Pg;

open (MyData, "...");
while (<MyData>) {
  chop;
  ...
  &insert_cust_sales ($NDate, $CustNo, $ArtGrp, $Qty, $Amount);
}
close MyData;

sub insert_cust_sales {
  my $SQL_QUERY = "insert into cust_sales values ('$_[0]',$_[1],$_[2],$_[3],$_[4],$_[5]);";
  my $conn = Pg::connectdb ("dbname = store3");
  my $result = $conn->exec ( $SQL_QUERY );
}
---------- cut ----------
but it is very slow and my disk works very havy.
So I created another solution, look below:
---------- cut ----------
#!/bin/perl

# Open connection to Postgres
open (PSQL, "| psql store3") || die "Can not open connection to Postgres\n";
print PSQL " copy cust_sales from stdin using delimiters '|';";

# Read data from file
open (MyData, "...");
while (<MyData>) {
  s/    //g; # remove \t
  s/\s+//g;    # ... and spaces
  print PSQL,$_,"\n";
}
print PSQL "\\.\n";
close PSQL;
close MyData;
--------- cut ----------
but while running this script I receive message "resetting connection".
Why ? What's wrong with this script ? I have seen in sources
of postgres that this message is when copy failed, but why ?

przemol


pgsql-general by date:

Previous
From: Marin D
Date:
Subject: no shared libs
Next
From: "Thomas G. Lockhart"
Date:
Subject: Re: [INTERFACES] no shared libs