Re: INSERT performance deteriorates quickly during a large import - Mailing list pgsql-general

From Tom Lane
Subject Re: INSERT performance deteriorates quickly during a large import
Date
Msg-id 28007.1194544919@sss.pgh.pa.us
Whole thread Raw
In response to INSERT performance deteriorates quickly during a large import  ("Krasimir Hristozov \(InterMedia Ltd\)" <krasi@imedia-dev.com>)
Responses Re: INSERT performance deteriorates quickly during a large import  (Julio Cesar Sánchez González<jsg@sistemasyconectividad.com.mx>)
List pgsql-general
"Krasimir Hristozov \(InterMedia Ltd\)" <krasi@imedia-dev.com> writes:
> We need to import data from a relatively large MySQL database into an
> existing PostgreSQL database, using a PHP5 script that SELECTs from MySQL
> and INSERTs in PostgreSQL. A part of the import involves moving about
> 1,300,000 records from one MySQL table to one of our PostgreSQL tables. The
> problem is that the insert performance inevitably deteriorates as the number
> of inserts increases.

Are you *certain* you've gotten rid of all the indexes and foreign keys?
A simple insert ought to be pretty much constant-time in Postgres, so it
seems to me that you've missed something.

It also seems possible that you are wrong to disregard PHP as a possible
source of the problem.  Have you tried watching the PHP and PG backend
processes with "top" (or similar tool) to see who's consuming CPU time
and/or memory space?

            regards, tom lane

pgsql-general by date:

Previous
From: Tom Lane
Date:
Subject: Re: what is the date format in binary query results
Next
From: Bill Moran
Date:
Subject: Re: INSERT performance deteriorates quickly during a large import