Re: psql is slow and it does not take much resources - Mailing list pgsql-general

From Joe Healy
Subject Re: psql is slow and it does not take much resources
Date
Msg-id 445942BB.5050106@omc-international.com.au
Whole thread Raw
In response to Re: psql is slow and it does not take much resources  ("Javier de la Torre" <jatorre@gmail.com>)
List pgsql-general
Javier de la Torre wrote:
> Great! Then there will be no problems.
>
> I would use COPY but I think I can not. While moving from MySQL to
> PostgreSQL I am also transforming a pair of fields, latitude,
> longitude, into a geometry field, POINT, that is understood for
> Potgis. I though I will not be able to use COPY when inserting data
> with functions.
I definitely recommend using copy if you are inserting a large amount of
data into postgis.
we use something like the following python code to read from a csv file
and insert into pgsql.
I can't remember the rate it works at but it was much quicker than
anything else we tried.

def insertData( header, delimiter, filename, table, SRID, dbname, user,
host ):

    f = open(filename, 'r')

    # Open a new process to enter data (~10x faster than psycopg)
    process = os.popen('psql %s %s -c "COPY %s (geom, elevation) from
stdin;" -h %s' % (dbname, user, table, host), "w")

    for a in f:
        unpackline = a[:-1].split(delimiter)
        easting, northing, elevation = unpackline
        process.write("SRID=%s;POINT(%s %s)\t%s\n" %( SRID, easting,
northing, elevation))

    f.close()
    process.close()

Hope that helps,

Joe

pgsql-general by date:

Previous
From: "Michael Artz"
Date:
Subject: Re: FATAL: database "dspace" does not exist"
Next
From: Don Y
Date:
Subject: ISSTRICT behavior