RE: Bulk Insert into PostgreSQL - Mailing list pgsql-hackers

From Tsunakawa, Takayuki
Subject RE: Bulk Insert into PostgreSQL
Date
Msg-id 0A3221C70F24FB45833433255569204D1FA27379@G01JPEXMBYT05
Whole thread Raw
In response to Re: Bulk Insert into PostgreSQL  (Srinivas Karthik V <skarthikv.iitb@gmail.com>)
Responses Re: Bulk Insert into PostgreSQL
List pgsql-hackers
From: Srinivas Karthik V [mailto:skarthikv.iitb@gmail.com]
> I was using copy command to load. Removing the primary key constraint on
> the table and then loading it helps a lot. In fact, a 400GB table was loaded
> and the primary constraint was added in around 15 hours.  Thanks for the
> wonderful suggestions.

400 GB / 15 hours = 7.6 MB/s

That looks too slow.  I experienced a similar slowness.  While our user tried to INSERT (not COPY) a billion record,
theyreported INSERTs slowed down by 10 times or so after inserting about 500 million records.  Periodic pstack runs on
Linuxshowed that the backend was busy in btree operations.  I didn't pursue the cause due to other businesses, but
theremight be something to be improved.
 


Regards
Takayuki Tsunakawa




pgsql-hackers by date:

Previous
From: Thomas Munro
Date:
Subject: Re: Commitfest 2018-07 is underway
Next
From: Thomas Munro
Date:
Subject: Re: Synchronous replay take III