Re: Bulk Insert into PostgreSQL - Mailing list pgsql-hackers

From Srinivas Karthik V
Subject Re: Bulk Insert into PostgreSQL
Date
Msg-id CAEfuzeRWJufow_pn7bPJ8MB9jzUNpHn0piY6xiv+NBhTbRtFuA@mail.gmail.com
Whole thread Raw
In response to Re: Bulk Insert into PostgreSQL  (Peter Geoghegan <pg@bowt.ie>)
Responses Re: Bulk Insert into PostgreSQL  (Peter Geoghegan <pg@bowt.ie>)
List pgsql-hackers
@Peter: I was indexing the primary key of all the tables in tpc-ds. Some of the fact tables has multiple columns as part of the primary key. Also, most of them are numeric type.

On Mon, Jul 2, 2018 at 7:09 AM, Peter Geoghegan <pg@bowt.ie> wrote:
On Sun, Jul 1, 2018 at 5:19 PM, Tsunakawa, Takayuki
<tsunakawa.takay@jp.fujitsu.com> wrote:
> 400 GB / 15 hours = 7.6 MB/s
>
> That looks too slow.  I experienced a similar slowness.  While our user tried to INSERT (not COPY) a billion record, they reported INSERTs slowed down by 10 times or so after inserting about 500 million records.  Periodic pstack runs on Linux showed that the backend was busy in btree operations.  I didn't pursue the cause due to other businesses, but there might be something to be improved.

What kind of data was indexed? Was it a bigserial primary key, or
something else?

--
Peter Geoghegan

pgsql-hackers by date:

Previous
From: Nikita Glukhov
Date:
Subject: Re: [HACKERS] [PATCH] kNN for SP-GiST
Next
From: Alvaro Herrera
Date:
Subject: Re: Cache invalidation after authentication (on-the-fly rolecreation)