Re: Parallel copy - Mailing list pgsql-hackers

From Amit Kapila
Subject Re: Parallel copy
Date
Msg-id CAA4eK1KF7FquLrSpM-8n-wvPKpZxZmLRusyKVETNGG91tHQq5g@mail.gmail.com
Whole thread Raw
In response to Re: Parallel copy  (vignesh C <vignesh21@gmail.com>)
Responses RE: Parallel copy
Re: Parallel copy
List pgsql-hackers
On Wed, Oct 14, 2020 at 6:51 PM vignesh C <vignesh21@gmail.com> wrote:
>
> On Fri, Oct 9, 2020 at 11:01 AM Amit Kapila <amit.kapila16@gmail.com> wrote:
> >
> > I am not able to properly parse the data but If understand the wal
> > data for non-parallel (1116 |       0 |   3587203) and parallel (1119
> > |       6 |   3624405) case doesn't seem to be the same. Is that
> > right? If so, why? Please ensure that no checkpoint happens for both
> > cases.
> >
>
> I have disabled checkpoint, the results with the checkpoint disabled
> are given below:
>                                            | wal_records | wal_fpi | wal_bytes
> Sequential Copy                   | 1116            |       0   |   3587669
> Parallel Copy(1 worker)         | 1116            |       0   |   3587669
> Parallel Copy(4 worker)         | 1121            |       0   |   3587668
> I noticed that for 1 worker wal_records & wal_bytes are same as
> sequential copy, but with different worker count I had noticed that
> there is difference in wal_records & wal_bytes, I think the difference
> should be ok because with more than 1 worker the order of records
> processed will be different based on which worker picks which records
> to process from input file. In the case of sequential copy/1 worker
> the order in which the records will be processed is always in the same
> order hence wal_bytes are the same.
>

Are all records of the same size in your test? If so, then why the
order should matter? Also, even the number of wal_records has
increased but wal_bytes are not increased, rather it is one-byte less.
Can we identify what is going on here? I don't intend to say that it
is a problem but we should know the reason clearly.

-- 
With Regards,
Amit Kapila.



pgsql-hackers by date:

Previous
From: Keisuke Kuroda
Date:
Subject: Re: Logical replication CPU-bound with TRUNCATE/DROP/CREATE many tables
Next
From: Amit Kapila
Date:
Subject: Re: Parallel Inserts in CREATE TABLE AS