Re: BUG #15115: temporary AND unlogged tables for parallel db import - Mailing list pgsql-bugs

From Enno Wein
Subject Re: BUG #15115: temporary AND unlogged tables for parallel db import
Date
Msg-id fb944b97-35b6-5a8b-a079-ff7a53dd830c@yahoo.com
Whole thread Raw
In response to Re: BUG #15115: temporary AND unlogged tables for parallel db import  (Tom Lane <tgl@sss.pgh.pa.us>)
Responses Re: BUG #15115: temporary AND unlogged tables for parallel db import
List pgsql-bugs
That sounds right.
Every session (72 for now) creates and drops several (temp-) tables and
indexes. Every session runs for about 30min, then does a final insert
and exits afterwards.
Depending on parallelism chosen, we run 8, 16 or 24 sessions in parallel.
The main, (global-) table which persists has been declared as either
logged or unlogged, there is much WAL traffic in any case.




On 3/16/18 08:25, Tom Lane wrote:
> Andrew Gierth <andrew@tao11.riddles.org.uk> writes:
>> "Enno" == Enno Wein <ennowein@yahoo.com> writes:
>>  Enno> However, it doesn't seem to work like that. Even with ONLY either
>>  Enno> unlogged (the global ones) and temprorary tables (the local ones), the
>>  Enno> system still produces a huge amount of WAL/checkpoint writes.
> 
>> So what you just said above is that you get a lot of WAL/checkpoint
>> writes when using unlogged non-temporary tables. What this implies is
>> that you have something _else_ going on, involving persistent tables
>> (i.e. non-temporary non-unlogged tables - these are the only kind that
>> get their data changes WAL-logged).
> 
> One theory is that the OP's workload involves a huge number of short-lived
> temp tables, and then the WAL traffic is coming from the system catalog
> changes involved in creating/dropping those tables (since even a temp
> table must have entries in pg_class, pg_attribute, etc).  That's only
> a theory though, given the lack of any hard evidence in this thread.
> 
>             regards, tom lane
> 


pgsql-bugs by date:

Previous
From: Devrim Gündüz
Date:
Subject: Re: BUG #15018: yum install postgis24_96 failure
Next
From: Tom Lane
Date:
Subject: Re: BUG #15115: temporary AND unlogged tables for parallel db import