RE: How to make partitioning scale better for larger numbers ofpartitions - Mailing list pgsql-hackers

From Tsunakawa, Takayuki
Subject RE: How to make partitioning scale better for larger numbers ofpartitions
Date
Msg-id 0A3221C70F24FB45833433255569204D1FA4FF58@G01JPEXMBYT05
Whole thread Raw
In response to Re: How to make partitioning scale better for larger numbers ofpartitions  (Amit Langote <Langote_Amit_f8@lab.ntt.co.jp>)
Responses Re: How to make partitioning scale better for larger numbers ofpartitions  (Justin Pryzby <pryzby@telsasoft.com>)
Re: How to make partitioning scale better for larger numbers of partitions  (David Rowley <david.rowley@2ndquadrant.com>)
Re: How to make partitioning scale better for larger numbers ofpartitions  (Amit Langote <Langote_Amit_f8@lab.ntt.co.jp>)
RE: How to make partitioning scale better for larger numbers ofpartitions  ("Kato, Sho" <kato-sho@jp.fujitsu.com>)
List pgsql-hackers
From: Amit Langote [mailto:Langote_Amit_f8@lab.ntt.co.jp]
> For SELECT/UPDATE/DELETE, overhead of partitioning in the planning phase
> is pretty significant and gets worse as the number of partitions grows.
> I
> had intended to fix that in PG 11, but we could only manage to get part
> of
> that work into PG 11, with significant help from David and others.  So,
> while PG 11's overhead of partitioning during planning is less than PG
> 10's due to improved pruning algorithm, it's still pretty far from ideal,
> because it isn't just the pruning algorithm that had overheads.  In fact,
> PG 11 only removes the pruning overhead for SELECT, so UPDATE/DELETE still
> carry the overhead that was in PG 10.

David has submitted multiple patches for PG 12, one of which speeds up pruning of UPDATE/DELETE (I couldn't find it in
thecurrent CF, though.)  What challenges are there for future versions, and which of them are being addressed by
patchesin progress for PG 12, and which issues are untouched?
 

> The overheads I mention stem from
> the fact that for partitioning we still rely on the old planner code
> that's used to perform inheritance planning, which requires to lock and
> open *all* partitions.  We have so far been able to refactor just enough
> to use the new code for partition pruning, but there is much refactoring
> work left to avoid needlessly locking and opening all partitions.

I remember the issue of opening and locking all partitions was discussed before.  Does this relate only to the planning
phase?

Kato-san, could you try pgbench -M prepared?


Regards
Takayuki Tsunakawa





pgsql-hackers by date:

Previous
From: Thomas Munro
Date:
Subject: Re: file cloning in pg_upgrade and CREATE DATABASE
Next
From: Thomas Munro
Date:
Subject: Re: [PATCH] pg_hba.conf : new auth option : clientcert=verify-full