Re: Query on partitioned table needs memory n_partitions * work_mem - Mailing list pgsql-general

From Dimitrios Apostolou
Subject Re: Query on partitioned table needs memory n_partitions * work_mem
Date
Msg-id 82476a55-bb26-b566-e6d9-907592ce732e@gmx.net
Whole thread Raw
In response to Re: Query on partitioned table needs memory n_partitions * work_mem  (Tom Lane <tgl@sss.pgh.pa.us>)
Responses Re: Query on partitioned table needs memory n_partitions * work_mem
List pgsql-general
On Thu, 11 Jul 2024, Tom Lane wrote:

> Dimitrios Apostolou <jimis@gmx.net> writes:
>> The TABLE test_runs_raw has 1000 partitions on RANGE(workitem_n).
>
> So don't do that.  Adding partitions is not cost-free.
>

I understand that, they also add an administrative cost that I'd rather
avoid. But I ended up adding all these partitions because of performance
issues on a multi-billion rows table. There is probably some message from
me on this list a couple of years ago.

At the moment I have a work-around. I'm thankful that everyone is willing
to provide workarounds to all potential issues/bugs I have presented, but
unfortunately workarounds are not fixes, one will hit the same wall again
at some point.

My current concern is **reporting my findings responsibly**. I want to
provide as much data needed to pinpoint the issue, so that the developers
know exactly what's going on. Having right data is half the fix.

A way to track the issue would be nice. I might revisit it and even try to
submit a patch. I wonder how the postgres development community is
tracking all these issues, I've even started forgetting the ones I have
found, and I'm sure I have previously reported (on this list) a couple of
should-be-easy issues that would be ideal for beginners.


Regards,
Dimitris




pgsql-general by date:

Previous
From: Adrian Klaver
Date:
Subject: Re: Running psql in a docker container
Next
From: "David G. Johnston"
Date:
Subject: Re: Query on partitioned table needs memory n_partitions * work_mem