Re: Assert failure on 'list_member_ptr(rel->joininfo, restrictinfo)' - Mailing list pgsql-hackers

From Richard Guo
Subject Re: Assert failure on 'list_member_ptr(rel->joininfo, restrictinfo)'
Date
Msg-id CAMbWs49oc3e+8JA-FOmPviMayFC1o1ugbq+wHO4Rpu9QwQohgg@mail.gmail.com
Whole thread Raw
In response to Re: Assert failure on 'list_member_ptr(rel->joininfo, restrictinfo)'  (Alexander Pyhalov <a.pyhalov@postgrespro.ru>)
Responses Re: Assert failure on 'list_member_ptr(rel->joininfo, restrictinfo)'
List pgsql-hackers

On Fri, Dec 8, 2023 at 3:13 PM Alexander Pyhalov <a.pyhalov@postgrespro.ru> wrote:
Andrei Lepikhov писал(а) 2023-12-08 07:37:
> I'd already clashed with Tom on copying the required_relids field and
> voluntarily made unnecessary copies in the project [1].
> And ... stuck into huge memory consumption. The reason was in
> Bitmapsets:
> When we have 1E3-1E4 partitions and try to reparameterize a join, one
> bitmapset field can have a size of about 1kB. Having bitmapset
> referencing Relation with a large index value, we had a lot of (for
> example, 1E4 * 1kB) copies on each reparametrization of such a field.
> Alexander Pyhalov should remember that case.

Yes. If it matters, this happened during reparametrization when 2
partitioned tables with 1000 partitions each were joined. Then
asymmetric  pw join managed to eat lots of memory for bitmapsets (by
lots of memory I mean all available on the test VM).

By reparametrization did you mean the work done in
reparameterize_path_by_child()?  If so maybe you'd be interested in the
patch [1] which postpones reparameterization of paths until createplan.c
and thus can help avoid unnecessary reparametrization work.

[1] https://www.postgresql.org/message-id/CAMbWs48PBwe1YadzgKGW_ES%3DV9BZhq00BaZTOTM6Oye8n_cDNg%40mail.gmail.com

Thanks
Richard

pgsql-hackers by date:

Previous
From: "Zhijie Hou (Fujitsu)"
Date:
Subject: RE: Synchronizing slots from primary to standby
Next
From: David Rowley
Date:
Subject: Re: Postgres db Update to Version 15