Andrei Lepikhov писал(а) 2023-12-08 07:37: > I'd already clashed with Tom on copying the required_relids field and > voluntarily made unnecessary copies in the project [1]. > And ... stuck into huge memory consumption. The reason was in > Bitmapsets: > When we have 1E3-1E4 partitions and try to reparameterize a join, one > bitmapset field can have a size of about 1kB. Having bitmapset > referencing Relation with a large index value, we had a lot of (for > example, 1E4 * 1kB) copies on each reparametrization of such a field. > Alexander Pyhalov should remember that case.
Yes. If it matters, this happened during reparametrization when 2 partitioned tables with 1000 partitions each were joined. Then asymmetric pw join managed to eat lots of memory for bitmapsets (by lots of memory I mean all available on the test VM).
By reparametrization did you mean the work done in reparameterize_path_by_child()? If so maybe you'd be interested in the patch [1] which postpones reparameterization of paths until createplan.c and thus can help avoid unnecessary reparametrization work.
