Justin Pryzby <pryzby@telsasoft.com> writes: > I've seen this before while doing SET STATISTICS on a larger number of columns > using xargs, but just came up while doing ADD of a large number of columns. > Seems to be roughly linear in number of children but superlinear WRT columns. > I think having to do with catalog update / cache invalidation with many > ALTERs*children*columns?
I poked into this a bit. The operation is necessarily roughly O(N^2) in the number of columns, because we rebuild the affected table's relcache entry after each elementary ADD COLUMN operation, and one of the principal components of that cost is reading all the pg_attribute entries. However, that should only be a time cost not a space cost. Eventually I traced the O(N^2) space consumption to RememberToFreeTupleDescAtEOX, which seems to have been introduced in Simon's commit e5550d5fe, and which strikes me as a kluge of the first magnitude. Unless I am missing something, that function's design concept can fairly be characterized as "let's leak memory like there's no tomorrow, on the off chance that somebody somewhere is ignoring basic coding rules".
I tried ripping that out, and the peak space consumption of your example (with 20 child tables and 1600 columns) decreased from ~3GB to ~200MB. Moreover, the system still passes make check-world, so it's not clear to me what excuse this code has to live.
It's probably a bit late in the v10 cycle to be taking any risks in this area, but I'd vote for ripping out RememberToFreeTupleDescAtEOX as soon as the v11 cycle opens, unless someone can show an example of non-broken coding that requires it. (And if so, there ought to be a regression test incorporating that.)
Just FYI, I believe Simon's currently on holiday, so may not notice this discussion as promptly as usual.
