I've done a more complete analysis with a real world database I have access to.
I've analyzed patched and current with default_statistics_target of 100 and 10000. (I also have some of the same data under 9.2, but that is not meaningfully different than unpatched head).
For easier interpretation I hacked up the analyzer so that it just reports the estimated number, never converting to the negative fraction.
The 10000 target was initially collected in an attempt to discern the truth when the 100 target methods disagreed, but I decided to just collect the gold-standard truth.
The truth is given by: select count(*) from (select distinct column from schema.table where column is not null) foo;
And number_not_null is given by: select count(*) from schema.table where column is not null;
It looks like the proposed method sometimes overestimates, although never by a huge amount, while the old one never overestimated. Overall it mostly seems to be more accurate, but occasionally it does substantially worse than the current method. I suspect most of the problems are related to the same issue reported in the last email. There are a lot of places where both underestimate, but where the new method does so by less than head.
If there are any columns anyone wants to examine further, give me the token for it and I'll try to create a generator that generates data with the same distribution (and clustering, if that seems relevant).
