> I don't think your proposed change would work. vardata->rel is the > CTE/subquery scan rel in the outer query, and its tuples count is the > CTE's output row count, not the base table's. Using it would be > equivalent to not converting at all, since get_variable_numdistinct() > already computes -stadistinct * vardata->rel->tuples. What we need > here is the base table's rel in the subroot, which gives us the > correct rowcount for interpreting the negative fraction.
> Thanks so much for working on this! While looking at the negative stadistinct conversion, I was wondering if we might run into a potential edge case with multi-level nested subqueries. What do you think? > > /* Convert negative stadistinct to absolute count */ > > if (stats->stadistinct < 0) > { > - RelOptInfo *baserel = find_base_rel(subroot, var->varno); > + RelOptInfo *baserel = vardata->rel; > > - if (baserel->tuples > 0) > + if (baserel && baserel->tuples > 0) > { > stats->stadistinct = (float4) > clamp_row_est(-stats->stadistinct * baserel->tuples); > } > }
I don't think your proposed change would work. vardata->rel is the CTE/subquery scan rel in the outer query, and its tuples count is the CTE's output row count, not the base table's. Using it would be equivalent to not converting at all, since get_variable_numdistinct() already computes -stadistinct * vardata->rel->tuples. What we need here is the base table's rel in the subroot, which gives us the correct rowcount for interpreting the negative fraction.
