Greg Hennessy <greg.hennessy@gmail.com> writes:
> I am not sure if this belongs in pgsql-general or pgsql-hackers, I am
> trying first in psgl-general.
Doesn't matter a lot, but as Rob noted, you should not hijack an
existing thread. Start your own thread, don't reply to an unrelated
message.
> When I actually execute this, I get output to the terminal of the form:
> WARNING: join_selectivity: operator id 16818 jointype 0 0
> WARNING: HERE0 pgq3c_seljoin.... 5.873266e-12
> WARNING: datum result 4438812783922730423 0.000000
> WARNING: HERE0 pgq3c_seljoin.... 5.873266e-12
> WARNING: join_selectivity: 0.000000 16818 jointype 0
> WARNING: join_selectivity: s1 0.000000
> WARNING: clause_selectivity: s1 0.000000
> where it seems to me the q3c code is returning a non zero value, but in
> the guts of postgres what is found is a zero value.
I don't see any discrepancy in your results, only in your choices of
printf specifiers. "%f" defaults to "%.6f", that is, print only six
digits after the decimal point. So a value down around 1e-12 is
going to print as all zeroes.
regards, tom lane
