Re: Assertion failure with LEFT JOINs among >500 relations - Mailing list pgsql-hackers

From Tom Lane
Subject Re: Assertion failure with LEFT JOINs among >500 relations
Date
Msg-id 2662943.1602198989@sss.pgh.pa.us
Whole thread Raw
In response to Re: Assertion failure with LEFT JOINs among >500 relations  (David Rowley <dgrowleyml@gmail.com>)
Responses Re: Assertion failure with LEFT JOINs among >500 relations  (David Rowley <dgrowleyml@gmail.com>)
List pgsql-hackers
David Rowley <dgrowleyml@gmail.com> writes:
> The reason it fails is that outer_path_rows has become infinity due to
> calc_joinrel_size_estimate continually multiplying in the join
> selectivity of 0.05 (due to our 200 default num distinct from lack of
> any stats) which after a number of iterations causes the number to
> become very large.

0.005, but yeah.  We're estimating that each additional join inflates
the output size by about 6x (1270 * 0.005), and after a few hundred
of those, it'll overflow.

> Perhaps the right fix is to modify clamp_row_est() with:

I thought of that too, but as you say, if the rowcount has overflowed a
double then we've got way worse problems.  It'd make more sense to try
to keep the count to a saner value in the first place.  

In the end, (a) this is an Assert, so not a problem for production
systems, and (b) it's going to take you longer than you want to
wait to join 500+ tables, anyhow, unless maybe they're empty.
I'm kind of disinclined to do anything in the way of a band-aid fix.

If somebody has an idea for a different way of estimating the join
size with no stats, we could talk about that.  I notice though that
the only way a plan of this sort isn't going to blow up at execution
is if the join multiplication factor is at most 1, ie the join
key is unique.  But guess what, we already know what to do in that
case.  Adding a unique or pkey constraint to users_table.user_id
causes the plan to collapse entirely (if they're left joins) or
at least still produce a small rowcount estimate (if plain joins).

            regards, tom lane



pgsql-hackers by date:

Previous
From: David Rowley
Date:
Subject: Re: Assertion failure with LEFT JOINs among >500 relations
Next
From: David Rowley
Date:
Subject: Re: Assertion failure with LEFT JOINs among >500 relations