On testing I found that the Assert(outer_rel->rows > 0) was failing in get_loop_count(). The relation which it was failing on was one that I had declared dead in remove_useless_joins(). I've not done very much work so far in the costing part of the planner, however I see that set_rel_size() seems to be in charge of dishing out row estimates in this case and that it naturally does nothing for rels marked with RELOPT_DEADREL.
I've not yet determined if I can exploit this with the existing join removal code, but I can give it a try if required.
In order to get my patch working with an Assert enabled build I've had to apply the attached patch.