One thing that's likely worth doing as part of the cross version upgrade test, even if it wouldn't even help in this case, is to run amcheck post upgrade. Just dumping data isn't going to touch indices at all.
A sequence of pg_upgrade; amcheck; upgrade all extensions; amcheck; would make sense.
