ACL lists in tables may potentially be large in size. I suggest adding TOAST support for system tables, namely pg_class, pg_attribute and pg_largeobject_metadata, for they include ACL columns.
In commit 96cdeae0 these tables were missed because of possible conflicts with VACUUM FULL and problem with seeing a non-empty new cluster by pg_upgrade. Given patch fixes both expected bugs. Also patch adds a new regress test for checking TOAST properly working with ACL attributes. Suggested feature has been used in Postgres Pro Enterprise for a long time, and it helps with large ACL's.