I think that the commit b1d32d3e3 won't be backpatched to PG13 but this behavior seems like a bug to me. I would expect that "DROP EXTENSION trusted-bloom" by non-superuser owner succeeds if it’s a trusted extension. (“trusted-bloom” here means “bloom” extension you made trusted by changing the control file)
Looking at the commit b1d32d3e3, the fact that that "DROP EXTENSION trusted-bloom” by non-superuser can drop the access method belonging to the extension seems like a side effect of this commit.
Yeah, I said much the same thing on the adjacent thread on this topic.