This is interesting because "VACUUM ANALYZE" solved the problem on postgresql 12.1, the planning time was cut down from 5165.742 ms to 517 ms.
This is great but I didn't think to do this on postgresql 12.1 because I did the same thing on the production server(postgresql 9.5) and the problem was not solved there by this command on the 9.5.