The issue is if some other application has set PGPORT in environment variable, it reads that. It doesn't check the port mentioned in postgresql.conf.
For example in my case, We have PostgresPlus advanced server setup on some port and this variable is set. Now , postgresql 9.4 is reading that variable and not the port mentioned in postgresql.conf. Thus , server start fails.