"The data types smallserial, serial and bigserial are not true types, but merely a notational convenience for creating unique identifier columns"
so really there's no difference, what you have is fine.
-Michel
Reading the CREATE TABLE pages in the manual did not give me sufficient insights to appreciate the differences or indicate how to change the column's data type.