The Postgres Pro source distribution includes several examples of index operator classes for SP-GiST, as described in Table 58.1. Look into src/backend/access/spgist/ and src/backend/utils/adt/ to see the code.