Do you have anything performing any updates or inserts to this table, even if it does not update the gist column, even if it does not update anything ?
pgsql-performance by date:
Соглашаюсь с условиями обработки персональных данных