Attempt fix situation with REINDEX INDEX jobs_pkey CONCURRENTLY;
lead to corrupted index again in different tuple:
ERROR: heap tuple (69306318,15) from table "jobs" lacks matching index tuple within index "jobs_pkey"
SELECT * FROM heap_page_item_attrs(get_raw_page('jobs', 69306318), 'jobs'::regclass) where lp=15;
-[ RECORD 1 ]---
lp | 15
lp_off | 24
lp_flags | 2
lp_len | 0
select ctid,id,updated_at,created_at from jobs where ctid='(69306318,24)'::tid;
-[ RECORD 1 ]--------------------------
ctid | (69306318,24)
id | 26192320674
updated_at | 2022-01-28 13:08:27.224275
created_at | 2021-11-17 14:35:59.441979
And again updated_at happens during the REINDEX run.
select ctid,id,updated_at,created_at from jobs where id=26192320674;
(0 rows)
SELECT t_ctid, raw_flags, combined_flags FROM heap_page_item_attrs(get_raw_page('jobs', 69306318), 'jobs'::regclass), LATERAL heap_tuple_infomask_flags(t_infomask, t_infomask2) where lp=24;
-[ RECORD 1 ]--+---------------------------------------------------------------------------------------------------------------------------------------------
t_ctid | (69306318,24)
raw_flags | {HEAP_HASNULL,HEAP_HASVARWIDTH,HEAP_XMAX_KEYSHR_LOCK,HEAP_XMAX_LOCK_ONLY,HEAP_XMIN_COMMITTED,HEAP_XMAX_INVALID,HEAP_UPDATED,HEAP_ONLY_TUPLE}
combined_flags | {}
Seems something broken in combination of HOT update and REINDEX CONCURRENTLY.
