Thread: Invalid Parent Index Issue Will Not Resolve Despite Multiple Attempts To Fix
Hello all,
We have been utilizing partitioned tables with indexes. We've recently had an issue where the parent table's index (id, date) became invalid (indisvalid=FALSE, indisready=FALSE in pg_index). For reference the parent table is partitioned on a date field within the table.
In order to find the indices causing the problem we utilized the following:
with invalid_indices as ( select n.nspname, c.relname as parent_index_name, i.indrelid parent_table_oid, i.indexrelid parent_index_oid, x.indexdef as parent_indexdef, substring(pg_get_indexdef(i.indexrelid), '.* btree \((.*)\)') as parent_index_cols from pg_catalog.pg_class c, pg_catalog.pg_namespace n, pg_catalog.pg_index i, pg_catalog.pg_indexes x where true and (i.indisvalid = false or i.indisready = false) and i.indexrelid = c.oid and c.relnamespace = n.oid and n.nspname != 'pg_catalog' and n.nspname != 'information_schema' and n.nspname != 'pg_toast' and n.nspname = x.schemaname and c.relname = x.indexname ), tables_with_invalid_indices as ( select i.*, c.relname as parent_table_name from invalid_indices i left join pg_class c on i.parent_table_oid = c.oid ), children_of_tables_with_invalid_indices as ( select t.*, i.inhrelid as child_table_oid, c.relname as child_table_name from tables_with_invalid_indices t left join pg_inherits i on t.parent_table_oid = i.inhparent left join pg_class c on i.inhrelid = c.oid ), -- for each index on parent table, left join against index on child table all_indices_on_children_of_tables_with_invalid_indices as ( select c.*, a.oid as child_index_oid, a.relname as child_index_name, a.relispartition as child_index_ispartition, h.inhparent as parent_of_child_index_oid, x.indexdef as child_indexdef, substring(pg_get_indexdef(a.oid), '.* btree \((.*)\)') as child_index_cols from children_of_tables_with_invalid_indices c left join pg_index i on c.child_table_oid = i.indrelid inner join pg_class a on i.indexrelid = a.oid and parent_index_cols = substring(pg_get_indexdef(a.oid), '.* btree \((.*)\)') left join pg_indexes x on a.relname = x.indexname left join pg_inherits h on h.inhrelid = a.oid ), unattached_indices_on_child_tables as ( select * from all_indices_on_children_of_tables_with_invalid_indices where not child_index_ispartition ), missing_indices_on_child_tables as ( select a.*, b.child_index_oid, b.child_index_name, b.child_index_ispartition, b.child_indexdef, b.parent_of_child_index_oid from children_of_tables_with_invalid_indices a left join all_indices_on_children_of_tables_with_invalid_indices b on a.child_table_name = b.child_table_name and a.parent_index_oid = b.parent_of_child_index_oid where b.parent_of_child_index_oid is null ), -- select * from all_indices_on_children_of_tables_with_invalid_indices problems as ( select u.parent_table_name, u.parent_index_name, u.child_table_name, u.child_index_name, u.parent_indexdef, concat_ws(' ','ALTER INDEX', u.parent_index_name, 'ATTACH PARTITION', u.child_index_name, ';') as fix_sql, u.child_index_cols, u.parent_index_cols from unattached_indices_on_child_tables u union select m.parent_table_name, m.parent_index_name, m.child_table_name, m.child_index_name, m.parent_indexdef, 'CREATE INDEX CONCURRENTLY IF NOT EXISTS ' || 'ix_ledger_' || m.child_table_name || '_' || replace(m.parent_index_cols, ', ', '_') || ' ON ' || m.child_table_name || ' USING btree (' || m.parent_index_cols || ');' as fix_sql, '' as child_index_cols, m.parent_index_cols from missing_indices_on_child_tables m ) select * from problems;
We attempted to fix the issue by doing the following:
ALTER TABLE table_parent DETACH PARTITION table_badpartition;
DROP INDEX brokenchildindex;
CREATE INDEX newchildindex on table_badpartition using btree (id, date);
ALTER TABLE table_parent ATTACH PARTITION table_badpartition
FOR VALUES FROM (date) TO (date+1);
This did not fix the issue so we attempted an alternate fix:
begin;
set role readwrite;
ALTER TABLE table_parent DETACH PARTITION table_badpartition;
ALTER TABLE table_badpartition RENAME TO table_badpartition_detached;
CREATE TABLE table_badpartition PARTITION OF table_parent
FOR VALUES FROM (date) TO (date+1);
ALTER TABLE table_badpartitioneplica identity full;
INSERT INTO table_badpartition (id, date, ...)
SELECT id, date, ... from table_badpartition_detached;
commit;
This new table was created with the correct columns, the accurate data, and the correct indices however the parent index is still listed with indisvalid = FALSE and indisready = FALSE.
We did some research within the mailing list archives and found a mention that this was an issue back in 2018 (https://postgrespro.com/list/thread-id/2416574) with a discussion in pghackers (https://www.postgresql.org/message-id/20181203225019.2vvdef2ybnkxt364@alvherre.pgsql) mentioning a patch.
Is this still a known issue? Or is there a way that we haven't thought of to fix the invalid parent index without reindexing?
Thanks,
Noel Parker
she/they
Re: Invalid Parent Index Issue Will Not Resolve Despite Multiple Attempts To Fix
From
Laurenz Albe
Date:
On Thu, 2023-12-07 at 16:38 -0600, Noel Jones wrote: > We have been utilizing partitioned tables with indexes. We've recently had an issue > where the parent table's index (id, date) became invalid (indisvalid=FALSE, > indisready=FALSE in pg_index). For reference the parent table is partitioned on a > date field within the table. Indexes don't become invalid just so. I wonder what happened. > We attempted to fix the issue by doing the following: > > ALTER TABLE table_parent DETACH PARTITION table_badpartition; > DROP INDEX brokenchildindex; > CREATE INDEX newchildindex on table_badpartition using btree (id, date); > ALTER TABLE table_parent ATTACH PARTITION table_badpartition > FOR VALUES FROM (date) TO (date+1); > > This did not fix the issue so we attempted an alternate fix: > > begin; > set role readwrite; > ALTER TABLE table_parent DETACH PARTITION table_badpartition; > ALTER TABLE table_badpartition RENAME TO table_badpartition_detached; > CREATE TABLE table_badpartition PARTITION OF table_parent > FOR VALUES FROM (date) TO (date+1); > ALTER TABLE table_badpartitioneplica identity full; > INSERT INTO table_badpartition (id, date, ...) > SELECT id, date, ... from table_badpartition_detached; > commit; > > This new table was created with the correct columns, the accurate data, and the > correct indices however the parent index is still listed with indisvalid = FALSE > and indisready = FALSE. You forgot to ALTER INDEX ... ATTACH PARTITION to turn the invalid index on the partitioned table into a valid index. Yours, Laurenz Albe
Re: Invalid Parent Index Issue Will Not Resolve Despite Multiple Attempts To Fix
From
Noel Jones
Date:
Thank you for your response. I forgot to include it but we did check for that prior to our submission. We used this query to see how many indexes were related to the parent index via the inherits table:
SELECT
count(inh.inhrelid)
FROM pg_class c
inner join pg_inherits inh on c.oid = inh.inhparent
WHERE c.oid = broken_parent_index_oid
This query returned a value equal to the number of partitions which implies that all of the indexes on the partitions are attached to the parent index that is showing up as broken.
As an additional check this morning I also confirmed that the indexes were of the same definition and that one of the others (date, not_id) did not somehow get attached to this parent index using this:From this I'm seeing that the columns in the definitions are matching between the two in addition to being attached.SELECT
par.relname parent_index,
chi.relname child_index,
substring(pg_get_indexdef(par.oid), '.* btree \((.*)\)') as parent_index_cols,
substring(pg_get_indexdef(chi.oid), '.* btree \((.*)\)') as child_index_cols,
par.oid parent_index_oid,
inh.inhparent inherit_parent_oid,
inh.inhrelid inh_child_oid,
ind.indexrelid child_index_oid
FROM pg_inherits inh
join pg_index ind on inh.inhrelid = ind.indexrelid
join pg_class chi on ind.indexrelid = chi.oid
join pg_class par on inh.inhparent = par.oid
WHERE inh.inhparent = broken_parent_index_oid;
On Thu, Dec 7, 2023 at 10:26 PM Laurenz Albe <laurenz.albe@cybertec.at> wrote:
This is an EXTERNAL EMAIL. STOP. THINK before you CLICK links or OPEN attachments.
______________________________________________________________________
On Thu, 2023-12-07 at 16:38 -0600, Noel Jones wrote:
> We have been utilizing partitioned tables with indexes. We've recently had an issue
> where the parent table's index (id, date) became invalid (indisvalid=FALSE,
> indisready=FALSE in pg_index). For reference the parent table is partitioned on a
> date field within the table.
Indexes don't become invalid just so. I wonder what happened.
> We attempted to fix the issue by doing the following:
>
> ALTER TABLE table_parent DETACH PARTITION table_badpartition;
> DROP INDEX brokenchildindex;
> CREATE INDEX newchildindex on table_badpartition using btree (id, date);
> ALTER TABLE table_parent ATTACH PARTITION table_badpartition
> FOR VALUES FROM (date) TO (date+1);
>
> This did not fix the issue so we attempted an alternate fix:
>
> begin;
> set role readwrite;
> ALTER TABLE table_parent DETACH PARTITION table_badpartition;
> ALTER TABLE table_badpartition RENAME TO table_badpartition_detached;
> CREATE TABLE table_badpartition PARTITION OF table_parent
> FOR VALUES FROM (date) TO (date+1);
> ALTER TABLE table_badpartitioneplica identity full;
> INSERT INTO table_badpartition (id, date, ...)
> SELECT id, date, ... from table_badpartition_detached;
> commit;
>
> This new table was created with the correct columns, the accurate data, and the
> correct indices however the parent index is still listed with indisvalid = FALSE
> and indisready = FALSE.
You forgot to ALTER INDEX ... ATTACH PARTITION to turn the invalid index on the
partitioned table into a valid index.
Yours,
Laurenz Albe