Postgres Pro
Downloads
Home   >   mailing lists

Thread: Index based search have issues with 11.20

Index based search have issues with 11.20

From
Hafeez Rahim
Date:

Hi

This is to check regarding an issue we came across on postgreSQL community version 11 with minor patch 11.20 along with few other changes
change details mention on end part of this mail

The issue is notice after the minor patching from postgreSQL 11.16 to 11.20
The particular database runs on a dockerized platform

Issue :

One of the table query not woks when string values are filtered with = clause
where as the query works with like clause using '%' flag  or using trim(column)
below are query examples describes the problem

not working

select * from atlas_order.gdp_ship_logistic_food cgslf where shipping_group_id = '2VxFRCOG9tc7SjQ2Ogg0bveve96';

working

select * from atlas_order.gdp_ship_logistic_food cgslf where shipping_group_id like '%2VxFRCOG9tc7SjQ2Ogg0bveve96%';
select * from atlas_order.gdp_ship_logistic_food cgslf where shipping_group_id like '2VxFR%COG9tc7SjQ2Ogg0bveve96';
 select * from atlas_order.gdp_ship_logistic_food cgslf where trim(shipping_group_id) = '2VxFR%COG9tc7SjQ2Ogg0bveve96';

Later as per the feedback from dev we have dropped and recreated the constraints and rebuild the indexes of the affected tables
and the data is accessible after on

the similar pattern is noticed across other tables related to primary - foreign key relationships

Could you please provide a feedback whether similar issues are reported on 11.20 release
or what could be the reason for the error

Changes executed
  1. PostgreSQL minor version upgrade from 11.16 to 11.20
  2. Change docker container base image from Debian to Alpine  (Debian Bookworm - alpine3.18)
  3. Adding PGSENTINEL extension to collect active sessions details
  4. Changing session limit for superuser from 3 to 20
  5. Kernel Patching from version 3.10.0-1160.76.1.el7.x86_64 to 3.10.0-1160.99.1.el7.x86_64
Note : The OS got restarted as part of the maintenance


--
Regards,
Hafeez
Database Technical Lead

Mindcurv TSPL
#6F, T-1, World Trade Centre,
Infopark CSEZ, P-1,
Kakkanad, Cochin - 682042

t: +91 (0)484 2100164
m: +91 9605445282
www.mindcurv.com


CONFIDENTIALITY NOTICE: This e-mail and any attachments hereto are intended only for use by the addressee(s) named herein and may contain
legally privileged and/or confidential information. If you are not the intended recipient of this e-mail, you are hereby notified that any
dissemination, distribution or copying of this e-mail, and any attachments thereto, is strictly prohibited. If you have received this
e-mail in error, please notify the sender by replying to this message and permanently delete the original and any copy of this e-mail and
any printout thereof.

--
CONFIDENTIALITY NOTICE: This e-mail and any attachments hereto are intended only for use by the addressee(s) named herein and may contain legally privileged and/or confidential information. If you are not the intended recipient of this e-mail, you are hereby notified that any dissemination, distribution or copying of this e-mail, and any attachments thereto, is strictly prohibited. If you have received this e-mail in error, please notify the sender by replying to this message and permanently delete the original and any copy of this e-mail and any printout thereof.

Re: Index based search have issues with 11.20

From
Tom Lane
Date:
Hafeez Rahim <hafeez.rahim@mindcurv.com> writes:
> Issue :

> One of the table query not woks when string values are filtered with =
> clause
> where as the query works with like clause using '%' flag  or using
> trim(column)
> below are query examples describes the problem

You've not really provided enough detail, but I'm going to guess
that your OS update included a change in collation definitions
that left any indexes on text columns essentially corrupt.
The LIKE queries probably weren't affected because they didn't
rely on indexes.

Reindexing the indexes should have been enough to fix it.
See

https://wiki.postgresql.org/wiki/Locale_data_changes

            regards, tom lane

Re: Index based search have issues with 11.20

From
Adrian Klaver
Date:
On 10/17/23 11:24, Hafeez Rahim wrote:
> 
> Hi
> 
> This is to check regarding an issue we came across on postgreSQL 
> community version 11 with minor patch 11.20 along with few other changes
> change details mention on end part of this mail
> 
> The issue is notice after the minor patching from postgreSQL 11.16 to 11.20
> The particular database runs on a dockerized platform
> 
> Issue :
> 
> One of the table query not woks when string values are filtered with = 
> clause
> where as the query works with like clause using '%' flag  or using 
> trim(column)
> below are query examples describes the problem
> 
> not working
> 
> select * from atlas_order.gdp_ship_logistic_food cgslf where 
> shipping_group_id = '2VxFRCOG9tc7SjQ2Ogg0bveve96';
> 
> working
> 
> select * from atlas_order.gdp_ship_logistic_food cgslf where 
> shipping_group_id like '%2VxFRCOG9tc7SjQ2Ogg0bveve96%';
> select * from atlas_order.gdp_ship_logistic_food cgslf where 
> shipping_group_id like '2VxFR%COG9tc7SjQ2Ogg0bveve96';
> select * from atlas_order.gdp_ship_logistic_food cgslf where 
> trim(shipping_group_id) = '2VxFR%COG9tc7SjQ2Ogg0bveve96';
> 
> Later as per the feedback from dev we have dropped and recreated the 
> constraints and rebuild the indexes of the affected tables
> and the data is accessibleafter on

I would say the above is due to the distro change below.

>  2. Change docker container base image from Debian to Alpine  (Debian
>     Bookworm - alpine3.18)

My guess something along lines of:

https://wiki.postgresql.org/wiki/Locale_data_changes

Changing distro's mid stream increases the likely hood there will be issues.


-- 
Adrian Klaver
adrian.klaver@aklaver.com

Re: Index based search have issues with 11.20

From
Hafeez Rahim
Date:
Thanks for the providing the references

we will go through the wiki link and identify the glibc version updates during the OS patching.


On Tue, Oct 17, 2023 at 10:18 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
Hafeez Rahim <hafeez.rahim@mindcurv.com> writes:
> Issue :

> One of the table query not woks when string values are filtered with =
> clause
> where as the query works with like clause using '%' flag  or using
> trim(column)
> below are query examples describes the problem

You've not really provided enough detail, but I'm going to guess
that your OS update included a change in collation definitions
that left any indexes on text columns essentially corrupt.
The LIKE queries probably weren't affected because they didn't
rely on indexes.

Reindexing the indexes should have been enough to fix it.
See

https://wiki.postgresql.org/wiki/Locale_data_changes

                        regards, tom lane


--
Regards,
Hafeez
Database Technical Lead

Mindcurv TSPL
#6F, T-1, World Trade Centre,
Infopark CSEZ, P-1,
Kakkanad, Cochin - 682042

t: +91 (0)484 2100164
m: +91 9605445282
www.mindcurv.com


CONFIDENTIALITY NOTICE: This e-mail and any attachments hereto are intended only for use by the addressee(s) named herein and may contain
legally privileged and/or confidential information. If you are not the intended recipient of this e-mail, you are hereby notified that any
dissemination, distribution or copying of this e-mail, and any attachments thereto, is strictly prohibited. If you have received this
e-mail in error, please notify the sender by replying to this message and permanently delete the original and any copy of this e-mail and
any printout thereof.

--
CONFIDENTIALITY NOTICE: This e-mail and any attachments hereto are intended only for use by the addressee(s) named herein and may contain legally privileged and/or confidential information. If you are not the intended recipient of this e-mail, you are hereby notified that any dissemination, distribution or copying of this e-mail, and any attachments thereto, is strictly prohibited. If you have received this e-mail in error, please notify the sender by replying to this message and permanently delete the original and any copy of this e-mail and any printout thereof.

Re: Index based search have issues with 11.20

From
Hafeez Rahim
Date:
Thanks for the details

We will further check on the references given

On Tue, Oct 17, 2023 at 10:20 PM Adrian Klaver <adrian.klaver@aklaver.com> wrote:
On 10/17/23 11:24, Hafeez Rahim wrote:
>
> Hi
>
> This is to check regarding an issue we came across on postgreSQL
> community version 11 with minor patch 11.20 along with few other changes
> change details mention on end part of this mail
>
> The issue is notice after the minor patching from postgreSQL 11.16 to 11.20
> The particular database runs on a dockerized platform
>
> Issue :
>
> One of the table query not woks when string values are filtered with =
> clause
> where as the query works with like clause using '%' flag  or using
> trim(column)
> below are query examples describes the problem
>
> not working
>
> select * from atlas_order.gdp_ship_logistic_food cgslf where
> shipping_group_id = '2VxFRCOG9tc7SjQ2Ogg0bveve96';
>
> working
>
> select * from atlas_order.gdp_ship_logistic_food cgslf where
> shipping_group_id like '%2VxFRCOG9tc7SjQ2Ogg0bveve96%';
> select * from atlas_order.gdp_ship_logistic_food cgslf where
> shipping_group_id like '2VxFR%COG9tc7SjQ2Ogg0bveve96';
> select * from atlas_order.gdp_ship_logistic_food cgslf where
> trim(shipping_group_id) = '2VxFR%COG9tc7SjQ2Ogg0bveve96';
>
> Later as per the feedback from dev we have dropped and recreated the
> constraints and rebuild the indexes of the affected tables
> and the data is accessibleafter on

I would say the above is due to the distro change below.

>  2. Change docker container base image from Debian to Alpine  (Debian
>     Bookworm - alpine3.18)

My guess something along lines of:

https://wiki.postgresql.org/wiki/Locale_data_changes

Changing distro's mid stream increases the likely hood there will be issues.


--
Adrian Klaver
adrian.klaver@aklaver.com



--
Regards,
Hafeez
Database Technical Lead

Mindcurv TSPL
#6F, T-1, World Trade Centre,
Infopark CSEZ, P-1,
Kakkanad, Cochin - 682042

t: +91 (0)484 2100164
m: +91 9605445282
www.mindcurv.com


CONFIDENTIALITY NOTICE: This e-mail and any attachments hereto are intended only for use by the addressee(s) named herein and may contain
legally privileged and/or confidential information. If you are not the intended recipient of this e-mail, you are hereby notified that any
dissemination, distribution or copying of this e-mail, and any attachments thereto, is strictly prohibited. If you have received this
e-mail in error, please notify the sender by replying to this message and permanently delete the original and any copy of this e-mail and
any printout thereof.

--
CONFIDENTIALITY NOTICE: This e-mail and any attachments hereto are intended only for use by the addressee(s) named herein and may contain legally privileged and/or confidential information. If you are not the intended recipient of this e-mail, you are hereby notified that any dissemination, distribution or copying of this e-mail, and any attachments thereto, is strictly prohibited. If you have received this e-mail in error, please notify the sender by replying to this message and permanently delete the original and any copy of this e-mail and any printout thereof.