Thread: Bug in tables column data in postgres database

Hello,

 

Are-you alone to work on this database?

 

Could you check with a:

select * from pg_stat_activity;

 

Regards

 

From: Sarita Sharma <saritakumarisharma61@gmail.com>
Sent: Thursday, March 3, 2022 5:46 PM
To: pgsql-sql@lists.postgresql.org
Subject: Bug in tables column data in postgres database

 

 

CAUTION: This email originated from outside of Moody's. Do not click links or open attachments unless you recognize the sender and know the content is safe.

 

  Hi Team,

I am a backend(server side) developer and I am using postgres database for storing my data to database. I have find a big challenge working with postgres database(version 13) as I have find  major issue working with  column in table of database as data keep toggling its value. 

eg::

if I have stored data into table 

bike_table

======================================

bike_name    bike_prefix    bike_line

V111               V                  111

W222              W                 222

X333               X                  333

===========================================

 

if some other time if query is made for same table data might look like this

bike_table

======================================

bike_name    bike_prefix    bike_line

W111               V                  111

X222              W                  222

X333               X                  333

===========================================

So usually bike_name  keep changing its name thats seems to me a toggling behaviour.

Please take this issue as bug and work on this.

 

Thanks and Regards,

Sarita

-----------------------------------------

Moody's monitors email communications through its networks for regulatory compliance purposes and to protect its customers, employees and business and where allowed to do so by applicable law. The information contained in this e-mail message, and any attachment thereto, is confidential and may not be disclosed without our express permission. If you are not the intended recipient or an employee or agent responsible for delivering this message to the intended recipient, you are hereby notified that you have received this message in error and that any review, dissemination, distribution or copying of this message, or any attachment thereto, in whole or in part, is strictly prohibited. If you have received this message in error, please immediately notify us by telephone, fax or e-mail and delete the message and all of its attachments. Every effort is made to keep our network free from viruses. You should, however, review this e-mail message, as well as any attachment thereto, for viruses. We take no responsibility and have no liability for any computer virus which may be transferred via this e-mail message.

-----------------------------------------

On Thu, Mar 3, 2022 at 11:27 AM David G. Johnston <david.g.johnston@gmail.com> wrote:

On Thu, Mar 3, 2022 at 12:13 PM Sarita Sharma <saritakumarisharma61@gmail.com> wrote:

I am working as a Java developer and for backend database I am using postgres database. I am generating this tables using entity classes of Java and Jpa configuration to handling database connectivity and table creation. 

I am finding this toggling issue on my day to day work activity . I am still working with same situation. And I only have to correct data whenever its value  keeps changing automatically.

Please take my concern .

You are going to have to produce a self-contained example program (and database schema) that demonstrates the issue if you want to have any hope of having it solved.  Once you've done that, figuring out which piece of software is "buggy" should become possible.  Until you can manage that, handing off the problem to someone to solve is not going to work.

David J.

Ditto this point, and to say that in my experience with databases in general and postgres specifically, I've never even heard of this type of problem manifesting, or similar to like it. 

Whereas, I've personally encountered this type of unexpected "toggling" of data in the ORM or other language-specific layers many, many times. So just from a parsimony perspective, it's wise to look at and instrument your java stack to find the problem: as it is almost certain that this type of problem is in those layers. If it isn't there, it could be in some kind of business logic code inside stored procedures in your postgres environment of course -- that's also a very viable path of inquiry. But the idea that postgres itself is toggling your data is so improbable, that you should discount it in your analysis. It's more likely you have flaky ram or a motherboard creating the issue (and those are also highly unlikely candidates for this problem).

Steve

Hello Team,

I am getting a genuine defect that from postgres database as I am working for a well recognised organization and I am only handling coding from layers that includes spring boot, Jpa configuration for database connectivity and postgres as database.

The toggling issue I told , I have analysed it many times and I only correcting data whenever my data got toggle like I have already explained.

If you find it a genuine issue , you can take it under consideration.

As a employee I have already informed this issue to my senior colleague.

Thanks and Regards,

Sarita

On Fri, 4 Mar 2022, 01:33 Steve Midgley, <science@misuse.org> wrote:

On Thu, Mar 3, 2022 at 11:27 AM David G. Johnston <david.g.johnston@gmail.com> wrote:

On Thu, Mar 3, 2022 at 12:13 PM Sarita Sharma <saritakumarisharma61@gmail.com> wrote:

I am working as a Java developer and for backend database I am using postgres database. I am generating this tables using entity classes of Java and Jpa configuration to handling database connectivity and table creation. 

I am finding this toggling issue on my day to day work activity . I am still working with same situation. And I only have to correct data whenever its value  keeps changing automatically.

Please take my concern .

You are going to have to produce a self-contained example program (and database schema) that demonstrates the issue if you want to have any hope of having it solved.  Once you've done that, figuring out which piece of software is "buggy" should become possible.  Until you can manage that, handing off the problem to someone to solve is not going to work.

David J.

Ditto this point, and to say that in my experience with databases in general and postgres specifically, I've never even heard of this type of problem manifesting, or similar to like it. 

Whereas, I've personally encountered this type of unexpected "toggling" of data in the ORM or other language-specific layers many, many times. So just from a parsimony perspective, it's wise to look at and instrument your java stack to find the problem: as it is almost certain that this type of problem is in those layers. If it isn't there, it could be in some kind of business logic code inside stored procedures in your postgres environment of course -- that's also a very viable path of inquiry. But the idea that postgres itself is toggling your data is so improbable, that you should discount it in your analysis. It's more likely you have flaky ram or a motherboard creating the issue (and those are also highly unlikely candidates for this problem).

Steve

On Fri, Mar 4, 2022, 6:33 AM Erik Brandsberg <erik@heimdalldata.com> wrote:

As others have said, it is nearly impossible for postgres to be usable in any environment with the bug you reported, and yet it is.  It is 99.9999 % likely the bug is somewhere else. You have provided no reproduction steps as well, so no action could even be taken if it were an issue.  This is not a customer support channel with Salad as well.  If you wish to have that type of support, you may wish to contact one of the many companies that provide support for postgresql.

On Fri, Mar 4, 2022, 9:17 AM Sarita Sharma <saritakumarisharma61@gmail.com> wrote:

Hello Team,

I am getting a genuine defect that from postgres database as I am working for a well recognised organization and I am only handling coding from layers that includes spring boot, Jpa configuration for database connectivity and postgres as database.

The toggling issue I told , I have analysed it many times and I only correcting data whenever my data got toggle like I have already explained.

If you find it a genuine issue , you can take it under consideration.

As a employee I have already informed this issue to my senior colleague.

Thanks and Regards,

Sarita

On Fri, 4 Mar 2022, 01:33 Steve Midgley, <science@misuse.org> wrote:

On Thu, Mar 3, 2022 at 11:27 AM David G. Johnston <david.g.johnston@gmail.com> wrote:

On Thu, Mar 3, 2022 at 12:13 PM Sarita Sharma <saritakumarisharma61@gmail.com> wrote:

I am working as a Java developer and for backend database I am using postgres database. I am generating this tables using entity classes of Java and Jpa configuration to handling database connectivity and table creation. 

I am finding this toggling issue on my day to day work activity . I am still working with same situation. And I only have to correct data whenever its value  keeps changing automatically.

Please take my concern .

You are going to have to produce a self-contained example program (and database schema) that demonstrates the issue if you want to have any hope of having it solved.  Once you've done that, figuring out which piece of software is "buggy" should become possible.  Until you can manage that, handing off the problem to someone to solve is not going to work.

David J.

Ditto this point, and to say that in my experience with databases in general and postgres specifically, I've never even heard of this type of problem manifesting, or similar to like it. 

Whereas, I've personally encountered this type of unexpected "toggling" of data in the ORM or other language-specific layers many, many times. So just from a parsimony perspective, it's wise to look at and instrument your java stack to find the problem: as it is almost certain that this type of problem is in those layers. If it isn't there, it could be in some kind of business logic code inside stored procedures in your postgres environment of course -- that's also a very viable path of inquiry. But the idea that postgres itself is toggling your data is so improbable, that you should discount it in your analysis. It's more likely you have flaky ram or a motherboard creating the issue (and those are also highly unlikely candidates for this problem).

Steve

I would add a little to this point. Whether it is postgres or it isn't, the steps to debug are the same. The easiest first step is to instrument your Java layer with something like New Relic and to start capturing your postgres transaction log and feeding it to to an analysis environment of some kind. 

Then start filtering for activity that will show changes to the data in question. We all are confident this is not postgres core. But I am even more confident it is not postgres core doing this silently with no evidence in the transaction log. So look for activity in New Relic and pg transactions that will show you when this is happening. It should be easier to identify the issue then. 

I also on occasion work for well recognized companies - and I see spectacular errors in those development processes. The difference is that they have extensive quality processes to catch issues before they go to production. Ideally you can also draw on those resources to help you here. 

Simplify and instrument your environment. Use analytic tools. Develop steps to reproduce (and steps where the problem won't manifest. Let that guide your problem identification. Seek help internally from those with expertise of your product, the code, and the systems. 

Steve 

On Sat, Mar 5, 2022 at 1:04 AM Sarita Sharma <saritakumarisharma61@gmail.com> wrote:

Here I have reported something that seems to a bug as I observe some toggling behaviour . Just as a user only I have reported. And its all upto postres db team to take it into consideration or not instead of pointing out java layers.

Upto this much responses I got I didn't got satisfied reply.

The bottom line is that, yes, you've reported a bug.  No one here has been able to reproduce that bug on their machine.  Without reproduction it is not possible to solve.  If that isn't satisfactory maybe you should reply with exactly what you expect from us.  Because as it stands your bug report has been "closed" with a note of "unable to reproduce".  As a developer of software yourself you should be able to appreciate this as being a valid result to a bug report.

David J.

Probably caused by wrong connection settings or in a transition, no commit was issued. Or, a crash may be happening with quick, unnoticeable start up.

On Sat, Mar 5, 2022, 12:51 AM David G. Johnston <david.g.johnston@gmail.com> wrote:

On Saturday, March 5, 2022, Sarita Sharma <saritakumarisharma61@gmail.com> wrote:

Hello Team,

As I said it's a toggling behaviour it means when data will come like this is not certain. All the times we can't expect it will come.

But what I observed , I already explained as a "select query " of table in my first email, it's coming but it's not coming regularly. And when with select query data will coming like that it's not certain so only I told it a toggling behaviour.

But the thing is the most used column of the table usually shown such behaviour.

I have only explained my observation on this platform in my first email itself.

And as I said your report has been noted and determined unable to replicate.

You are welcome to accept or ignore the offered free advice being provided as to how to maybe get a better bug report produced yourself, hire your own person to figure things out while having direct access to your systems, or ignore the problem and live with its consequences.  But repeating the exact same emails will not magically result in a different outcome.

David J.

On Sat, Mar 5, 2022 at 3:29 AM Sarita Sharma <saritakumarisharma61@gmail.com> wrote:

I am only hoping that some point of time the bug reported by me will get reproduced from your end and you will find time to work on this.

I can assure you that absent further feedback and details from you no one will be working on this.  Hundreds of thousands of people have hundreds, if not thousands, of tables, in production databases, running 24x7x365, and do not experience this behavior.  The problem is specific to your organization and thus your organization's responsibility.

David J.

You are not doing what we have asked you to do.

I asked you to send me the ddl for the table including any triggers and also asked you to turn on debugging to capture dml  changes (log_statement=mod) and you didn’t do it.

You are not cooperating with all the folks that tried to help you.

Sent from my iPad

On Mar 5, 2022, at 6:14 PM, Sarita Sharma <saritakumarisharma61@gmail.com> wrote:



Hello Team,

I don't understand why Java layer is involved here. If data is saved in database table and with a select query wherever result will come that only getting fetched from java layers as a result set.

 

If toggling behaviour is observed by me in tables column of postgres database then how Java layer involvement comes.

At last my point of view is:

Here I have reported something that seems to a bug as I observe some toggling behaviour . Just as a user only I have reported. And its all upto postres db team to take it into consideration or not instead of pointing out java layers.

Upto this much responses I got I didn't got satisfied reply.

Thankyou

On Fri, 4 Mar 2022, 20:22 Steve Midgley, <science@misuse.org> wrote:

On Fri, Mar 4, 2022, 6:33 AM Erik Brandsberg <erik@heimdalldata.com> wrote:

As others have said, it is nearly impossible for postgres to be usable in any environment with the bug you reported, and yet it is.  It is 99.9999 % likely the bug is somewhere else. You have provided no reproduction steps as well, so no action could even be taken if it were an issue.  This is not a customer support channel with Salad as well.  If you wish to have that type of support, you may wish to contact one of the many companies that provide support for postgresql.

On Fri, Mar 4, 2022, 9:17 AM Sarita Sharma <saritakumarisharma61@gmail.com> wrote:

Hello Team,

I am getting a genuine defect that from postgres database as I am working for a well recognised organization and I am only handling coding from layers that includes spring boot, Jpa configuration for database connectivity and postgres as database.

The toggling issue I told , I have analysed it many times and I only correcting data whenever my data got toggle like I have already explained.

If you find it a genuine issue , you can take it under consideration.

As a employee I have already informed this issue to my senior colleague.

Thanks and Regards,

Sarita

On Fri, 4 Mar 2022, 01:33 Steve Midgley, <science@misuse.org> wrote:

On Thu, Mar 3, 2022 at 11:27 AM David G. Johnston <david.g.johnston@gmail.com> wrote:

On Thu, Mar 3, 2022 at 12:13 PM Sarita Sharma <saritakumarisharma61@gmail.com> wrote:

I am working as a Java developer and for backend database I am using postgres database. I am generating this tables using entity classes of Java and Jpa configuration to handling database connectivity and table creation. 

I am finding this toggling issue on my day to day work activity . I am still working with same situation. And I only have to correct data whenever its value  keeps changing automatically.

Please take my concern .

You are going to have to produce a self-contained example program (and database schema) that demonstrates the issue if you want to have any hope of having it solved.  Once you've done that, figuring out which piece of software is "buggy" should become possible.  Until you can manage that, handing off the problem to someone to solve is not going to work.

David J.

Ditto this point, and to say that in my experience with databases in general and postgres specifically, I've never even heard of this type of problem manifesting, or similar to like it. 

Whereas, I've personally encountered this type of unexpected "toggling" of data in the ORM or other language-specific layers many, many times. So just from a parsimony perspective, it's wise to look at and instrument your java stack to find the problem: as it is almost certain that this type of problem is in those layers. If it isn't there, it could be in some kind of business logic code inside stored procedures in your postgres environment of course -- that's also a very viable path of inquiry. But the idea that postgres itself is toggling your data is so improbable, that you should discount it in your analysis. It's more likely you have flaky ram or a motherboard creating the issue (and those are also highly unlikely candidates for this problem).

Steve

I would add a little to this point. Whether it is postgres or it isn't, the steps to debug are the same. The easiest first step is to instrument your Java layer with something like New Relic and to start capturing your postgres transaction log and feeding it to to an analysis environment of some kind. 

Then start filtering for activity that will show changes to the data in question. We all are confident this is not postgres core. But I am even more confident it is not postgres core doing this silently with no evidence in the transaction log. So look for activity in New Relic and pg transactions that will show you when this is happening. It should be easier to identify the issue then. 

I also on occasion work for well recognized companies - and I see spectacular errors in those development processes. The difference is that they have extensive quality processes to catch issues before they go to production. Ideally you can also draw on those resources to help you here. 

Simplify and instrument your environment. Use analytic tools. Develop steps to reproduce (and steps where the problem won't manifest. Let that guide your problem identification. Seek help internally from those with expertise of your product, the code, and the systems. 

Steve