Thread: Suppress 'duplicate key' at errorlog?
Hi,
Is there a way to suppress a ‘duplicate key’ msg from the errorlog? Due to a bug in a 3rd party application, this keeps on appearing and I am concerned that it will fill up the filesystem. I can delete the old errorlog, but if there is any other meaningful msgs out there, I would lose that too.
Thanks.
Maria
Hi,
Is there a way to suppress a ‘duplicate key’ msg from the errorlog? Due to a bug in a 3rd party application, this keeps on appearing and I am concerned that it will fill up the filesystem. I can delete the old errorlog, but if there is any other meaningful msgs out there, I would lose that too.
Thanks.
Maria
Not in any way that doesn't also entail the "but if there is any other meaningful msgs out there, I would lose that too." risk.
Stop thinking about the limited problem of removing the excessive messages and just learn how to deal with high-volume log situations generally. At worse you could a simple log rotation and archive with a cleanup "sed" script that deletes these messages. Suggest setting "application_name" in order to be more targeted.
David J.
<p dir="ltr">I _do_ want to phone in on the whole duplicate key issue.<p dir="ltr">Sure, this could be a race condition.It could be bad programming. It could be a variety of things.<p dir="ltr">You ABSOLUTELY do not want to ignorethat. Ignoring and moving on will cause consistency problems. Catch it and add it to a log or an email. Continue.<pdir="ltr">Do NOT silently ignore something trying to add a duplicate key.<br /><div class="cm_quote" style=" color:#787878">On Mon, Mar 30, 2015 at 5:45 PM, David G. Johnston <<a href="mailto:david.g.johnston@gmail.com">david.g.johnston@gmail.com</a>>wrote:</div><br /><div id="oldcontent" style="background-color:rgb(255, 255, 255); background-position: initial initial; background-repeat: initial initial;"><blockquotestyle=""><div dir="ltr"><div class="gmail_default" style="font-family:arial,helvetica,sans-serif"><spanstyle="font-family:arial,sans-serif">On Mon, Mar 30, 2015 at 2:03 PM,Rossi, Maria </span><span dir="ltr" style="font-family:arial,sans-serif"><<a href="mailto:maria.rossi@us.pgds.com">maria.rossi@us.pgds.com</a>></span><spanstyle="font-family:arial,sans-serif"> wrote:</span><br/></div><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:00 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div lang="EN-US"><div><p class="MsoNormal">Hi,<u></u><u></u><pclass="MsoNormal"><u></u> <u></u><p class="MsoNormal">Is there a way to suppress a ‘duplicatekey’ msg from the errorlog? Due to a bug in a 3<sup>rd</sup> party application, this keeps on appearing andI am concerned that it will fill up the filesystem. I can delete the old errorlog, but if there is any other meaningfulmsgs out there, I would lose that too.<u></u><u></u><p class="MsoNormal"><u></u> <u></u><p class="MsoNormal">Thanks.<span class="HOEnZb"><font color="#888888"><u></u><u></u></font></span><span class="HOEnZb"><fontcolor="#888888"><p class="MsoNormal"><u></u> <u></u><p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"TimesNew Roman",serif;color:teal">Maria </span></b><span style="color:#1f497d"><u></u><u></u></span><p class="MsoNormal"><u></u> <u></u></font></span></div></div></blockquote></div><br/></div><div class="gmail_extra"><div class="gmail_default"style="font-family:arial,helvetica,sans-serif">Not in any way that doesn't also entail the "but if thereis any other meaningful msgs out there, I would lose that too." risk.</div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif"><br/></div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif">Stopthinking about the limited problem of removing the excessive messagesand just learn how to deal with high-volume log situations generally. At worse you could a simple log rotation andarchive with a cleanup "sed" script that deletes these messages. Suggest setting "application_name" in order to be moretargeted.</div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif"><br /></div><div class="gmail_default"style="font-family:arial,helvetica,sans-serif">David J.</div><br /></div></div></blockquote></div>
On Mon, Mar 30, 2015 at 5:45 PM, David G. Johnston <david.g.johnston@gmail.com> wrote:Hi,
Is there a way to suppress a ‘duplicate key’ msg from the errorlog? Due to a bug in a 3rd party application, this keeps on appearing and I am concerned that it will fill up the filesystem. I can delete the old errorlog, but if there is any other meaningful msgs out there, I would lose that too.
Thanks.
Maria
Not in any way that doesn't also entail the "but if there is any other meaningful msgs out there, I would lose that too." risk.Stop thinking about the limited problem of removing the excessive messages and just learn how to deal with high-volume log situations generally. At worse you could a simple log rotation and archive with a cleanup "sed" script that deletes these messages. Suggest setting "application_name" in order to be more targeted.David J.
On Mon, Mar 30, 2015 at 3:54 PM, Scott Whitney <scott@journyx.com> wrote:I _do_ want to phone in on the whole duplicate key issue.
Sure, this could be a race condition. It could be bad programming. It could be a variety of things.
You ABSOLUTELY do not want to ignore that. Ignoring and moving on will cause consistency problems. Catch it and add it to a log or an email. Continue.
Do NOT silently ignore something trying to add a duplicate key.
The OP is "logging" the error...
A reasonably valid process would raise the error to the application which would then retry the insert using a different key. Application works fine but the error logs have a lot of expected errors in them. There is a performance concern but not a data integrity one.
David J.
On Mar 30, 2015, at 3:03 PM, Rossi, Maria <maria.rossi@us.pgds.com> wrote: > > Hi, > > Is there a way to suppress a ‘duplicate key’ msg from the errorlog? Due to a bug in a 3rd party application, this keepson appearing and I am concerned that it will fill up the filesystem. I can delete the old errorlog, but if thereis any other meaningful msgs out there, I would lose that too. No way to turn off that specific error message, but you could filter them out of the log, being careful of course not tofilter any messages which might be more meaningful. You could also prevent the errors from being raised by using a trigger,but I don't like that idea very much at all. -- Scott Ribe scott_ribe@elevated-dev.com http://www.elevated-dev.com/ https://www.linkedin.com/in/scottribe/ (303) 722-0567 voice
<p dir="ltr">Am I the only one concerned about a product trying to duplicate a pkey? This is not usually the LAST step. It'searly on in the logical process. You have to find and solve the underlying cause.<br /><div class="cm_quote" style="color: #787878">On Mon, Mar 30, 2015 at 6:17 PM, Scott Ribe <<a href="mailto:scott_ribe@elevated-dev.com">scott_ribe@elevated-dev.com</a>>wrote:</div><br /><div id="oldcontent" style="background-color:rgb(255, 255, 255); background-position: initial initial; background-repeat: initial initial;"><blockquotestyle=""><p dir="ltr">On Mar 30, 2015, at 3:03 PM, Rossi, Maria <maria.rossi@us.pgds.com> wrote:<br /> > <br /> > Hi, <br /> > <br /> > Is there a way to suppress a ‘duplicate key’ msg from the errorlog? Due to a bug in a 3rd party application, this keeps on appearing and I am concerned that it will fill up the filesystem. I can delete the old errorlog, but if there is any other meaningful msgs out there, I would lose that too.<br /><br /> No way to turn off that specific error message, but you could filter them out of the log, being carefulof course not to filter any messages which might be more meaningful. You could also prevent the errors from beingraised by using a trigger, but I don't like that idea very much at all. <br /><br /><br /> -- <br /> Scott Ribe <br/> scott_ribe@elevated-dev.com <br /> http://www.elevated-dev.com/ <br /> https://www.linkedin.com/in/scottribe/ <br />(303) 722-0567 voice <br /><br /><br /><br /><br /><br /><br /><br /> -- <br /> Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)<br /> To make changes to your subscription: <br /> http://www.postgresql.org/mailpref/pgsql-admin<br /></blockquote></div>
Scott Whitney <scott@journyx.com> writes: > Am I the only one concerned about a product trying to duplicate a pkey? This is not usually the LAST step. It's early onin the logical process. You have to find and > solve the underlying cause. I've seen lazy developers use a try insert/update if fail to implement a poor man's upsert... a feature much sought after and only now being seriously developed for Postgres. So... whether or not it's a bug is application dependent. Yes I'd question a 3rd party about this if the product wasn't developed in house and/or fully open to code review to have them tell me "we did it that way on purpose" though rather than just declaring it a nuisance. FWIW > On Mon, Mar 30, 2015 at 6:17 PM, Scott Ribe <scott_ribe@elevated-dev.com> wrote: > > On Mar 30, 2015, at 3:03 PM, Rossi, Maria <maria.rossi@us.pgds.com> wrote: > > > > Hi, > > > > Is there a way to suppress a âduplicate keyâ msg from the errorlog? Due to a bug in a 3rd party application, this keeps on appearing and I am concerned that it > will fill up the filesystem. I can delete the old errorlog, but if there is any other meaningful msgs out there,I would lose that too. > > No way to turn off that specific error message, but you could filter them out of the log, being careful of course notto filter any messages which might be more > meaningful. You could also prevent the errors from being raised by using a trigger, but I don't like that idea verymuch at all. > > -- > Scott Ribe > scott_ribe@elevated-dev.com > http://www.elevated-dev.com/ > https://www.linkedin.com/in/scottribe/ > (303) 722-0567 voice > > -- > Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org) > To make changes to your subscription: > http://www.postgresql.org/mailpref/pgsql-admin > -- Jerry Sievers Postgres DBA/Development Consulting e: postgres.consulting@comcast.net p: 312.241.7800
Thanks for all the comments/suggestions. As I mentioned, this is a 3rd-party software so what I can do to it is none or limited at best. I will try to filterthe errorlog. Another option is to set the log_min_messages and log_min_error_statement to log. Maria -----Original Message----- From: Jerry Sievers [mailto:gsievers19@comcast.net] Sent: Monday, March 30, 2015 8:54 PM To: Scott Whitney Cc: Scott Ribe; Rossi, Maria; pgsql-admin@postgresql.org Subject: Re: [ADMIN] Suppress 'duplicate key' at errorlog? Scott Whitney <scott@journyx.com> writes: > Am I the only one concerned about a product trying to duplicate a > pkey? This is not usually the LAST step. It's early on in the logical process. You have to find and solve the underlyingcause. I've seen lazy developers use a try insert/update if fail to implement a poor man's upsert... a feature much sought afterand only now being seriously developed for Postgres. So... whether or not it's a bug is application dependent. Yes I'd question a 3rd party about this if the product wasn't developed in house and/or fully open to code review to havethem tell me "we did it that way on purpose" though rather than just declaring it a nuisance. FWIW > On Mon, Mar 30, 2015 at 6:17 PM, Scott Ribe <scott_ribe@elevated-dev.com> wrote: > > On Mar 30, 2015, at 3:03 PM, Rossi, Maria <maria.rossi@us.pgds.com> wrote: > > > > Hi, > > > > Is there a way to suppress a ‘duplicate key’ msg from the errorlog? Due to a bug in a 3rd party application, this keeps on appearing and I am concerned that it > will fill up the filesystem. I can delete the old errorlog, but if there is any other meaningful msgs out there,I would lose that too. > > No way to turn off that specific error message, but you could filter them out of the log, being careful of course notto filter any messages which might be more > meaningful. You could also prevent the errors from being raised by using a trigger, but I don't like that idea verymuch at all. > > -- > Scott Ribe > scott_ribe@elevated-dev.com > http://www.elevated-dev.com/ > https://www.linkedin.com/in/scottribe/ > (303) 722-0567 voice > > -- > Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org) > To make changes to your subscription: > http://www.postgresql.org/mailpref/pgsql-admin > -- Jerry Sievers Postgres DBA/Development Consulting e: postgres.consulting@comcast.net p: 312.241.7800