Thread: Suppress 'duplicate key' at errorlog?

Suppress 'duplicate key' at errorlog?

From
"Rossi, Maria"
Date:

Hi,

 

Is there a way to suppress a ‘duplicate key’ msg  from the errorlog?  Due to a bug in a 3rd party application,  this keeps on appearing and I am concerned that it will fill up the filesystem.    I can delete  the old errorlog,  but if there is any other meaningful msgs out there, I would lose that too.

 

Thanks.

 

Maria

 

Re: Suppress 'duplicate key' at errorlog?

From
"David G. Johnston"
Date:
On Mon, Mar 30, 2015 at 2:03 PM, Rossi, Maria <maria.rossi@us.pgds.com> wrote:

Hi,

 

Is there a way to suppress a ‘duplicate key’ msg  from the errorlog?  Due to a bug in a 3rd party application,  this keeps on appearing and I am concerned that it will fill up the filesystem.    I can delete  the old errorlog,  but if there is any other meaningful msgs out there, I would lose that too.

 

Thanks.

 

Maria

 


Not in any way that doesn't also entail the "but if there is any other meaningful msgs out there, I would lose that too." risk.

Stop thinking about the limited problem of removing the excessive messages and just learn how to deal with high-volume log situations generally.  At worse you could a simple log rotation and archive with a cleanup "sed" script that deletes these messages.  Suggest setting "application_name" in order to be more targeted.

David J.​

Re: Suppress 'duplicate key' at errorlog?

From
Scott Whitney
Date:
<p dir="ltr">I _do_ want to phone in on the whole duplicate key issue.<p dir="ltr">Sure, this could be a race
condition.It could be bad programming. It could be a variety of things.<p dir="ltr">You ABSOLUTELY do not want to
ignorethat. Ignoring and moving on will cause consistency problems. Catch it and add it to a log or an email.
Continue.<pdir="ltr">Do NOT silently ignore something trying to add a duplicate key.<br /><div class="cm_quote" style="
color:#787878">On Mon, Mar 30, 2015 at 5:45 PM, David G. Johnston <<a
href="mailto:david.g.johnston@gmail.com">david.g.johnston@gmail.com</a>>wrote:</div><br /><div id="oldcontent"
style="background-color:rgb(255, 255, 255); background-position: initial initial; background-repeat: initial
initial;"><blockquotestyle=""><div dir="ltr"><div class="gmail_default"
style="font-family:arial,helvetica,sans-serif"><spanstyle="font-family:arial,sans-serif">On Mon, Mar 30, 2015 at 2:03
PM,Rossi, Maria </span><span dir="ltr" style="font-family:arial,sans-serif"><<a
href="mailto:maria.rossi@us.pgds.com">maria.rossi@us.pgds.com</a>></span><spanstyle="font-family:arial,sans-serif">
wrote:</span><br/></div><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote"
style="margin:00 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div lang="EN-US"><div><p
class="MsoNormal">Hi,<u></u><u></u><pclass="MsoNormal"><u></u> <u></u><p class="MsoNormal">Is there a way to suppress a
‘duplicatekey’ msg  from the errorlog?  Due to a bug in a 3<sup>rd</sup> party application,  this keeps on appearing
andI am concerned that it will fill up the filesystem.    I can delete  the old errorlog,  but if there is any other
meaningfulmsgs out there, I would lose that too.<u></u><u></u><p class="MsoNormal"><u></u> <u></u><p
class="MsoNormal">Thanks.<span class="HOEnZb"><font color="#888888"><u></u><u></u></font></span><span
class="HOEnZb"><fontcolor="#888888"><p class="MsoNormal"><u></u> <u></u><p class="MsoNormal"><b><span
style="font-size:10.0pt;font-family:"TimesNew Roman",serif;color:teal">Maria </span></b><span
style="color:#1f497d"><u></u><u></u></span><p
class="MsoNormal"><u></u> <u></u></font></span></div></div></blockquote></div><br/></div><div class="gmail_extra"><div
class="gmail_default"style="font-family:arial,helvetica,sans-serif">Not in any way that doesn't also entail the "but if
thereis any other meaningful msgs out there, I would lose that too." risk.</div><div class="gmail_default"
style="font-family:arial,helvetica,sans-serif"><br/></div><div class="gmail_default"
style="font-family:arial,helvetica,sans-serif">Stopthinking about the limited problem of removing the excessive
messagesand just learn how to deal with high-volume log situations generally.  At worse you could a simple log rotation
andarchive with a cleanup "sed" script that deletes these messages.  Suggest setting "application_name" in order to be
moretargeted.</div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif"><br /></div><div
class="gmail_default"style="font-family:arial,helvetica,sans-serif">David J.​</div><br
/></div></div></blockquote></div>

Re: Suppress 'duplicate key' at errorlog?

From
"David G. Johnston"
Date:


On Mon, Mar 30, 2015 at 5:45 PM, David G. Johnston <david.g.johnston@gmail.com> wrote:

On Mon, Mar 30, 2015 at 2:03 PM, Rossi, Maria <maria.rossi@us.pgds.com> wrote:

Hi,

 

Is there a way to suppress a ‘duplicate key’ msg  from the errorlog?  Due to a bug in a 3rd party application,  this keeps on appearing and I am concerned that it will fill up the filesystem.    I can delete  the old errorlog,  but if there is any other meaningful msgs out there, I would lose that too.

 

Thanks.

 

Maria

 


Not in any way that doesn't also entail the "but if there is any other meaningful msgs out there, I would lose that too." risk.

Stop thinking about the limited problem of removing the excessive messages and just learn how to deal with high-volume log situations generally.  At worse you could a simple log rotation and archive with a cleanup "sed" script that deletes these messages.  Suggest setting "application_name" in order to be more targeted.

David J.​

On Mon, Mar 30, 2015 at 3:54 PM, Scott Whitney <scott@journyx.com> wrote:

I _do_ want to phone in on the whole duplicate key issue.

Sure, this could be a race condition. It could be bad programming. It could be a variety of things.

You ABSOLUTELY do not want to ignore that. Ignoring and moving on will cause consistency problems. Catch it and add it to a log or an email. Continue.

Do NOT silently ignore something trying to add a duplicate key.

The OP is "logging" the error...
 
A reasonably valid process would raise the error to the application which would then retry the insert using a different key.  Application works fine but the error logs have a lot of expected errors in them.  There is a performance concern but not a data integrity one.​

David J.

Re: Suppress 'duplicate key' at errorlog?

From
Scott Ribe
Date:
On Mar 30, 2015, at 3:03 PM, Rossi, Maria <maria.rossi@us.pgds.com> wrote:
>
> Hi,
>
> Is there a way to suppress a ‘duplicate key’ msg  from the errorlog?  Due to a bug in a 3rd party application,  this
keepson appearing and I am concerned that it will fill up the filesystem.    I can delete  the old errorlog,  but if
thereis any other meaningful msgs out there, I would lose that too. 

No way to turn off that specific error message, but you could filter them out of the log, being careful of course not
tofilter any messages which might be more meaningful. You could also prevent the errors from being raised by using a
trigger,but I don't like that idea very much at all. 


--
Scott Ribe
scott_ribe@elevated-dev.com
http://www.elevated-dev.com/
https://www.linkedin.com/in/scottribe/
(303) 722-0567 voice







Re: Suppress 'duplicate key' at errorlog?

From
Scott Whitney
Date:
<p dir="ltr">Am I the only one concerned about a product trying to duplicate a pkey? This is not usually the LAST step.
It'searly on in the logical process. You have to find and solve the underlying cause.<br /><div class="cm_quote"
style="color: #787878">On Mon, Mar 30, 2015 at 6:17 PM, Scott Ribe <<a
href="mailto:scott_ribe@elevated-dev.com">scott_ribe@elevated-dev.com</a>>wrote:</div><br /><div id="oldcontent"
style="background-color:rgb(255, 255, 255); background-position: initial initial; background-repeat: initial
initial;"><blockquotestyle=""><p dir="ltr">On Mar 30, 2015, at 3:03 PM, Rossi, Maria <maria.rossi@us.pgds.com>
wrote:<br /> > <br /> > Hi, <br /> >  <br /> > Is there a way to suppress a ‘duplicate key’ msg  from the
errorlog? Due to a bug in a 3rd party application,  this keeps on appearing and I am concerned that it will fill up the
filesystem.   I can delete  the old errorlog,  but if there is any other meaningful msgs out there, I would lose that
too.<br /><br /> No way to turn off that specific error message, but you could filter them out of the log, being
carefulof course not to filter any messages which might be more meaningful. You could also prevent the errors from
beingraised by using a trigger, but I don't like that idea very much at all. <br /><br /><br /> -- <br /> Scott Ribe
<br/> scott_ribe@elevated-dev.com <br /> http://www.elevated-dev.com/ <br /> https://www.linkedin.com/in/scottribe/ <br
/>(303) 722-0567 voice <br /><br /><br /><br /><br /><br /><br /><br /> -- <br /> Sent via pgsql-admin mailing list
(pgsql-admin@postgresql.org)<br /> To make changes to your subscription: <br />
http://www.postgresql.org/mailpref/pgsql-admin<br /></blockquote></div> 

Re: Suppress 'duplicate key' at errorlog?

From
Jerry Sievers
Date:
Scott Whitney <scott@journyx.com> writes:

> Am I the only one concerned about a product trying to duplicate a pkey? This is not usually the LAST step. It's early
onin the logical process. You have to find and 
> solve the underlying cause.

I've seen lazy developers use a try insert/update if fail to implement a
poor man's upsert...  a feature much sought after and only now being
seriously developed for Postgres.

So... whether or not it's a bug is application dependent.

Yes I'd question a 3rd party about this if the product wasn't developed
in house and/or fully open to code review to have them tell me "we did
it that way on purpose" though rather than just declaring it a nuisance.

FWIW


> On Mon, Mar 30, 2015 at 6:17 PM, Scott Ribe <scott_ribe@elevated-dev.com> wrote:
>
>     On Mar 30, 2015, at 3:03 PM, Rossi, Maria <maria.rossi@us.pgds.com> wrote:
>     >
>     > Hi,
>     >
>     > Is there a way to suppress a ‘duplicate key’ msg  from the errorlog?  Due to a bug in a 3rd party
application, this keeps on appearing and I am concerned that it 
>     will fill up the filesystem.    I can delete  the old errorlog,  but if there is any other meaningful msgs out
there,I would lose that too. 
>
>     No way to turn off that specific error message, but you could filter them out of the log, being careful of course
notto filter any messages which might be more 
>     meaningful. You could also prevent the errors from being raised by using a trigger, but I don't like that idea
verymuch at all. 
>
>     --
>     Scott Ribe
>     scott_ribe@elevated-dev.com
>     http://www.elevated-dev.com/
>     https://www.linkedin.com/in/scottribe/
>     (303) 722-0567 voice
>
>     --
>     Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
>     To make changes to your subscription:
>     http://www.postgresql.org/mailpref/pgsql-admin
>

--
Jerry Sievers
Postgres DBA/Development Consulting
e: postgres.consulting@comcast.net
p: 312.241.7800


Re: Suppress 'duplicate key' at errorlog?

From
"Rossi, Maria"
Date:
Thanks for all the comments/suggestions.

As I mentioned, this is a 3rd-party software so what I can do to it is none or limited at best.     I will try to
filterthe errorlog.  Another option is to set the log_min_messages and log_min_error_statement to log.
 

Maria 

-----Original Message-----
From: Jerry Sievers [mailto:gsievers19@comcast.net] 
Sent: Monday, March 30, 2015 8:54 PM
To: Scott Whitney
Cc: Scott Ribe; Rossi, Maria; pgsql-admin@postgresql.org
Subject: Re: [ADMIN] Suppress 'duplicate key' at errorlog?

Scott Whitney <scott@journyx.com> writes:

> Am I the only one concerned about a product trying to duplicate a 
> pkey? This is not usually the LAST step. It's early on in the logical process. You have to find and solve the
underlyingcause.
 

I've seen lazy developers use a try insert/update if fail to implement a poor man's upsert...  a feature much sought
afterand only now being seriously developed for Postgres.
 

So... whether or not it's a bug is application dependent.

Yes I'd question a 3rd party about this if the product wasn't developed in house and/or fully open to code review to
havethem tell me "we did it that way on purpose" though rather than just declaring it a nuisance.
 

FWIW


> On Mon, Mar 30, 2015 at 6:17 PM, Scott Ribe <scott_ribe@elevated-dev.com> wrote:
>
>     On Mar 30, 2015, at 3:03 PM, Rossi, Maria <maria.rossi@us.pgds.com> wrote:
>     >
>     > Hi,
>     > 
>     > Is there a way to suppress a ‘duplicate key’ msg  from the errorlog?  Due to a bug in a 3rd party
application, this keeps on appearing and I am concerned that it
 
>     will fill up the filesystem.    I can delete  the old errorlog,  but if there is any other meaningful msgs out
there,I would lose that too.
 
>    
>     No way to turn off that specific error message, but you could filter them out of the log, being careful of course
notto filter any messages which might be more
 
>     meaningful. You could also prevent the errors from being raised by using a trigger, but I don't like that idea
verymuch at all.
 
>
>     --
>     Scott Ribe
>     scott_ribe@elevated-dev.com
>     http://www.elevated-dev.com/
>     https://www.linkedin.com/in/scottribe/
>     (303) 722-0567 voice
>
>     --
>     Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
>     To make changes to your subscription:
>     http://www.postgresql.org/mailpref/pgsql-admin
>

--
Jerry Sievers
Postgres DBA/Development Consulting
e: postgres.consulting@comcast.net
p: 312.241.7800