Thread: Problems with pgsql-announce

Problems with pgsql-announce

From
Josh Berkus
Date:
All,

1) I thougth we'd set up pgsql-announce so that it didn't bounce to the
sender?   I'm still getting 20-30 bounce messages every time I post to
-announce.

2) For some reason, I got four duplicate checksum warnings for *one*
posting to announce.  Example:

Duplicate posting detected
Duplicate Message-ID - <200701080946.29019.josh@postgresql.org> (Mon Jan  8
13:46:12 2007)
Duplicate Partial Message Checksum (Mon Jan  8 13:46:12 2007)

I'm certain that I sent the message only once, and it showed up on the list
promptly.  Huh?

--
--Josh

Josh Berkus
PostgreSQL @ Sun
San Francisco

Re: Problems with pgsql-announce

From
"Dan Langille"
Date:
On 8 Jan 2007 at 10:55, Josh Berkus wrote:

> 1) I thougth we'd set up pgsql-announce so that it didn't bounce to the
> sender?   I'm still getting 20-30 bounce messages every time I post to
> -announce.

That is often not the sending MTAs fault.  It is often the fault of
the receiving MTA which does not or can not differentiate between
sender and author, and replies to the author regardless.

--
Dan Langille : Software Developer looking for work
my resume: http://www.freebsddiary.org/dan_langille.php
PGCon - The PostgreSQL Conference - http://www.pgcon.org/



Re: Problems with pgsql-announce

From
"Greg Sabino Mullane"
Date:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


> 1) I thougth we'd set up pgsql-announce so that it didn't bounce to the
> sender?   I'm still getting 20-30 bounce messages every time I post to
> -announce.

Could be misconfigured on the sender's end. If you want, send me a list
of the offenders and I'll remove 'em. That goes for anyone else as well.

> 2) For some reason, I got four duplicate checksum warnings for *one*
> posting to announce.  Example:

Yeah, something weird is going on there. From looking at the moderation
queue, it seems everyone is getting three or four duplicates. Maybe
we can simply turn that off, Marc?

- --
Greg Sabino Mullane greg@turnstep.com
PGP Key: 0x14964AC8 200701081430
http://biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964AC8
-----BEGIN PGP SIGNATURE-----

iD8DBQFFopwdvJuQZxSWSsgRAqU5AKDXYzVQOHso+Lx4Aqpil4Cz8hvz7gCdHkwq
/fqhpGKT3qtjg5MEymDT9ss=
=b5kX
-----END PGP SIGNATURE-----



Re: Problems with pgsql-announce

From
Magnus Hagander
Date:
Greg Sabino Mullane wrote:
>
>> 1) I thougth we'd set up pgsql-announce so that it didn't bounce to the
>> sender?   I'm still getting 20-30 bounce messages every time I post to
>> -announce.
>
> Could be misconfigured on the sender's end. If you want, send me a list
> of the offenders and I'll remove 'em. That goes for anyone else as well.
>
>> 2) For some reason, I got four duplicate checksum warnings for *one*
>> posting to announce.  Example:
>
> Yeah, something weird is going on there. From looking at the moderation
> queue, it seems everyone is getting three or four duplicates. Maybe
> we can simply turn that off, Marc?

I think we've seen a lot of examples of that "partial message duplicate"
filter blocking things it shouldn't. I know Dave had some problems
before, and at some point it started blocking almost all mail to the
slaves list (because those are very similar *by design*).

Do we have any indication that this helps *at all*? If not, we should
just turn it off on all the lists...


Which reminds me - I saw several emails pop through today after spending
>48 hours in the Maia Mailguard. Does that one actually help noticably
for list traffic, or just for mailboxes? Because it certainly happens
often that it slows down or stops list traffic completely..

//Magnus

Re: Problems with pgsql-announce

From
Josh Berkus
Date:
Greg,

> Yeah, something weird is going on there. From looking at the moderation
> queue, it seems everyone is getting three or four duplicates. Maybe
> we can simply turn that off, Marc?

... not if duplicates are actually being received; we don't want 3-4 or
each message.

--
--Josh

Josh Berkus
PostgreSQL @ Sun
San Francisco

Re: Problems with pgsql-announce

From
"Greg Sabino Mullane"
Date:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


Josh Berkus replied:
>> Yeah, something weird is going on there. From looking at the moderation
>> queue, it seems everyone is getting three or four duplicates. Maybe
>> we can simply turn that off, Marc?
>
> ... not if duplicates are actually being received; we don't want 3-4 or
> each message.

I'm not sure what's going on, but it was working before, and now is not,
so the recent "cure" may be worse than the disease. I'm pretty sure these
are not people *sending* duplicate messages: for example, yesterday's
Weekly News appeared three times in the moderation queue, which it never
has before. Perhaps it was getting flagged as a duplicate before but not
sent to the moderation queue? Somehow I doubt that everyone's mail clients
are misconfigured to be sending duplicate mails, so the problem is on
our end.

Hm...just checked the -announce queue...their are three identical messages
from Josh Berkus about the new releases, all claiming the checksum errors.
Josh, did you ever get those checksum errors bounced back to you before the
change when you posted previously to announce? ISTR this change started with
someone complaining about getting those sent back to them?

- --
Greg Sabino Mullane greg@turnstep.com
PGP Key: 0x14964AC8 200701081540
http://biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964AC8
-----BEGIN PGP SIGNATURE-----

iD8DBQFFoq1ivJuQZxSWSsgRAmRyAJ9me1wnE1yHnsNS4abijAA4e4joMwCg7R94
y9T3a1gwpGyxBbDixo2snww=
=P3zI
-----END PGP SIGNATURE-----



Re: Problems with pgsql-announce

From
Josh Berkus
Date:
Greg,

> Hm...just checked the -announce queue...their are three identical
> messages from Josh Berkus about the new releases, all claiming the
> checksum errors. Josh, did you ever get those checksum errors bounced
> back to you before the change when you posted previously to announce?
> ISTR this change started with someone complaining about getting those
> sent back to them?

Today's announcement is the first time it's happened to me.

--
--Josh

Josh Berkus
PostgreSQL @ Sun
San Francisco

Re: Problems with pgsql-announce

From
"Marc G. Fournier"
Date:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1



- --On Monday, January 08, 2007 20:59:19 +0100 Magnus Hagander
<magnus@hagander.net> wrote:

> Greg Sabino Mullane wrote:
>>
>>> 1) I thougth we'd set up pgsql-announce so that it didn't bounce to the
>>> sender?   I'm still getting 20-30 bounce messages every time I post to
>>> -announce.
>>
>> Could be misconfigured on the sender's end. If you want, send me a list
>> of the offenders and I'll remove 'em. That goes for anyone else as well.
>>
>>> 2) For some reason, I got four duplicate checksum warnings for *one*
>>> posting to announce.  Example:
>>
>> Yeah, something weird is going on there. From looking at the moderation
>> queue, it seems everyone is getting three or four duplicates. Maybe
>> we can simply turn that off, Marc?
>
> I think we've seen a lot of examples of that "partial message duplicate"
> filter blocking things it shouldn't. I know Dave had some problems
> before, and at some point it started blocking almost all mail to the
> slaves list (because those are very similar *by design*).
>
> Do we have any indication that this helps *at all*? If not, we should
> just turn it off on all the lists...

If ppl want me to remove it altogether, I can ... and we'll see what happens
... you call it :)

>
>
> Which reminds me - I saw several emails pop through today after spending
>> 48 hours in the Maia Mailguard. Does that one actually help noticably
> for list traffic, or just for mailboxes? Because it certainly happens
> often that it slows down or stops list traffic completely..

'k, nothing shoudl be spending >48 hours in the Maia Mailguard ... all it is is
amavisd on steroids ... it only quarantines email marked as spam (>5) and I go
through it periodically to make sure un-quarantine emails to help train the
bayes filters ... most of what gets 'falsely stuck' tend to be
subscribe/unsubscribe requests though ... if you see one that has been
'spending 48 hours', please forward it to me with full headers, as something
else is wrong ...


Marc G. Fournier           Hub.Org Networking Services (http://www.hub.org)
Email . scrappy@hub.org                              MSN . scrappy@hub.org
Yahoo . yscrappy               Skype: hub.org        ICQ . 7615664
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (FreeBSD)

iD8DBQFFpvyx4QvfyHIvDvMRAgXeAJ48nr50VHXoVJc3pDm9EZXafXd4tACeOsNH
lKlzu5ukgdgDxcZrvj4CQjQ=
=PoNQ
-----END PGP SIGNATURE-----


Re: Problems with pgsql-announce

From
"Marc G. Fournier"
Date:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1



- --On Monday, January 08, 2007 12:02:53 -0800 Josh Berkus <josh@agliodbs.com>
wrote:

> Greg,
>
>> Yeah, something weird is going on there. From looking at the moderation
>> queue, it seems everyone is getting three or four duplicates. Maybe
>> we can simply turn that off, Marc?
>
> ... not if duplicates are actually being received; we don't want 3-4 or
> each message.

Right now, any duplicates are supposed to be shuffled to the moderators queue
to deal with ... what the filter is *supposed* to deal with is spam, which
tends to be alot of duplicates of the same message with different headers ...

If I remove the duplicate altogether, I suspect that there won't be much change
on the lists themselves, but I don't know for sure ...

- ----
Marc G. Fournier           Hub.Org Networking Services (http://www.hub.org)
Email . scrappy@hub.org                              MSN . scrappy@hub.org
Yahoo . yscrappy               Skype: hub.org        ICQ . 7615664
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (FreeBSD)

iD8DBQFFpv0o4QvfyHIvDvMRAh4fAJ46RyMZKD5fi2VfgfDc1uWUhAAMCgCeOV8F
PtPbLhUsWcMTQCHYbKeru4o=
=nnDt
-----END PGP SIGNATURE-----


Re: Problems with pgsql-announce

From
Dave Page
Date:
Marc G. Fournier wrote:
>
>
> --On Monday, January 08, 2007 12:02:53 -0800 Josh Berkus <josh@agliodbs.com>
> wrote:
>
>> Greg,
>
>>> Yeah, something weird is going on there. From looking at the moderation
>>> queue, it seems everyone is getting three or four duplicates. Maybe
>>> we can simply turn that off, Marc?
>> ... not if duplicates are actually being received; we don't want 3-4 or
>> each message.
>
> Right now, any duplicates are supposed to be shuffled to the moderators queue
> to deal with ... what the filter is *supposed* to deal with is spam, which
> tends to be alot of duplicates of the same message with different headers ...

Don't think I ever saw one in the queue. Plenty of bounces though...

> If I remove the duplicate altogether, I suspect that there won't be much change
> on the lists themselves, but I don't know for sure ...

It hasn't made any difference whatsoever to the slaves list, except that
we now get the reports we expect to see. I realise there's a difference
between that and other lists, but it does still get spam.

/D


Re: Problems with pgsql-announce

From
Robert Treat
Date:
On Friday 12 January 2007 03:39, Dave Page wrote:
> Marc G. Fournier wrote:
> > --On Monday, January 08, 2007 12:02:53 -0800 Josh Berkus
> > <josh@agliodbs.com>
> >
> > wrote:
> >> Greg,
> >>
> >>> Yeah, something weird is going on there. From looking at the moderation
> >>> queue, it seems everyone is getting three or four duplicates. Maybe
> >>> we can simply turn that off, Marc?
> >>
> >> ... not if duplicates are actually being received; we don't want 3-4 or
> >> each message.
> >
> > Right now, any duplicates are supposed to be shuffled to the moderators
> > queue to deal with ... what the filter is *supposed* to deal with is
> > spam, which tends to be alot of duplicates of the same message with
> > different headers ...
>
> Don't think I ever saw one in the queue. Plenty of bounces though...
>
> > If I remove the duplicate altogether, I suspect that there won't be much
> > change on the lists themselves, but I don't know for sure ...
>
> It hasn't made any difference whatsoever to the slaves list, except that
> we now get the reports we expect to see. I realise there's a difference
> between that and other lists, but it does still get spam.
>

We could change some of the more public but lower traffic lists, like
pgsql-www, first and see how that goes.

--
Robert Treat
Build A Brighter LAMP :: Linux Apache {middleware} PostgreSQL

Re: Problems with pgsql-announce

From
Magnus Hagander
Date:
> >>Which reminds me - I saw several emails pop through today after spending
> >> 48 hours in the Maia Mailguard. Does that one actually help noticably
> > for list traffic, or just for mailboxes? Because it certainly happens
> > often that it slows down or stops list traffic completely..
>
> 'k, nothing shoudl be spending >48 hours in the Maia Mailguard ... all it is is
> amavisd on steroids ... it only quarantines email marked as spam (>5) and I go
> through it periodically to make sure un-quarantine emails to help train the
> bayes filters ... most of what gets 'falsely stuck' tend to be
> subscribe/unsubscribe requests though ... if you see one that has been
> 'spending 48 hours', please forward it to me with full headers, as something
> else is wrong ...

I've definitly seen that happen and more than once. DOn't have one
around, but I'll eb sure to collect the headers next time it happens.

//Magnus

Re: Problems with pgsql-announce

From
"Greg Sabino Mullane"
Date:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


> Right now, any duplicates are supposed to be shuffled to the
> moderators queue to deal with ... what the filter is *supposed*
> to deal with is spam, which tends to be alot of duplicates of the
> same message with different headers ...

Please turn it off entirely. I can easily identify spam, but
lately I've also had to deal with duplicate legitimate messages,
which means checking to see if the message has really been already
delivered or not, which means in effect I'm acting as a human checksum
checker, which kind of defeats the purpose of the automatic one.
I could simply trust the system and reject all messages marked as
duplicate checksums, but a false positive is what started all these
changes, if I recall correctly. If you want a good test, you could
remove it from -general: it gets many, many duplicates (and spam)
throughout the day, so I'll be able to give you feedback quick
on how the change affects that queue.

Thanks,
- --
Greg Sabino Mullane greg@turnstep.com
PGP Key: 0x14964AC8 200701120941
http://biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964AC8
-----BEGIN PGP SIGNATURE-----

iD8DBQFFp54gvJuQZxSWSsgRAkCCAJ4lCtdiNr+v4y8mGT0OjXOF73v4ZgCg72BD
NUWrrNTkv2REIs2l8JsC+zY=
=W7jB
-----END PGP SIGNATURE-----



Re: Problems with pgsql-announce

From
Tom Lane
Date:
Magnus Hagander <magnus@hagander.net> writes:
>> most of what gets 'falsely stuck' tend to be
>> subscribe/unsubscribe requests though ... if you see one that has been
>> 'spending 48 hours', please forward it to me with full headers, as something
>> else is wrong ...

> I've definitly seen that happen and more than once. DOn't have one
> around, but I'll eb sure to collect the headers next time it happens.

Something else I've noticed saveral times in the last week is getting a
delayed copy of something that already came through.  These weren't the
usual newbie-sends-another-copy-after-subscribing deal because they came
from established subscribers.  I didn't save copies but will next time.

            regards, tom lane

Re: Problems with pgsql-announce

From
Magnus Hagander
Date:
On Fri, Jan 12, 2007 at 10:08:51AM -0500, Tom Lane wrote:
> Magnus Hagander <magnus@hagander.net> writes:
> >> most of what gets 'falsely stuck' tend to be
> >> subscribe/unsubscribe requests though ... if you see one that has been
> >> 'spending 48 hours', please forward it to me with full headers, as something
> >> else is wrong ...
>
> > I've definitly seen that happen and more than once. DOn't have one
> > around, but I'll eb sure to collect the headers next time it happens.
>
> Something else I've noticed saveral times in the last week is getting a
> delayed copy of something that already came through.  These weren't the
> usual newbie-sends-another-copy-after-subscribing deal because they came
> from established subscribers.  I didn't save copies but will next time.

I'm glad it's not just me. I've noticed this a couple of times, but
didn't have time to investigate at the time either - I thought it might
be my new MUA on the cellphone acting up ;-)

//Magnus

Re: Problems with pgsql-announce

From
"Marc G. Fournier"
Date:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1



- --On Friday, January 12, 2007 10:08:51 -0500 Tom Lane <tgl@sss.pgh.pa.us>
wrote:

> Magnus Hagander <magnus@hagander.net> writes:
>>> most of what gets 'falsely stuck' tend to be
>>> subscribe/unsubscribe requests though ... if you see one that has been
>>> 'spending 48 hours', please forward it to me with full headers, as something
>>> else is wrong ...
>
>> I've definitly seen that happen and more than once. DOn't have one
>> around, but I'll eb sure to collect the headers next time it happens.
>
> Something else I've noticed saveral times in the last week is getting a
> delayed copy of something that already came through.  These weren't the
> usual newbie-sends-another-copy-after-subscribing deal because they came
> from established subscribers.  I didn't save copies but will next time.

Which is why I'm hesitant to remove the duplicate checks ...

Just curious though, next time you see this, if you notice it on time, could
you send me the full headers?  I know this will be a hard one, since I imagine
the second comes through as almost a 'deja vu' kind of experience, not
simultaneously, but if you get 'lucky' enough to have them both at the same
time, be interseting to see the complete message with full headers to compare
...

- ----
Marc G. Fournier           Hub.Org Networking Services (http://www.hub.org)
Email . scrappy@hub.org                              MSN . scrappy@hub.org
Yahoo . yscrappy               Skype: hub.org        ICQ . 7615664
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (FreeBSD)

iD8DBQFFp8TG4QvfyHIvDvMRAlpDAJ9brwpCOjOBDrrY8zQ+bK0F3o10EACglwVh
0RCqqNj0RUJidxLmt8mMYRg=
=4fIm
-----END PGP SIGNATURE-----


Re: Problems with pgsql-announce

From
"Greg Sabino Mullane"
Date:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


>> Something else I've noticed saveral times in the last week is getting a
>> delayed copy of something that already came through.  These weren't the
>> usual newbie-sends-another-copy-after-subscribing deal because they came
>> from established subscribers.  I didn't save copies but will next time.

That was probably me as a result of the checksum moderation thing - I tend
to be conservative and approve messages when I am not sure, as it seems
better to have duplicates posted rather than to have nothing at all go
through. I just approved a duplicate checksum post on hackers by Christopher
Browne as a test.

It's still early, but so far the -general changes seem to be working out, Marc.

- --
Greg Sabino Mullane greg@turnstep.com
PGP Key: 0x14964AC8 200701131322
http://biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964AC8
-----BEGIN PGP SIGNATURE-----

iD8DBQFFqSNvvJuQZxSWSsgRAtO5AJ9ASA0MTE7RCVN/aSFv7B2L8YA4aACg3e0S
XTZOHZDEDlk4wbRR78r7vFY=
=EGmx
-----END PGP SIGNATURE-----



Re: Problems with pgsql-announce

From
"Marc G. Fournier"
Date:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


Try that ... see if there are a bunch of dups hitting the list now ... I can
turn it back on quickly if there are ...

- --On Friday, January 12, 2007 14:45:08 +0000 Greg Sabino Mullane
<greg@turnstep.com> wrote:

>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
>
>> Right now, any duplicates are supposed to be shuffled to the
>> moderators queue to deal with ... what the filter is *supposed*
>> to deal with is spam, which tends to be alot of duplicates of the
>> same message with different headers ...
>
> Please turn it off entirely. I can easily identify spam, but
> lately I've also had to deal with duplicate legitimate messages,
> which means checking to see if the message has really been already
> delivered or not, which means in effect I'm acting as a human checksum
> checker, which kind of defeats the purpose of the automatic one.
> I could simply trust the system and reject all messages marked as
> duplicate checksums, but a false positive is what started all these
> changes, if I recall correctly. If you want a good test, you could
> remove it from -general: it gets many, many duplicates (and spam)
> throughout the day, so I'll be able to give you feedback quick
> on how the change affects that queue.
>
> Thanks,
> - --
> Greg Sabino Mullane greg@turnstep.com
> PGP Key: 0x14964AC8 200701120941
> http://biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964AC8
> -----BEGIN PGP SIGNATURE-----
>
> iD8DBQFFp54gvJuQZxSWSsgRAkCCAJ4lCtdiNr+v4y8mGT0OjXOF73v4ZgCg72BD
> NUWrrNTkv2REIs2l8JsC+zY=
> =W7jB
> -----END PGP SIGNATURE-----
>
>



- ----
Marc G. Fournier           Hub.Org Networking Services (http://www.hub.org)
Email . scrappy@hub.org                              MSN . scrappy@hub.org
Yahoo . yscrappy               Skype: hub.org        ICQ . 7615664
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (FreeBSD)

iD8DBQFFqRx94QvfyHIvDvMRAhNjAKDftyOE7/rnOT5IU4trSJWUrQW31QCgqsPj
CBAlUnHYh5g83JrYj8pNjZQ=
=VCsd
-----END PGP SIGNATURE-----