On 09/30/2015 09:53 AM, Stefan Kaltenbrunner wrote:
> On 09/30/2015 03:27 AM, Amir Rohan wrote:
>> On 09/29/2015 10:51 PM, Stefan Kaltenbrunner wrote:
>>> On 09/29/2015 09:34 PM, Amir Rohan wrote:
>>>
>>> for most accesses to the archives the string for the basic auth reply
>>> quotes the "archives" and "password" strings with ' - see
>>
>> Fixed.
>
> I think you missed at least one spot in the code you added and also at
> least one occurance in existing code.
>
Hmmm, then that is a 100% miss rate when "antispam" greps precisely
twice in views.py. But, I double checked the patch (V3) and both are fixed.
Did you notice that the file contains multiple patches?
>>
>> You're right to be concerned, I raised the issue myself to begin with.
>> We can solve any particular problem, but how to optimize depends too
>> much on particulars I don't have.
>>
>> If you have both cpu and memory shortage, we could trade storage.
>> You already serve monthly mbox's, having per thread mboxes which are
>> updated in batch (say hourly) could be managable, and that code
>> is practically written already. Serving static is as cheap as it gets
>> on noth cpu and memory.
>
> yeah that is what I was thinking - though I dont think we want hourly.
> Went went a long way to actually get the current system to be "almost
> instant" in terms of having the archives in sync with the lists
>
I'm pretty sure the monthly mboxes are lagging by at least a few hours,
if they are not actually nightlies.
This reads like a rejection for this whole "generate from db" approach.
And I can't help implement the static solution, as that's cron/root
stuff.
For closure only, here is the final V4, which:
1) Batches fetches from the DB (but not as far as one network roundtrip
per message), with a new settings.THREAD_MBOX_DB_BATCH_SIZE.
2) Bails out with 403, as soon as a batch exceeds mbox size hard limit.
3) Sets Content-Disposition so the user gets a descriptive filename
instead of a hashy Message ID.
4) returns 200 ("this feature disabled") instead of 403 as previously,
Since disabling downloads shouldn't log requests as errors in your
monitoring infra.
Cheers,
Amir
