Re: spamassassin

["Mariano Absatz" <zmailer@lists.com.ar>]

Well,

I _do_ know the MailScanner solution... we wrote it :-) (the ZMailer, 
part, I mean).

Regarding queue size, the problem with MailScanner is that it tries to be 
fair and treat the queue like FIFO, which makes it not only read the 
queue (incoming router) directory, but stat every file in it to get its 
creation time and sort on that.

The current version (4.24), which I still didn't install (I'm using 
4.23), has a new feature which allows you to config a kind of "panic 
treshold" (in number of messages in the input router queue). If 
MailScanner reads the directory and sees that the queue size is over that 
treshold it will switch to "panic mode" (I don't recall the exact name 
Julian gave it) and it won't try to sort the files in the queue, just 
grab them as it sees them... even after the size of the queue drops below 
the treshold, it stays in "panic mode" for some 20 or 40 batches more, 
just to avoid switching modes every time during heavy peaks.

The point is that a bad quick test I made (throwing 10,000 messages to 
the router input queue that MailScanner processes) got me about 1 msg/sec 
which is awful.

I know I can optimize some of the things MailScanner does, and fine tune 
some parameters I left as default, but I don't think it will get over 3 
msgs/sec.

MailScanner does more that its share of moving things thru the hard disk 
and I think that, if I manage to somehow integrate spamc within the 
router, with a permanent spamd process running (locally or remotely) I'll 
be able to get better performance (nothing along the lines of pure 
zmailer performance, but maybe get closer to 10 than to 1)...

Regards.


El 7 Nov 2003 a las 0:01, Jeff Warnica escribió:

> Im in a simmilar enviroment (ISP, tag, delete on only extreemly high
> scores). We have dedicated hardware to the task, using an incomming
> $POSTMASTER, and MailScanner to handle the Spam, and Virus filtering.
> 
> It works, but how MS deals with its internal IPC (or rather, dosent),
> and how it deals with the queue, is concerning... Each MS process, once
> its done processing its batch, rescans the incomming queue.. So when the
> incomming queue gets 'large' things start bogging down significantly.. I
> dont remember the specific numbers, but our PIII 1GHz 512Mb (linux,
> ext3, IDE) started serious thrashing when the queue got >5000 messages.
> MS is so busy scanning the queue little mail moves through... Which only
> happened three times in a year or so of production, during spam / virus
> storms that Im sure bogged down mail servers everywhere :) Durring the
> last storm we purchesed a second scanning machine.. All is good so far,
> but OTOH, no mail storms either.
> 
> So far as I can tell, this only exists in the graphics in the zmailer
> manual, but there is mention of a $POSTOFFICE/input queue "possible
> pre-router spool for eg some email virus scanner"
> 
> In the last week I was thinking about asking aobut this.... Clearly
> ZMailer has a good idea of whats in its queues, ZMailer should direct
> some more limited scanner to individule messages, rather then a
> sepearate system handling its queue and what not. (even if MS was
> 'fixed' and was less stupid about its love affair with scanning the
> queue). I diddnt get as far as stumbling accross process.cf :) But that
> idea agrees with my somewhat vague idea of the Right Idea.
> 
> Im not sure what happens if (when!) process() fails.. But Im going to
> reinstall a machine tonight, so Ill have a platform for testing.. Not
> that I can guarentee any level of sucuess, or even sanity of this idea.
> 
> As for a practical solution that you can implement 'today': Dedicated
> hardware with MailScanner. Scan, tag, and send on....
> 
> On Thu, 2003-11-06 at 16:12, Mariano Absatz wrote:
> > I know, I know... every 2 months someone (like me) comes to the list 
> > asking how to integrate spamassassin with zmailer...
> > 
> > I also know what Eugene will say: "spamassassin is waaaaay too slow to 
> > handle any real traffic" :-)
> > 
> > However, I'm being asked to do AntiSpam tagging (not deleting) for a 
> > relatively high volume ISP, and the only open tool I know is 
> > spamassassin...
> > 
> 


--
Mariano Absatz
El Baby
----------------------------------------------------------
Justify my text? I'm sorry but it has no excuse.


-
To unsubscribe from this list: send the line "unsubscribe zmailer" in
the body of a message to majordomo@nic.funet.fi