[Raw Msg Headers][Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: Need advice on large mailing lists
> I am setting up a very large mailing list on a Linux(2.0.0) machine
> and I am looking for any advice on what the best way to do this might
> be. The list has approx 50,000 recipients and will be used once
> a month to send out a newsletter. The current solution uses
> majordomo and sendmail. When a mail is sent out, it kills the machine
> (to put it lightly) for a few days which is not accpetable.
For that rare posting I suggest running some batch-sending
script which picks circa 100 addresses a time from the list,
and sends them in one message, then next 100, and so on.
Sending 50 000 individual letters is also an option, but I
don't suggest it -- though it would not kill ZMailer, while
sendmail (and system) would die a horrible death..
Right now the ZMailer is not very good at expanding lists with
many thousands of addresses. The router processes producing
route analysis for such a bulks will easily grow into humongous
sizes (tens of megabytes. It used to grow in N^2 manner, but
I was fortunately able to contain that a bit..)
A datapoint on that process growth: VGER.RUTGERS.EDU has its
router processes grown to about 10 MB, which happens when it
expands largest list it has: 1830 addresses. The growth is
fairly linear, and the zero size is about 4 MB. With circa
3.3kB/recipient it gives circa 170 MB router process size,
if it has to expand the list of 50 000 addresses in core...
(The router process does not shrink, because the malloc()'s
free() does not deallocate memory pages..)
> The solutions I am looking at include replacing sendmail with zmailer
> and useing ProcList (free versions) or SmartList. We want people to be
> able to subscribe and unsubscribe to the list (we will probably also
> develop some type of web interface for this).
Yes, handling subscriptions, and doing postings can be done
entirely separately from postings, and I suggest it warmly.
Usually postings are done in a sort of list-expand operations,
however that too easily kills your mailers...
However, nothing prevents you from running the posting script
on a pipe thru your mailer -- you have some alias, which expands
to a pipe running proper scripts, and takeing the incoming
message into some temporary storage, and then sending copies
> Any thoughts, bits of wisdom, good/bad experiences would be
In the ZMailer source package there is an example of a posting
script written in PERL (utils/lightpost.pl), which posts INDIVIDUAL
messages. You can follow its lead, and write a posting program
that posts the message in bundles of a few-dozen-to-few-hundred
recipient addresses a time.
The purpose of that sample utility is to show HOW to do it in your
own code, and while it is a "mostly functional" replacement for
the /usr/lib/sendmail, it is not intended to be run for each
recipient (It being PERL should make that obvious :) )
If possible, have your posting program to sort the recipient
email domains so that (if possible) you pool recipients to some
target system to be submitted at the same time. In the SMTP
transport all those recipients will be delivered on one session
with SINGLE message block associated with all of the addresses.
> Andrew Wyllie
/Matti Aarnio <firstname.lastname@example.org>