[Raw Msg Headers][Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: "large customer" cases
On Wed, Oct 11, 2000 at 11:54:05AM -0300, Mariano Absatz wrote:
> We would need info about really large ISP's installation using ZMailer.
> The point is that a large customer of us is asking us to consider other
> software (payware) with large marketing behind them (software.com,
> critical path, etc).
> We are targeting large ISP's with virtual domain hosting (from a few
> hundreds domains to tens of thousands domains and from tens of thousands
> users to hundreds of thousends users).
> We are interesting in the following info (but would settle for less, or
> even different info).
> Don't forget to tag the info that you want us to keep confidential. That
> info will only be used in aggregate statistics so it can not be
> individually inferred.
> Name of provider:
> (provider is the organization providing mail services, it
> could be you, your organization, or your customer)
Sonera Corporation, several installations around
the world. (Also using payware systems, and not
always happy with them.)
> (ISP / ASP / Internet Portal / etc.)
ISP (and many other things)
> Target profile of provider's customers/users:
> (i.e. what kind of customers/users does the provider have:
> individual home users, individual business users, small
> businesses, medium business, large corporations, vertical
> markets, etc.; or combinations of the above)
Retail internet users, also small and medium business
(it all really depends on account management system,
which is external to MTA/MS system, plus a bit of
service collection, e.g. is IMAP4 supported or not.)
> Number of virtual domains hosted:
Over 30 000 per cluster.
(Indeed no technical limit.)
> Number of mailboxes hosted:
About 300 000 per message store machine.
(CPU/filesystem IOPS limited depending on how frequently
users poll their mailbox, and how much they keep email
in the mailboxes.)
(Less with UFS filesystem, more with VxFS filesystem.)
> Average number of messages/day:
MRTG data suggests: 211/min average over 24h --> 300 000/d
(per busiest system in the set.)
Roughly one message per user per day.
> Peak number of messages/hour:
500 msg/min -> 30 000/h
(top load runs daily for 2-4 hours.)
> Total number of servers running ZMailer:
> (either as inbound relay, outbound relay, local delivery, etc.)
A L4(-aware) switch fronting services to the real
backend servers. Does load-balance on users, although
if a storage service node is down, that subset of users
A crash-resistant high-available message store is
very difficult, possibly impossible, in traditional
UNIX filesystem based solutions.
That is separate issue from Crash Surviving systems,
which are apparently fairly easy in UNIXes.
Having a crash (runaway overload usually) every 2-3 months
might not necessarily be sufficient reason for spending
$10-$30 per user on extra pricy hardware, plus software
> Describe succintly (if you can/wish) the server distribution:
> 2 sun 250 with 512M RAM / 36G HD for inbound relay and
> mailbox remote access
> 2 sun netra t1 with 256M RAM / 18G HD for outbound relay
A set of Sun Ultra2 dual-cpu boxes with SSA disks.
512 MB RAM, VxFS filesystems on a stripped/mirrored
(With current hardware models, propably E220/netra + A1000)
$POSTOFFICE/ is some 4-6 GB per machine
$LOGDIR/ is 8-20 GB per machine
$MAILDIR/ is, say 1 MB * 300 000 users = 300 GB gross
(at $MAILDIR the more is the merrier..)
The $MAILDIR/ subsystem is split into two layers of
A-thru-Z subdirs giving 676 subdirs using double -X
option at mailbox program for CRC32 hash of username.
(That is the most even of various hash functions available.)
The $POSTOFFICE/ has several of its dirs divided into one
or two layers of subdir hashes. Especially the scheduler
is run with two -H options.
Usage of the L4-aware switch allows easy separation of
input SMTP servers from message store servers, although
setups have also been run with SMTP input, output,
redirection, pop3-proxy function, imap4 proxy function,
and actual pop3/imap4 servers in same set of machines.
Users were distributed among them with having multiple
A-records for site specific 'mail.xxxx.com' domain.
user-visible pop3/imap4 proxies don't need ZMailer at
their machines (if separate at all)
> Means of mailbox access (in order of usage):
> (POP3, IMAP4, WebMail, telnet, etc.)
Webmail is an external entity accessing message-
store via IMAP4
> Server software used for the following services:
> local delivery:
pop-proxy (see zmailer sources), plus seriously hacked
UCDAVIS pop3 server.
(Access only mailbox file per a: login username, b: hash
of admins choice -- use same as 'mailbox'.)
somewhat hacked UW-IMAP with similar features as pop3 server.
> Whatever else that you think applies:
> Add any info that you think is relevant:
Proprietary set of libraries contacting backend
databases via load-balance/failover arrangements
in the libraries, plus external interface appearing
Add there 'ldap-like' interface to same backend DB
to be used to map email@example.com to
firstname.lastname@example.org for delivery to the
appropriate message store node.
> We thank you in advance for any info you can provide.
> Mariano Absatz
> PERT Consultores
/Matti Aarnio <email@example.com>