[Raw Msg Headers][Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: parallel smtp sessions to same target ost
> > sizebins="4 64 1000 0" # Bin upper bound values in kilobytes
> > # "0" is magic meaning "no limit"
> There might be situation when you have an unknown mix of jobs and, say,
> a sattelite link with looong roundtrip. Then you would prefer to feed
> jobs to several threads *un*sorted, in random order. Actually, I beleive
> that this would be more useful than `bins'. Just get next message from
> the queue and pass it to an agent that have just done previous job...
Method called PIPELINING was created to solve this exact
problem. It turns the half-duplex behaviour of the SMTP
into far more streamlined where it essentially does check-
point at DATA and at "." verbs. You can feed all of your
MAIL FROM and RCPT TO lines without waiting their individual
reports to arrive. Then you send DATA, and start listening
on all of the reports up to and including the DATA.
Of course this turns the control flow somewhat weird..
(see ZMailer's smtp transport agent, and if you can follow
the pipelined flow without severe headache, congratulations.)
The receiving end must support PIPELINING at reception,
but at simplest that means just that it does not throw
anything away at the smtp input stream even with an error
status at previous command. For example Dan Bernstain
(of qmail fame) was at first very much opposed to the
whole idea, until I (and perhaps others too) were able
to get him to verify the server behaviour, and just slap
in the magic EHLO capability token.
(Server side is easy, even the optimized case, transport-
agent is not..)
My primary inspiration for implementing it were sluggish
feed problems on Linux lists from vger.rutgers.edu to
here at nic.funet.fi -- classical half-duplex facility
was vacuuming hard silicates with messages carrying up
to 2000 recipient addresses, and perhaps 1 kB of headers
and body.. With pipelining such is handled with 2-3
second time from MAIL FROM to the ack of ".", while
classical system took 2000 * 0.3 seconds = 600 seconds...
(Ok, perhaps it takes 10 seconds for the feed with pipe-
line, but not more than that!)
I think the parallel sub-channel priorization and SMTP
PIPELINING can handle most narrow/long pipe problems.
... and with priorized sub-channels, users will soon learn
that small messages get thru, and they start asking mail
from (file-)servers in small multipart messages...