[Raw Msg Headers][Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

RE: 2.99.55 caused DoS attack

Hi there,

Can't tell whether this was really DoS attack, but we have had encountered
the same "mx0.onetmustfall.com" domain problem on 07/03/2003. Our DNS server
was responding ServFail when queried about mx0.onemustfall.com. This
apparently caused Zmailer to immediately reissue the DNS requests, thousands
of times, probably in some buggy loop. Most likely there was only one such
message, and it managed to slow down our systems completely by blocking DNS.
We had resolved the problem by actually configuring our DNS server such that
it became authoritative source for mx0.onemustfall.com, and it respoded
NXDOMAIN. Unfortunately, we do not have the copy of the message in question.
Also, as the DNS is reconfigured, the getmxrr-test doesn't give any clue.
I'll try to convince DNS guys on Monday to "unconfigure" the mx0
authoritative answers, just to trace it.

For the time being:
$ getmxrr-test  mx0.onemustfall.com
DNS lookup reply: len=92 rcode=3 qdcount=1 ancount=0 nscount=1 arcount=0
RD=1 TC=0 AA=1 QR=1 RA=1
 NXDOMAIN smtp; 500 (DNS: no such domain: mx0.onemustfall.com)
getmxrr() rc=68 EX_NOHOST; mxcount=0
getaddrinfo('mx0.onemustfall.com','smtp') -> r=-2 (Name or service not
known), ai=(nil)

Some data on our system:
Kernel:  2.4.16
Libc 2.2.5
Zmailer: 2.99.55, compiled from sources.


-----Original Message-----
From: Matti Aarnio [mailto:mea@nic.funet.fi] 
Sent: 11 kwietnia 2003 21:38
To: Roy Bixler
Cc: zmailer@nic.funet.fi
Subject: Re: 2.99.55 caused DoS attack

On Fri, Apr 11, 2003 at 01:21:01PM -0500, Roy Bixler wrote:
> My day today has been rather "interesting" so far.
> Campus networking twice had to disable the network port for our 
> Z-Mailer 2.99.55 mail server due to DoS attack on their name servers. 
> The first time, I had already concluded that a transport agent was 
> responcible.  It was trying to resolve an MX to mx0.onemustfall.com 
> and making constant queries to the DNS to do that.  It was most likely 
> a piece of bounced spam and certainly onemustfall.com has defective 
> DNS records, but it's equally a mystery why it would cause the 
> transport

Single message ?  Or hundres of them ? 
(and all to that domain ?)

> agent to bomb our DNS servers.  In the interest of getting our mail 
> server back on the network as soon as possible, I deleted all the spam 
> from the postoffice.  After re-starting Z-Mailer, the problem was 
> solved.

That domain is yielding SERVFAIL,  I recall something of that type was once
upon a time causing quick loop kind of problems.

I pulled the 2.99.55 version, and compiled it.
There is test tool  'getmxrr-test',  which tests the DNS lookup facility.
Indeed that tool should be in your  $MAILBIN/  directory, too.

  $MAILBIN/getmxrr-test  mx0.onemustfall.com

In my test system, the freshly compiled 2.99.55 version runs very slowly.
(Observed by running it in  strace  )  Same as does up to date CVS version.

There have been glibc related problems, which may cause some problems of
their own, e.g. address lookups spin fast.

> The system is running Debian woody on x86 with Linux
> v. 2.4.19-pre7aa1.  The ZMailer is the 2.99.55-3 Debian packaging, but 
> looking at the Debian patches, no significant changes were made to 
> ZMailer source code to produce the package.  One other interesting 
> note is that I had 3 nameserver entries in my /etc/resolv.conf file 
> and our campus DNS people were complaining that, in addition to the 
> DoS attack, DNS queries were going out to all 3 nameservers at once.
> In the aftermath, I configured a BIND9 caching DNS server on localhost 
> and added a special route for messages to the 'onemustfall.com' 
> domain:
> .onemustfall.com    bitbucket!
> If it would be useful for debugging, I could try to dig up the exact 
> message causing the problem.  Even better, hopefully this bug has been 
> fixed already and my problem is running an old version of ZMailer.

Do look at what the  getmxrr-test  tool does.
If runs very fast, it would definitely indicate something to be amiss.
Possibly in system libc ...

Reading the ChangeLog  file, I have done some changes relating to that a
week after 2.99.55-patch1 was published.  I can't say for sure of those
might help.

> TIA,
> --
> Roy Bixler <rcb@ucp.uchicago.edu>
/Matti Aarnio	<mea@nic.funet.fi>
To unsubscribe from this list: send the line "unsubscribe zmailer" in the
body of a message to majordomo@nic.funet.fi
To unsubscribe from this list: send the line "unsubscribe zmailer" in
the body of a message to majordomo@nic.funet.fi