Re: router, mailing lists, memory and suns...

[Matti Aarnio <mea@utu.fi>]

> It appears that the router can corrupt disks on SunOS 4.1.2 and
> probably other SunOS releases. This appears related to memory bloat in
> the router when expanding mailing lists.  At one time I believed this
> only happened with Rayans zmailer, and I switched to Matti's. While
> Matti's has some wonderful improvements, it seems that slowly, over
> time the router grows in memory usage until it scribbles all over the
> disk.

	You can grow it rather rapidly to that large mode too, but
	then it stabilizes.  I have heard of real leak on Linux systems
	with DNS resolver appearing to be the culprit.

	There are two different reasons for the bloat:
	- using zmsh script attribute processing does about
	  N^2 memory usage in respect to the number of recipients
	- ZMSH scripting language has a bit inefficient memory
	  management, while being able to do  mark()/release()
	  type of thing on objects, it copies them a bit too much..

> This is obviously a SunOS bug, but we don't have a maintanence
> contract and want to stick with something more BSD-ish than their
> latest releases, so I was hoping someone here could suggest something,
> or perhaps Matti will be able to track it down.
> 
> Here is a ps -auxww after expanding two large (> 1500) lists:
> 
> root     26148  2.7  5.0 5508 1144 ?  R    Feb 14  4:38 /usr/local/lib/zmailer/bin/router -dkn 3
> root     26147  2.3  5.5 5516 1252 ?  R    Feb 14  4:36 /usr/local/lib/zmailer/bin/router -dkn 3
> root     26139  0.0  4.2  888  960 ?  S    Feb 14  2:43 /usr/local/lib/zmailer/bin/router -dkn 3

	For a comparison, on SunOS 4.1.4:  (vger.rutgers.edu)

bash# ps auxww|grep router
root     16892  2.0  4.4 6864 3104 ?  R    Feb 17123:16 /vger/u3/zmailer/bin/router -dkn 4
root     16894  2.0  4.1 6872 2880 ?  R    Feb 17123:18 /vger/u3/zmailer/bin/router -dkn 4
root     16889  2.0  6.2 6976 4424 ?  R    Feb 17122:57 /vger/u3/zmailer/bin/router -dkn 4
root     16895  1.6  4.8 6752 3416 ?  R    Feb 17123:40 /vger/u3/zmailer/bin/router -dkn 4

	There are several lists of about 1500-1700 recipients apiece..

> I now have to kill and restart the router or else spend a morning
> restoring disks and listening to user complaints :-(

	Restoring disks ?  Does your partitioning have some overlaps ?
	I recall several bug-fixes to SunOS 4.1.3 with regards to UFS
	panics, in fact one of my systems was practically unusable until
	those fixes were applied.

> Thanks for any advice - I wish I had time to debug this myself, but I
> am presently overloaded at work.
> 
> The router version is 2.99.20.

	At ..25 I found one nasty bug at  mailbox  TA, which did caused
	mailbox zero-truncation, in case the write failed.

> Thanks again,
> sdb
> -- 
> sdb@ssr.com

	/Matti Aarnio <mea@utu.fi> <mea@nic.funet.fi>