Re: zmailer scheduler problems (Lai Yiu Fai's report)

[Matti Aarnio <mea@nic.funet.fi>]

Hello,

	As I suspected, it was SMTP transport agent problem, but to
	be able to debug it, I had to make a set of spool-files for
	testing the phenomena, and then place debugger into the smtp
	transporter while feeding jobs to it manually.

	Problem turned out to be in main(), where there is a loop
	starting with  printf("#hungry\n"), continuing with ctlopen(),
	and then if the target host has changed (or was not opened
	before) closure of old connection, and opening of a new one.

	The handling continues with process(), which takes  smtpstatus
	as one of its inputs, but in case of sending to same host, the
	re-open does not happen, and the variable contains status of
	the PREVIOUS message... And because it was failure to begin
	with, process() thinks it failed too, and ALL messages after
	the one that failed, became failures.

	Now the status variable is reset before the optional connection
	open is done, and thus has sensible value in connection optimization
	case, and won't cause process() to puke on delivery attempt...

	It was a hairy one.

	I have also been tinkering with  "MIME-2", that is, mechanisms
	to add translations of 8-bit chars in email headers to "MIME-2"
	encoded words.

	Because the system has no knowledge of what charsets user is using,
	I decided to create coded words of style:

		=?UNKNOWN-8BIT?Q?=E4=E4kk=F6=stesti?=

	(That is computer slang word in Finnish, when coded in ISO-8859-1,
	 but because the transport is not in direct contact with user,
	 this "UNKNOWN-8BIT" is the best approximation of a charset..
	 I am trying to think a way to make guesses in LOCAL environment,
	 by storing the value of LC_CTYPE variable, but the string in there
	 appears to vary in between systems :-/ )

	This stuff is now on SMTP transporter, because there are servers
	out there that reject the messages right away, when there are 8-bit
	characters in headers (like in personal names!), like this:

		From:  Manu =?ISO-8859-1?Q?M=E4h=F6nen?=
			<Listserv.Maintenance@fiport.funet.fi>

	In fact this is an alteration of a scheme I made for SMTP-receiver,
	where I decided that rejecting a message because it has 8-bit chars
	in headers is a bit drastic, and now the smtpserver does this
	"coded words" conversion on input.  (Certainly not as good one as
	it should be..)


	YOUR OPINION SOUGHT:
		Right now that "MIME-2" processing is involuntary,
	        that is, there is NO option to disable it.

		Should there be something, and if yes, what ?

		A thing that is likely to survive is to send "Subject:"
		in plain 8-bit, but if we feed personal names in coded
		words, and receiver handles them, is there any need ?

		Also, If we can get something like LC_CTYPE from user
		via  /usr/lib/sendmail  into mailer's knowledge, how
		should the knowledge be used ?  I would get from there
		instead of "UNKNOWN-8BIT", either "iso_8859_1" (Linux, Sunos),
		or "fi_FI.ISO8859-1" (OSF/1), neither of which are valid
		charsets in the sense of MIME specs..

	/Matti Aarnio <mea@nic.funet.fi>

PS: USA-NORDUnet 34 Mbps link should go via PTAT-1, but that cable
    has now TWO faults in it, and two cable-ships repairing it.
    Backup routing via satellite is also down, and we are down to
    "mere" 2x H12 connection (H12 is a "hyper channel" set of 31
    out of 32 timeslots on 2Mbps PCM link)

PS2: The new scheduler evolves radically, the source you see in
     2.99.16 is fading away rapidly, as the new thread scheduling
     system builds up, and eats away globs of old code :-)