new-scheduler's "-l"-log

[Matti Aarnio <mea@nic.funet.fi>]

I have been tinkering with the performance tracer log, and during
the last 24 hours (roughly), the nic.funet.fi has processed some
30 000 recipients, and if I analyzed them correctly, there were
"only" 5000 messages sent -- thus having on average 6 recipients
per letter..

Anyway, here is a snap-shot of scheduler performance log showing
speeds of the system -- I am still a bit amazed of it:

812931252 90354-2 0 4 ok local/raussi@nic.funet.fi
812931252 90780-2 0 7 ok smtp/finhutc.hut.fi
812931372 90780-4 0 4 ok smtp/finhutc.hut.fi
812931388 90780-4 0 9 ok smtp/kotanet.fi
812931462 90780-2 0 5 ok smtp/finhutc.hut.fi
812931493 90780-2 0 15 ok smtp/sdf.com
812931643 90354-3 0 3 ok smtp/finhutc.hut.fi
812931642 90780-3 0 5 ok smtp/finhutc.hut.fi
812931703 90780-3 0 5 ok smtp/listserv.funet.fi

These fields are explained in the ChangeLog, but for a summary:

	- spoolfile ctime
	- spoolfile filename (after processing)
	- Delta-t from spool-file creation to transport
	  descriptor creation
	- Delta-T from transport-descriptor creation to
	  successfull recipient address processing
	- status (ok/error/expired)
	- $channel/$host

They show amazingly fast performance on routers (usually 0 seconds!),
which I frankly doubt, but the other time difference is from
transport-descriptor creation to status report writeup, and I
have seen cases where I can hardly blink my eyes before my reply
to somebody has its Cc:-copy arriving to my ELM..
In such a case the log shows 1 (one!) second used for the delivery..

Talk about speed :)  The norm appears to be in between 10-20
seconds, of which roughly 10 seconds comes from the directory
scanning interval (every 20 seconds), and the rest few secs
come from the work that the transport agent needs to do.

I am running this now in production mode, but it has some small
changes which didn't get into 2.99.18. (of cosmetic nature, not
related to thread scheduling algorithms.)

There is largeish open issue related to a good way to handle
SMTP-connection cache, that is, how to pick right TA from
idle pool -- now we pick the first one from idle chain.

	/Matti Aarnio