$POSTOFFICE/transports/XX/spoolfile

[Matti Aarnio <mea@nic.funet.fi>]

I have been tinkering with an idea of having an intermediate hash layer
at the  $POSTOFFICE/transports/  and at the $POSTOFFICE/queue/ -directories.
This became again motivated to me when I was monitoring vger.rutgers.edu
feeding 50 000 recipients from its queues -- and with 2500 files in same
directory.

As I have understood, directory accesses do become sluggish when there are
more than about 100 entries at the same directory at any common unix system.
I have now implemented a way to hash files into a set of subdirs.

It is not easy to measure the impact of this change, but assuming that each
directory entry causes a constant quanta of time needed for the directory
search, it is easy to see that cutting directory sizes down to 26th part
('A' .. 'Z' in case you wonder the number) should speed up an average
reference by same factor, altough subdirs do mean additional delay at
going thru them at first.

I have also found and fixed a couple other problems at the scheduler,
thanks to vger, which pushes unbelievable loads thru an undersize machine..


The whole subdirectory handling is in the care of the scheduler, the router
works as before, and only the  ctlopen()  function needed a small change
at the transport clients.  If the subdirs are in use (with having '-H',
or '-HH' as option to the scheduler -- two H's mean two-level hash),
the scheduler moves all new jobs into hash-function determined subdirs,
and then processes them there.   When the scheduler is started anew,
it looks up all subdirs, and keeps the hash values.

This evening I have been developeing, and testing this at nic.funet.fi,
tomorrow I can say something regarding vger.  Now good night folks.

/Matti Aarnio <mea@nic.funet.fi>