[Raw Msg Headers][Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Priority queues

On Fri, Feb 27, 2004 at 08:04:00AM -0600, Dave Rogers wrote:
> We have a need for priority based message queues. The way I do that now
> is to inject high priority messages into a completely separate
> postoffice which is handled by another Zmailer instance on a second
> server. What I would like to do is find some way to handle at least two
> different message priorities (normal and high priority) on a single
> instance of Zmailer. High priority messages would go out ahead of any
> normal priority messages in the queue. I have searched the docs and
> config files but I cannot find a way to do this. Does anyone have a
> suggestion as to how I might accomplish this?

I have been busy elsewere (obviously..)

There is a sort of way to achieve this.

* You will need multiple 'ROUTERDIRS=' parallel router directories
  in $POSTOFFICE/ hierarchy
* Define e.g.  ROUTERDIRS=router2:router3   in  zmailer.conf  ZENV
  definition file  (and do make them in $POSTOFFICE too!)
* Run e.g.   /usr/lib/sendmail   with  environment variable
  MAILPRIORITY,  observe:

Presuming you have two additional router spools:

Posting with environment value  MAILPRIORITY=0  will put message
into the   $POSTOFFICE/router/  directory.

Posting with environment value  MAILPRIORITY=1  will put message
into the   $POSTOFFICE/router2/  directory.

Posting with environment value  MAILPRIORITY=2  will put message
into the   $POSTOFFICE/router3/  directory.

Posting with environment value  MAILPRIORITY=3  will put message
into the   $POSTOFFICE/router3/  directory.  (it is last directory,
so larger values will stop at it the subset selection.)

These mechanisms have existed for quite a while.  At least since
mid  1990es, I think.

Now steps in very recent code behaviour (fall 2003):

- Primary router directory gets predefined number of parallel
  processing routers  e.g.: ROUTEROPTIONS="-dk -n 4" starts
  there four parallel routers
- Each additional router directory gets _one_ router process.

Queues are processed in _parallel_, primary getting possibly
more processors than others.

Previously additional queues were processed only, when primary
queue was empty.  There was no guarantee that far end ROUTERDIRS
directory content got ever processed, if higher priority queues
got enough jobs.

Processing priorization fairness was simpler to make that way,
than constructing some complicated scheduling algorithm to 
manage spool of processing processes, and tasks fed to them.

Each router process(es) are  chdir()ed  into their respective
router directory.   Previously processes were chdir()ed just
into the primary directory, and jobs selected for work were
moved from their location into primary router directory root
for processing. 

Now there can also be hash subdirs ('A' thru 'Z') in these router
dirs, and with ROUTERDIRHASH=1  ZENV variable the incoming messages
end up in those (which allows around 26 times larger queue to fit
in the router, than without it)   Current code (and process.cf)
has smarts to understand this, and not to need moving files around
before processing them.  (Less directory ops -> faster system.)

> TIA,
> dave
> --
> Dave Rogers
> Plano, Texas
/Matti Aarnio	<mea@nic.funet.fi>
To unsubscribe from this list: send the line "unsubscribe zmailer" in
the body of a message to majordomo@nic.funet.fi