Re: SMTP file naming issue?

[Matti Aarnio <mea@nic.funet.fi>]

On Sun, Jun 04, 2006 at 11:23:11PM -0700, Neal Morgan wrote:
> After some time looking through the code, I see my previous questions
> were dumb.  
> 
> So, let me try again:
> 
> Is there a way to configure router/transport(/scheduler) so that:
> 
> 1) messages received by smtpserver are "routed" to an alternate
>    directory

The answer here is "INPUTDIRS=" ZENV variable.

See: http://zmailer.org/man/zmailer.conf.5zm.html

> 2) metadata about the message is maintained

Message metadata consists of two things:
  - Envelope data inside the spoolfile
  - File i-node number and mtime forming up a spool-id

With very rapid message lifetimes it is possible, that two messages do get
same spool-id.  First one does get completely processed before second one
has even been created...

Additional unique data source would be nice - but what that would be ?
Light-weight, secure, persistent ?


> 3) an external process operates on the message, then
> 4) the external processes "resubmits" or "renables" the message, router
> and transport pick back up with the SAME message information (but
> altered message contents) and continue where they left off with routing
> and delivery?

In practice by preservation of i-node number and mtime of the file
is all you need for claiming the file to have same spool-id.

E.g. Write processing result at first to a temp file named  .#####
(see how  libc/mail.c names temporary files.)  

Then copy the result back to the original spool file, truncate it
if the new one is smaller than the original, and  utimes(2) the
st_mtime to match original stamp  ->  you get same spoolid.

Then move the spool file to ROUTER= subdirs, and possibly use notification
mechanism to announce existence of the file to the router queue manager.

> It seems like this should be possible with router and sm? I just can't
> figure out the correct configuration to make ZMailer view this as the
> same message, rather than a new one.

Having same i-node and same st_mtime timestamp will keep the spool-id
the same.   It is entirely possible to have spool-file name differ from
decimal representation of i-node number, but then you have to find some
other way to generate guaranteed unique filenames - this is simple in
e.g. sendmail who keeps all spool-files in same directory, but not so
in ZMailer, which does spread files all over the place.


       /Matti Aarnio


> -----Original Message-----
> From: Carlos Mendioroz [mailto:tron@huapi.ba.ar] 
> Sent: Saturday, June 03, 2006 4:29 PM
> To: Neal Morgan
> Cc: zmailer@nic.funet.fi
> Subject: Re: SMTP file naming issue?
> 
> I would use message-id, I guess, but there may be issues with that.
> Sorry, I'm not able to help you more...
> 
> 
> Neal Morgan  @ 3/6/2006 17:36 -0400 dixit:
> > Thanks.  I don't see any indication of a problem with the messages
> being
> > handled properly.  I agree Zmailer is working fine.
> > 
> > My issue is visibility.  I don't see an easy way from my logs to match
> > messages leaving SMTP with those traversing external processes, and
> back
> > when re-entering the router.
> > 
> > Maybe I should ask my question differently.  Is there an easy way to
> > include the spoolID in the envelope (on the comment line) or append it
> > to the message header?
> > 
> > Alternatively, how difficult would it be to use spoolID as the file
> > name?  ...would that cause problems elsewhere?
> > 
> > I'm guessing the envelope approach would be easiest.  Suggestions?
> > 
> > 
> > 
> > 
> > -----Original Message-----
> > From: zmailer-owner@nic.funet.fi [mailto:zmailer-owner@nic.funet.fi]
> On
> > Behalf Of Carlos Mendioroz
> > Sent: Saturday, June 03, 2006 4:40 AM
> > To: Neal Morgan
> > Cc: zmailer@nic.funet.fi
> > Subject: Re: SMTP file naming issue?
> > 
> > If the naming scheme is the old one, name comes from inode number.
> > So if the name is reused, you should count on the old one having been
> > taken care of.
> > -Carlos
> > 
> > Neal Morgan  @ 3/6/2006 2:58 -0400 dixit:
> >> Hello Matti!
> >>
> >> I've been watching the files dropped into postoffice/router by
> > incoming
> >> SMTP sessions and I note the same file name is used several times.
> > I'm
> >> wondering if this is configurable, or a bug...  I have an external
> >> process that acts on the messages before the router picks them up for
> >> delivery.  I'd like to be able to track "chain of custody" per
> message
> >> using this file name, but am unable to do so if SMTP reuses the same
> >> name.
> >>
> >>
> >> Regards,
> >>
> >> Neal Morgan
> >>
-- 
/Matti Aarnio	<mea@nic.funet.fi>
FUNET:  Finnish Academic and Research Network
	Network Information/Software Archival Service
-
To unsubscribe from this list: send the line "unsubscribe zmailer" in
the body of a message to majordomo@nic.funet.fi