Re: Multiple Routers Processing A Single Message (patch enclosed)

[Andy Poling <andy@realbig.com>]

Matti-

As we have discussed recently, one can enforce single-threadedness of
renaming of the message files in $POSTOFFICE/router by using link() and
unlink() and checking the return from unlink().  Participants seemed to be of
the opinion that unlink() should always be an atomic operation, whereas
rename() is apparently not always atomic.

I've implemented this change after testing the theory using file locks for
several days.  The duplicate messages ceased.

The diff below changes lib/esyslib.c to never use rename(), but to use link()
and unlink() instead and check the outcome of the unlink().  If the unlink()
of "from" fails, then "to" is also unlink()ed and we return -1 to indicate
failure - either due to an error or because another router process link()ed
and unlink()ed the file before we could.  This should enforce single-threaded
renaming in the routers.

I submit the following diff (against v2.99.46) for your consideration.  I
have placed the resultant router into production use on several IRIX 5.3
systems.

-Andy

==============================================================================

*** esyslib.c.dist	Thu Dec 12 11:35:35 1996
--- esyslib.c	Wed Feb 12 22:27:36 1997
***************
*** 207,230 ****
  erename(from, to)
  	char *from, *to;
  {
! 	int	r;
! 
! #ifdef	HAVE_RENAME
! 	
! 	if ((r = rename(from, to)) < 0) {
! #else	/* !HAVE_RENAME */
! 	
! 	if (((r = unlink(to)) < 0 && errno != ENOENT)
! 	    || (r = link(from, to)) < 0) {
! #endif	/* HAVE_RENAME */
  		fprintf(stderr, "%s: rename(%s,%s): %s\n", progname,
  			from, to, strerror(errno));
  	}
! #ifndef	HAVE_RENAME
! 	else
! 		unlink(from);
! #endif	/* !HAVE_RENAME */
! 	return r;
  }
  
  int
--- 207,227 ----
  erename(from, to)
  	char *from, *to;
  {
! 	if (((unlink(to)) < 0 && errno != ENOENT)
! 	    || (link(from, to)) < 0) {
  		fprintf(stderr, "%s: rename(%s,%s): %s\n", progname,
  			from, to, strerror(errno));
+ 		return(-1);
+ 	}
+ 	if (unlink(from))
+ 	{
+ 		if (errno != ENOENT)
+ 			fprintf(stderr, "%s: unlink(%s): %s\n", progname,
+ 				to, strerror(errno));
+ 		unlink(to);
+ 		return(-1);
  	}
! 	return(0);
  }
  
  int
***************
*** 231,250 ****
  eqrename(from, to)
  	char *from, *to;
  {
! 	int	r;
! 
! #ifdef	HAVE_RENAME
! 	
! 	if ((r = rename(from, to)) < 0) {
! #else	/* !HAVE_RENAME */
! 	
! 	if (((r = unlink(to)) < 0 && errno != ENOENT)
! 	    || (r = link(from, to)) < 0) {
! #endif	/* HAVE_RENAME */
  	}
! #ifndef	HAVE_RENAME
! 	else
! 		unlink(from);
! #endif	/* !HAVE_RENAME */
! 	return r;
  }
--- 228,241 ----
  eqrename(from, to)
  	char *from, *to;
  {
! 	if (((unlink(to)) < 0 && errno != ENOENT)
! 	    || (link(from, to)) < 0) {
! 		return(-1);
! 	}
! 	if (unlink(from))
! 	{
! 		unlink(to);
! 		return(-1);
  	}
! 	return(0);
  }