Duplicate removal - solution

[Swen Thuemmler <swen@uni-paderborn.de>]

Greetings,

since we had a large number of complaints from our users about duplicate
mails, I sat down and tried do remove them. Below is the solution I came
up with. It is a patch against standard.cf. I first tried to generate a
new list without the duplicates with something like
	newres=$(list $(elements $newres) $temp)
but this let the router allocate insane amounts of memory - old values are
probably not freed when assigning a new value to a variable.
Unfortunately, I found no way to remove an entry from a list, so decided
to replace duplicates with (prog duplicate '"/dev/null"' something).
We need the prog channel, since we do local delivery with procmail, which
does not grok pipes and filenames. If you use mailbox, you may replace
prog with local.

Perhaps some of you find this useful. If there is anything wrong with this
aproach, please tell me.

Greetings, Swen


--- standard.cf.orig	Mon Nov 11 16:30:11 1996
+++ standard.cf	Tue Dec  3 13:38:32 1996
@@ -173,13 +173,16 @@
 #| value of the type property indicates whether or not alias expansion
 #| should be done.  If it is 'sender', no alias expansion is done.
 
+# duplicate removal
+relation -bt incore duplicates
+
 
 router (address, attributes) {
 
 #| This is the address routing entry point.  The function name "router"
 #| is known by the router program.
 
-	local res
+	local res i temp channel user host didexpand
 	# set up default attributes for casual interactive use.  Any
 	# privilege set here is only important when router is run as root,
 	# otherwise it won't matter.
@@ -202,6 +205,7 @@
 	# function instead of here.
 
 	db flush expansions
+	db flush duplicates
 
 	res=$(rrouter "$address" "$address" $attributes)
 
@@ -212,6 +216,22 @@
 		log deferred: $defer: "$address"
 		return (((hold "$defer" "$address" $attributes)))
 	fi
+
+#| remove duplicate addresses
+#|
+	for i in $(elements $res); do
+		temp=$(car $i)
+		channel=$(channel $temp)
+		user=$(user $temp)
+		host=$(host $temp)
+		key="${channel}_${user}_${host}"
+		didexpand=$(duplicates "$key")
+		if [ "x$didexpand" = "x" ]; then
+			db add duplicates "$key" 0
+		else
+			$(setf $(car $i) (prog duplicate '"/dev/null"' $(attributes $temp)))
+		fi
+	done
 
 #| If an internal problem was detected during the routing, we ignore the
 #| results of routing and redirect this address to the hold transport