Re: cluster-ready gauges?

[Matti Aarnio <mea@nic.funet.fi>]

On Tue, May 27, 2003 at 11:10:01AM +0400, Eugene Crosser wrote:
> Matti,
> 
> speaking about gauges and counters, do you think it makes sense to make
> the "counter-keeper" cluster(/farm) ready from the beginning?  Aside
> from the fact that it's nice to have aggregated statistics information,
> it could be used for such an important thing as keeping
> "connections-from-same-ip" counter which is rather important to limit
> incoming spam flow.

  Present counters are purely local memory thing, and don't contain
  e.g. smtpserver's "connections from same IP" analyzers.

  I did add a "mailq -QQQQ" (e.g. MAIL-v2 protocol "SHOW COUNTERS")
  over the weekend.  It sure makes asking for data from remote system
  somewhat easier, but requires, of course, that the scheduler is
  up and about to serve it.

  How to do "cluster wide" tracking, and have some change of the protocol
  to keep coherency ?

    - Presumption here is that a "cluster" is a set of machines in
	same Layer-2 multicast area (otherwise you need Layer-3
	multicast over routers...)
    - Nodes ( = smtpserver master processes ) join some configured
	(IPv4) multicast group
    - Protocol encodes everything in network byte order
    - Protocol contains some node-id (source IP address ?), 
	set transmit id (32 bit continuously growing counter ?),
	number of packets in that transmit, and packet number
	of that particular packet.  Plus a checksum of each
	packet (nobody trusts UDP, right ?)
	Oh!  And start each packet with some protocol-id magic,
	e.g. "ZM01"   (Is there need for authenticators ?
	A shared-secret MD5 checksum of each packet -- does
	well that packet checksumming...)
    - Protocol primary payload is an array of tuples:  IP-address, count
    - They send their parallel connect datasets in that group
	every time there is some change, and at least every 14
	seconds, or so.
    - Each node collects the data (discarding their own, if they hear it)
	and new data replaces old from same node-id (but only
	when received complete)
    - Data that is more than 60 seconds old from its reception into
	a node is considered stale, and discarded, time is node
	internal, not encoded in protocol.

  That kind of thing is fairly simple to code.   If a machine does not
  support multicast (all sensible modern machines you are likely to
  use in clusters do), perhaps a peer-to-peer UDP unicasts would
  make sense.  Configuring them will be trouble, though.  Some sort
  of "PARAM cluster-partners-xyz  IP IP IP IP IP ..."
  (On the other hand, ETRN-cluster already has this kind of peer
   parametrization mechanism)

  To what an extent it would make sense to broadcast also the ZMailer
  instance wide SHM segment ?  Separate broadcaster/aggregator server ?
  (not integrated into any other master process.)  Such a beast could
  also be SNMP responder...  What a pickle..    (Counters should never
  go backwards..)   SMUX, AgentX,  net-snmp dynamic module, embedded
  perl in net-snmp ?   Furthermore, how to arrange the object hierarchy,
  when there can be multiple instances of ZMailer in one machine, and
  also multiple machines worth of instances being published by a single 
  SNMP proxy ?  (E.g. that cluster thingie...)

  Instance-ID I use is essentially the path of  POSTOFFICE.
  Presumption being, that each ZMailer instance has its own POSTOFFICE
  directory tree.

...
> We might have to incorporate "sitewide connection counter" into SpIn but
> it won't be neccessary if it'd be a native Zmailer feature...

  Care (time?) to try your hand at this multicast client thing coding ?
  I would localize it into  smtpserver/smtpchild.c,  with perhaps one
  or two hooks to be called from the server main loop (e.g. knowing
  socket-fd, and tracking timeout.)  

  I might do it too, but can't give schedule.   Nor have I a cluster of
  machines in which to test it.  (That is mostly arrangement question,
  of course.  Even my laptop could be used as a cluster member...)

> -- 
> Eugene Crosser, head of Internet Applications section, +7 501 787 1000
> ROL, EDN Sovintel, Golden Telecom, http://user.rol.ru/~crosser/
-- 
/Matti Aarnio	<mea@nic.funet.fi>
-
To unsubscribe from this list: send the line "unsubscribe zmailer" in
the body of a message to majordomo@nic.funet.fi