xmlsysd (v 1.0.4, i386 built for FC4) is a daemon that runs on a workstation or workstation-style cluster node in a beowulf or compute cluster or departmental LAN. It provides a socket connection to an (unprivileged) engine that parses various parts of /proc and executes certain system information calls on behalf of the remote user without the expense of a shell connection or multiple tools or pipes. This information is wrapped up in a customized xmlish and returned via socket to the client.
The daemon has several features that make it especially lightweight on the client. It can be "throttled", so that it only returns information that the user specifically wants to monitor. It opens once and then uses a rewind on all the /proc files that it can to avoid excessive fstat and file open overhead. In most cases all the information of interest to a user can be transmitted in a single TCP packet and hence at a fixed, minimal "cost" in terms of systems resources like CPU and network bandwidth.
xmlsysd can currently monitor load average, memory utilization, network utilization, a variety of systems parameters of interest in constructing a load-balanced distribution of jobs (e.g. CPU type, clock, cache), date and time, "duty cycle" (fraction of CPU clocks actually used since the last reboot) and running jobs, masked by task name or user.
The associated wulfstat application provides a simple tty (xterm) interface to the daemon, permitting an entire LAN's worth of workstations or beowulf nodes to be monitored from virtually anywhere. wulflogger is a related application built from the same library as wulfstat that dumps wulfstat-like displays to stdout, from where they can easily be parsed by e.g. a perl or python application for further display. wulfweb is such a demo application -- wulfweb uses wulflogger to monitor a cluster and create an automatically updated web page with (currently only load average) data colorfully and efficiently displayed for the whole cluster.
The xmlish language xmlsysd uses to return information should make it easy to construct e.g. a web-based display or a native GUI, should either of these be preferred. Eventually I'll tackle these tasks using the libwulf library provided with wulfstat or wulflogger, but for now the tools above are pretty useful as they stand.
SPECIAL NOTE: From 0.6.0 on, xmlsysd will not distribute binary rpms as there is a distinct split in architecture upon us on top of the already wide split in distribution. Instead it will be distributed in tarball format and in hopefully easily rebuildable source rpms. Binary rpm distribution will resume only in the form of a yum-accessible repository, per architecture, at least for architectures I can locally build. Since xmlsysd is appearing in more regular linux distributions, one can hope that this won't be much of a problem.
Ideas and patches have been contributed by Morten Kjeldgaard, Andy Fant, and Erwan Velu, and rgb gratefully acknowledges the help.