The increasing ubiquity of complex agent organizations has led to an
increasing need for the on-line monitoring of such organizations.
However, in many domains (e.g., those with information agents), we
cannot observe the agents' actions. In addition, we rarely have the
ability to change the agents themselves to force them to communicate
their state to us. Fortunately, we can often eavesdrop on messages communicated
by the agents as part of their natural coordination. This paper presents
an approach for on-line monitoring of organizations using messages as observations.
This approach includes the following key novel ideas: (i) a linear time
probabilistic plan-recognition algorithm, particularly well-suited for
processing communications in agent organizations; (ii) a technique for
modeling the agent organization as a single coherent entity--trading expressivity
for scalability; and (iii) an approach to exploiting general knowledge
of teamwork to predict organizational responses during normal and failing
execution, to reduce monitoring uncertainty. We present an empirical evaluation
of these ideas in the context of monitoring a complex, multi-agent information
system.