To control multi-agent systems one can exploit the recurrence properties of stochastic processes. We illustrate this principle through two applications. In both applications, systems are modeled as hybrid systems where Markov transitions on the discrete variables depend on the continuous variables.

In the first application, stochasticity is introduced to overcome uncertainty about the environment. Inspired by bacterial chemotaxis, we design algorithms that control the spatial distribution of mobile agents that can take point measurements of a monitored function but cannot measure their own positions. Applications include source-seeking, monitoring and deployment. We prove that the probability density of agents is led to converge exponentially to a predetermined function of the measurements, much like in Markov Chain Monte Carlo methods. In the process of designing these control algorithms, we prove results on piecewise deterministic Markov processes that can find application outside this particular design problem.

In the second application, we control the level of stochasticity in a networked control system. Given that the probability of successful communication can be significantly increased by transmitting multiple copies of the same message, we show that, by dynamically assigning the number of transmitted copies of the same data, one can obtain significant performance gains with only a modest increase in the total number of transmissions. We develop techniques to design communication protocols that exploit the transmission of multiple packets, while seeking a balance between stability/estimation performance and communication rate. An average cost optimality criterion is employed to obtain a number of optimal protocols applicable to networks with different computational capabilities. Other capacity scheduling techniques are also explored.