Scope and method of study. Security often comes with overhead that will impact link Quality of Service (QoS) performance. In this dissertation, we propose an on-demand security and QoS optimization architecture in mobile ad hoc networks that automatically adapts network security level to changes in network topology, traffic condition, and link QoS requirements, so as to keep the security and QoS at optimum conditions. In order to achieve the overall objective, we introduce three basic frameworks: a policy based plug-in security framework, a multi-layer QoS guided routing algorithm, and a Proportional Integral Derivative (PID) feedback control based security and QoS optimization framework. The research has been evaluated with the network simulator ns-2. Finally, we propose an attack tree and state machine based security evaluation mechanism for ad hoc networks: a new security measurement metric.

Findings and conclusions. Simulations have been done for small and large network sizes, low and high communication ratios, as well as low and high mobility scenarios. The simulations show that the proposed on-demand security and QoS optimization architecture can produce similar performance to non-secure QoS routing protocol under various traffic loads. It provides more secure ad hoc networks without compromising the QoS performance, especially under light and medium traffic conditions.