In this dissertation we will present a stochastic optimization algorithm and use it and other mathematical techniques to tackle problems arising in medicinal chemistry. In Chapter 1, we present some background about stochastic optimization and the Accelerated Random Search (ARS) algorithm. We then present a novel improvement of the ARS algorithm, Directed Accelerated Random Search (DARS), motivated by some theoretical results, and demonstrate through numerical results that it improves upon ARS. In Chapter 2, we use DARS and other methods to address issues arising from the use of mixture-based combinatorial libraries in drug discovery. In particular, we look at models associated with the biological activity of these mixtures and use them to answer questions about sensitivity and robustness, and also present a novel method for determining the integrity of the synthesis. Finally, in Chapter 3 we present an in-depth analysis of some statistical and mathematical techniques in combinatorial chemistry, including a novel probabilistic approach to using structural similarity to predict the activity landscape.