In a variety of computational domains, the number of samples available for learning remains relatively small as compared to increasing data dimensions. This is common in computational biology and vision, posing even greater challenges when combined with complex interactions among variables. Consequently, the bias-variance trade-off requires one to invest model parameters with the utmost care for robust learning. In the small sample context, we argue for incorporating the available prior knowledge, and introducing carefully chosen biases to reduce variance. Motivated by this philosophy and particular problems from biology and vision, we propose two new generative approaches within a graphical model formalism: (a) A comprehensive statistical model for analyzing cell signaling networks, and (b) A restricted family of latent variable forest models for discovery of complex dependencies.

The first method is particular to protein signaling networks, which play a central role in transcriptional regulation and the etiology of many diseases. With known molecular connections, our model is anchored to a pre-defined core signaling topology. It has a limited complexity due to parameter sharing and uses expression data of target genes as the only observable components. Specifically, we account for cell heterogeneity and a multi-level process, representing signaling as a Bayesian network at the cell level, modeling measurements as ensemble averages at the tissue level and incorporating patient-to-patient differences at the population level. We applied our method to the RAS-RAF network using a breast cancer study. We demonstrated robust statistical inference, established reproducibility through simulations and the ability to recover receptor status from available microarray data.

Our second method addresses the deeper endeavor of model selection. We propose a restricted family of forest structured distributions which are Markov with observed leaf variables regulated hierarchically by non-terminal latent variables. With a nested design, our model family allows a well-principled stepwise discovery of dependencies via sequential aggregations of pending substructures. Using particular parametric choices, we prove identifiability of our models and exact inference via dynamic programming. We apply our generative approach to synthesis and classification of handwritten digits, and to phenotype prediction from microarray data, with performances comparable to the state-of-the-art discriminative methods.