The brains of mammals are very efficient learning machines. Many aspects of mammalian learning are yet to be incorporated into machine learning algorithms. For instance, vision is typically considered to be a spatial problem in which a learning system needs to be trained with labeled examples of object images. Yet, mammals learn with continuously flowing unlabeled data. It is also generally accepted that the visual cortex in mammals is organized as a hierarchy and that many aspects of visual perception can be modeled using Bayesian computations.

This dissertation introduces algorithms and networks that combine hierarchical and temporal learning with Bayesian inference for pattern recognition. These algorithms and networks, collectively called Hierarchical Temporal Memory (HTM), can be used to learn hierarchical-temporal models of data. Temporal continuity is used to learn multiple levels of the hierarchy without supervision. The HTM algorithms, when applied to a visual pattern recognition problem, exhibit invariant recognition, robustness to noise, and generalization. Inference in the hierarchy is performed using Bayesian belief propagation equations that are adapted to this problem setting.

In order to understand the generalization properties of HTMs, a generative model for HTMs is developed. This model enables the generation of synthetic data from HTM networks. These data are used to analyze and characterize learning and generalization in hierarchical-temporal systems. Two existing hierarchical pattern recognition models are mapped to HTMs to explain the source of generalization in those models.

Finally, the HTM Bayesian belief propagation equations are used to suggest a mathematical model for cortical microcircuits. The microcircuit model is derived by combining known anatomical constraints with the computational specifications of HTM belief propagation. The proposed model has a laminar and columnar organization that matches many known anatomical features. The proposed circuits are then used in the modeling of two well known physiological phenomena.