Creating autonomous long-lived agent that can robustly function in a complex object-based environment has been a persistent goal in the field of artificial intelligence. Learning the appropriate functional categories of objects is one of the keys to achieve this goal, and is the theme of this thesis.

We formulate the research problem as finding efficient value function approximation algorithms, where the input to the function is an object-based state representation, and output of the function is the utility value of that input state. The challenges arise from the requirements of efficient learning, and incremental learning of complex nonlinear value functions, whose input consists of diverse objects in high dimensional feature space. Our solutions are based on three key principles. The first is that the value function representation can usefully exploit the compositional structure of object-based environments, where the state representations consist of independent objects with their own perceptual features and functional properties. The second is that hierarchical symbolic category representations, inspired by human cognitive models, can help achieve efficient learning. The third is that the object categorization criteria must be consistent and coherent with the target utility value function. We provide two implementations based on these key principles, with evaluations both based on functionality and on cognitive plausibility.

Traditionally, category learning and value function approximation are studied as separate problems. The thesis presents a unique synthesis of the two. On one hand, it provides efficient value function approximation algorithms that can take advantage of compact representational basis adaptively generated by hierarchical category learning. On the other hand, it provides a utility based category learning model that offers new computational insights to human category learning behaviors.