Natural Signal Structure and its Role in Audiovisual Communication
by Chandrasekaran, Chandramouli, Ph.D., PRINCETON UNIVERSITY, 2011, 374 pages; 3459200

Abstract:

The efficient encoding and perception of communication signals is essential for survival and reproductive fitness in primates. As communication signals are often embedded in noisy acoustic habitats, primate brains are thought to combine visual and auditory components to boost their perception of communication signals. What are the mechanisms by which primate brains combine the visual and auditory components of communication signals? Through neuroimaging studies in humans and electrophysiological studies in passive monkeys, we have an excellent idea of the neural structures involved. However, a mechanistic basis is still not available. This dissertation attempts to derive a mechanistic basis by studying the statistical structure of natural communication signals and performing psychophysical experiments in humans and combined psychophysical and neurophysiological experiments in monkeys. I demonstrate that in audiovisual speech, visual components (mouth opening) are correlated with the auditory components (envelope, spectral frequencies). In addition, onset of mouth motion occurs up to 300 milliseconds before the sound. This means that visual cues can sculpt, bias and assist in the processing of subsequent auditory input. I show that visual cues speed up reaction times during audiovisual discrimination. Similarly, visual cues act as temporal markers for the auditory system and assist in the chunking of auditory input. In addition, this delay between visual and auditory cues modulates the degree of integration observed in the audiovisual local field potential (LFP) recorded in the superior temporal sulcus (STS), a key node implicated in the processing of audiovisual communication signals. I provide one of the first demonstrations that monkeys can also integrate auditory and visual cues from vocalizations—responding faster to audiovisual compared to auditory- and visual-only vocalizations. Perhaps mediating this behavior, neurons and LFP in auditory cortex of monkeys performing this task also respond faster to audiovisual compared to auditory vocalizations. Based on these results, I posit a simple mechanistic model of integration which suggests that activity due to visual and auditory cues is linearly combined in primate brains. This model is a simple but powerful mechanism for explaining behavioral reaction times in monkeys and humans as well as neurophysiological responses to audiovisual communication signals in monkeys.

 
AdviserAsif A. Ghazanfar
SchoolPRINCETON UNIVERSITY
SourceDAI/B 72-09, p. , Jul 2011
Source TypeDissertation
SubjectsNeurosciences; Behavioral sciences; Animal behavior
Publication Number3459200
Adobe PDF Access the complete dissertation:
 

» Find an electronic copy at your library.
  Use the link below to access a full citation record of this graduate work:
  http://gateway.proquest.com/openurl%3furl_ver=Z39.88-2004%26res_dat=xri:pqdiss%26rft_val_fmt=info:ofi/fmt:kev:mtx:dissertation%26rft_dat=xri:pqdiss:3459200
  If your library subscribes to the ProQuest Dissertations & Theses (PQDT) database, you may be entitled to a free electronic version of this graduate work. If not, you will have the option to purchase one, and access a 24 page preview for free (if available).

About ProQuest Dissertations & Theses
With over 2.3 million records, the ProQuest Dissertations & Theses (PQDT) database is the most comprehensive collection of dissertations and theses in the world. It is the database of record for graduate research.

The database includes citations of graduate works ranging from the first U.S. dissertation, accepted in 1861, to those accepted as recently as last semester. Of the 2.3 million graduate works included in the database, ProQuest offers more than 1.9 million in full text formats. Of those, over 860,000 are available in PDF format. More than 60,000 dissertations and theses are added to the database each year.

If you have questions, please feel free to visit the ProQuest Web site - http://www.proquest.com - or call ProQuest Hotline Customer Support at 1-800-521-3042.