Multi-utterance Language Production

For this project we use complex, real-world scenes to examine the linearization of complex thoughts into sequenced utterances. This work, in collaboration with Dr. John Henderson, uses precisely quantified image and semantic characteristics of complex scenes to generate predictions about the allocation of attention as a scene is viewed and described. The project examines the conditions under which scene image features exogenously draw the eyes to specific visual areas which the linguistic system then describes, and under what conditions the cognitive system guides the eyes to meaningful regions of the scene, allowing the language system to prepare a description even before the relevant object or region has been fixated. Our model assumes that the language and cognitive systems interactively use scene meaning  to formulate a linearization plan for coordinating the production of multiple utterances.

Neural Correlates of Syntax

Work in collaboration with Dr. John Henderson’s lab has examined how patterns of brain activation are modulated by syntactic surprisal — a computational linguistic measure estimating how syntactically unexpected each word in a sentence is, given what has come before it. This work employs concurrent recording of eye movements and functional MRI to assess changes in brain activation during natural reading.

Processing of Disfluent Speech

Everyday speech is full of um’s, uh’s, and other types of disfluencies, yet surprisingly little is known about how disfluencies are processed during language comprehension.  Some of our work has shown that the presence of um‘s and uh‘s can affect how listeners parse a sentence.  We are also interested in repair disfluencies (e.g., “Turn left, uh I mean, right at the next light”) — particularly in understanding the extent to which the reparandum (e.g., “left”) continues to influence processing even after it is replaced by the repair (e.g., “right”), as well as whether listeners actively predict the upcoming repair before it is spoken.

Processing of Multiword Sequences

A surprisingly large percentage of the language produced by native speakers contains formulaic patterns, such as recurring sequences of words (known as multiword sequences). Past research has shown that our knowledge of formulaic language supports fast and efficient communication. In this project, we focus on the processing of recurrent, familiar three-word phrases known as binomial phrases (e.g., salt and pepper, bride and groom), and examine the extent to which these phrases are fixed expressions.

Processing of Nested Epistemic Expressions

Epistemic modals, such as “certainly”, “probable”, and “may”, indicate speakers’ commitment to the truth value of what is said, and serve as an important means to modify the strength of an argument. In daily conversation, it is not uncommon to find the use of more than one epistemic modal in a single clause, such as “he may certainly have forgotten”. Semantic theories tend to analyze the structure of such nested epistemic expression as having the inner epistemic modal embedded within the scope of the outer epistemic modal. In this way, the sentence “he may certainly have forgotten” means something different from “he certainly may have forgotten”. However, since research on the processing of nested epistemic expressions is so limited,  it remains a question whether or not in everyday situations, interlocutors process the meaning of nested epistemic expressions according to the linguistic representations that are assumed to underlie the forms. This project explores the semantic processing of nested epistemic expressions, focusing on processor’s sensitivity to the compositionality of the nested epistemic expression and the order of embedding.

Prosody and Information Structure

Language comprehension involves the task of identifying the most important or informative bits of information within a discourse. The information structure of an utterance (i.e., which parts are new and which are given or presupposed) can be marked through prosody (I saw HIM!) as well as syntactic focus constructions (It was him that I saw!). We use eye-tracking and ERPs to assess how different ways to mark information structure guide comprehenders’ expectations and influence language processing.