UC San Diego

In this dissertation I address how we coordinate perceptual (visual) and linguistic processing to perform common tasks like speak about our environment or read a text. This is important because perception provides the input the linguistic system requires to activate relevant internal representations. Using eye tracking and gaze- contingent display change paradigms I assessed preview benefit--facilitated processing of a target when an item previously in its location (the preview) was related compared to unrelated. Preview benefit indexes the success of visual-linguistic coordination, indicating that one had (1) obtained information from an item before fixating it and (2) used that information to speed processing, upon fixation. In Studies 1 (Schotter, Jia, Ferreira & Rayner, under review) and 2 (Schotter, Ferreira & Rayner, 2013), a target object was revealed when the speaker fixated it; before, it was masked and then a preview object (representing the same or a different concept as the target) appeared briefly in its location. Processing of the target was unaffected by the timing of the preview or subjects' awareness of it (Study 1), suggesting that speakers access information from upcoming objects opportunistically (i.e., whenever the preview is available). Furthermore, preview benefit was provided by previews in to-be-named locations but not by previews in to-be-ignored locations (Study 2), suggesting that speakers do not access information from non-fixated objects indiscriminately. In Study 3 (Schotter, under revision), I investigated how the linguistic system uses this information to address a debate over whether semantic information is obtained from upcoming words. Research in German and Chinese has found semantic preview benefit but research in English has not. This may be due to the deep orthography of English delaying semantic access due to more effortful phonological decoding. Supporting this idea, semantic preview benefit occurred in English when the preview and target were synonyms but not when they were associatively related, possibly because associated words have looser connections in semantic networks than synonyms. Together, these studies imply that we achieve efficient reading and speaking via sophisticated (opportunistic but not indiscriminate) access of visual information in service of the linguistic system to activate appropriate mental representations