Modeling human performance in statistical word segmentation

The ability to discover groupings in continuous stimuli on the basis of distributional information is present across species and across perceptual modalities. We investigate the nature of the computations underlying this ability using statistical word segmentation experiments in which we vary the le...

Full description

Bibliographic Details
Main Authors: Frank, Michael C., Goldwater, Sharon, Griffiths, Thomas L., Tenenbaum, Joshua B.
Other Authors: Massachusetts Institute of Technology. Department of Brain and Cognitive Sciences
Format: Article
Language:en_US
Published: Elsevier 2016
Online Access:http://hdl.handle.net/1721.1/102506
https://orcid.org/0000-0002-1925-2035
Description
Summary:The ability to discover groupings in continuous stimuli on the basis of distributional information is present across species and across perceptual modalities. We investigate the nature of the computations underlying this ability using statistical word segmentation experiments in which we vary the length of sentences, the amount of exposure, and the number of words in the languages being learned. Although the results are intuitive from the perspective of a language learner (longer sentences, less training, and a larger language all make learning more difficult), standard computational proposals fail to capture several of these results. We describe how probabilistic models of segmentation can be modified to take into account some notion of memory or resource limitations in order to provide a closer match to human performance.