Where’s Wally: The influence of visual salience on referring expression generation
Referring expression generation (REG) presents the converse problem to visualsearch: Given a scene and a specified target, how does one generate adescription which would allow somebody else to quickly and accurately locatethe target? Previous work in psycholinguistics and natural language processin...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2013-06-01
|
| Series: | Frontiers in Psychology |
| Subjects: | |
| Online Access: | http://journal.frontiersin.org/Journal/10.3389/fpsyg.2013.00329/full |
| _version_ | 1818501603041214464 |
|---|---|
| author | Alasdair Daniel Francis Clarke Micha eElsner Hannah eRohde |
| author_facet | Alasdair Daniel Francis Clarke Micha eElsner Hannah eRohde |
| author_sort | Alasdair Daniel Francis Clarke |
| collection | DOAJ |
| description | Referring expression generation (REG) presents the converse problem to visualsearch: Given a scene and a specified target, how does one generate adescription which would allow somebody else to quickly and accurately locatethe target? Previous work in psycholinguistics and natural language processingthat has addressed this question identifies only a limited role for vision inthis task. That previous work, which relies largely on simple scenes, tends totreat vision as a pre-process for extracting feature categories that arerelevant to disambiguation. However, the visual search literature suggeststhat some descriptions are better than others at enabling listeners to searchefficiently within complex stimuli. This paper presents the results of a studytesting whether speakers are sensitive to visual features that allow them tocompose such `good' descriptions. Our results show that visual properties(salience, clutter, area, and distance) influence REG for targets embedded inimages from the *Where's Wally?* books, which are an order of magnitudemore complex than traditional stimuli. Referring expressions for large salienttargets are shorter than those for smaller and less salient targets, and targets within highly cluttered scenes are described using more words.We also find that speakers are more likely to mention non-target landmarks thatare large, salient, and in close proximity to the target. These findingsidentfy a key role for visual salience in language production decisions and highlight the importance of scene complexity for REG. |
| first_indexed | 2024-12-10T20:58:32Z |
| format | Article |
| id | doaj.art-3505a645462f4fc7894dfab9da57f831 |
| institution | Directory Open Access Journal |
| issn | 1664-1078 |
| language | English |
| last_indexed | 2024-12-10T20:58:32Z |
| publishDate | 2013-06-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Psychology |
| spelling | doaj.art-3505a645462f4fc7894dfab9da57f8312022-12-22T01:33:52ZengFrontiers Media S.A.Frontiers in Psychology1664-10782013-06-01410.3389/fpsyg.2013.0032949717Where’s Wally: The influence of visual salience on referring expression generationAlasdair Daniel Francis Clarke0Micha eElsner1Hannah eRohde2University of EdinburghThe Ohio State UniversityUniversity of EdinburghReferring expression generation (REG) presents the converse problem to visualsearch: Given a scene and a specified target, how does one generate adescription which would allow somebody else to quickly and accurately locatethe target? Previous work in psycholinguistics and natural language processingthat has addressed this question identifies only a limited role for vision inthis task. That previous work, which relies largely on simple scenes, tends totreat vision as a pre-process for extracting feature categories that arerelevant to disambiguation. However, the visual search literature suggeststhat some descriptions are better than others at enabling listeners to searchefficiently within complex stimuli. This paper presents the results of a studytesting whether speakers are sensitive to visual features that allow them tocompose such `good' descriptions. Our results show that visual properties(salience, clutter, area, and distance) influence REG for targets embedded inimages from the *Where's Wally?* books, which are an order of magnitudemore complex than traditional stimuli. Referring expressions for large salienttargets are shorter than those for smaller and less salient targets, and targets within highly cluttered scenes are described using more words.We also find that speakers are more likely to mention non-target landmarks thatare large, salient, and in close proximity to the target. These findingsidentfy a key role for visual salience in language production decisions and highlight the importance of scene complexity for REG.http://journal.frontiersin.org/Journal/10.3389/fpsyg.2013.00329/fullvisual searchvisual saliencyreferring expressionsvisual clutterreferring expression generation |
| spellingShingle | Alasdair Daniel Francis Clarke Micha eElsner Hannah eRohde Where’s Wally: The influence of visual salience on referring expression generation Frontiers in Psychology visual search visual saliency referring expressions visual clutter referring expression generation |
| title | Where’s Wally: The influence of visual salience on referring expression generation |
| title_full | Where’s Wally: The influence of visual salience on referring expression generation |
| title_fullStr | Where’s Wally: The influence of visual salience on referring expression generation |
| title_full_unstemmed | Where’s Wally: The influence of visual salience on referring expression generation |
| title_short | Where’s Wally: The influence of visual salience on referring expression generation |
| title_sort | where s wally the influence of visual salience on referring expression generation |
| topic | visual search visual saliency referring expressions visual clutter referring expression generation |
| url | http://journal.frontiersin.org/Journal/10.3389/fpsyg.2013.00329/full |
| work_keys_str_mv | AT alasdairdanielfrancisclarke whereswallytheinfluenceofvisualsalienceonreferringexpressiongeneration AT michaeelsner whereswallytheinfluenceofvisualsalienceonreferringexpressiongeneration AT hannaherohde whereswallytheinfluenceofvisualsalienceonreferringexpressiongeneration |