| Summary: | <p>Sonification is a complementary technique to visualization that uses sound to describe data. Kramer defines sonification as "the use of nonspeech audio to convey information. More specifically, sonification is the transformation of data relations into perceived relations in an acoustic signal for the purposes of facilitating communication or interpretation." While providing new opportunities for communicating through the human perceptual and cognitive apparatus, sonification poses challenges with presenting the exploratory patterns in data to the user as it is a less familiar medium for this purpose.</p> <p>We describe work to sonify variants of Hamlet to aid exploratory textual analysis. The sonification presented focuses on using pitch and tones to help the user listen to differences in the structure between variations of a text or texts encoded in Text Encoding Initiative (TEI) XML. Our approach is inspired by the Hinman Collator, an opto-mechanical device originally used to highlight print variants in Shakespeare texts, whereby visual differences between two texts literally stood out through a stereoscopic effect. Using an audio stream for each text, this project aims to produce a stereo audio image of the text, so creating an audio version of the stereoscopic illusion used in collating machines. The timing and frequencies are extracted for storage and transformation into alternate formats or to repeat the analysis.</p> <p>We present initial work on XML variants of Shakespeare's Hamlet using the Bodleian Libraries' First Folio XML and their earlier work on the Quartos. We extracted document entities such as act, scenes, lines, and stage directions for the analysis. These are viewed as hyperstructures that may be separated from the text for sonification and comparison with other variants. Analytical perceptions can be altered through the presentation of the tones, pitches and icons. Audio displays demand the creator to rethink how structural data is presented to the user, and about the hyperstructures extracted to give potential for conversion of the analysis into hypermedia using visualization as well as sonification. Early results show promise for the auditory comparison.</p> <p>We look at related work and present the case study. We then consider the use of audio beacons to help the user locate within the document, and discuss the integration with visualization. Finally we look at future work and conclude the paper.</p>
|
|---|