Driving singing behaviour in songbirds using a multi-modal, multi-agent virtual environment
Abstract Interactive biorobotics provides unique experimental potential to study the mechanisms underlying social communication but is limited by our ability to build expressive robots that exhibit the complex behaviours of birds and small mammals. An alternative to physical robots is to use virtual...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2022-08-01
|
| Series: | Scientific Reports |
| Online Access: | https://doi.org/10.1038/s41598-022-16456-0 |
| _version_ | 1811315248928391168 |
|---|---|
| author | Leon Bonde Larsen Iris Adam Gordon J. Berman John Hallam Coen P. H. Elemans |
| author_facet | Leon Bonde Larsen Iris Adam Gordon J. Berman John Hallam Coen P. H. Elemans |
| author_sort | Leon Bonde Larsen |
| collection | DOAJ |
| description | Abstract Interactive biorobotics provides unique experimental potential to study the mechanisms underlying social communication but is limited by our ability to build expressive robots that exhibit the complex behaviours of birds and small mammals. An alternative to physical robots is to use virtual environments. Here, we designed and built a modular, audio-visual 2D virtual environment that allows multi-modal, multi-agent interaction to study mechanisms underlying social communication. The strength of the system is an implementation based on event processing that allows for complex computation. We tested this system in songbirds, which provide an exceptionally powerful and tractable model system to study social communication. We show that pair-bonded zebra finches (Taeniopygia guttata) communicating through the virtual environment exhibit normal call timing behaviour, males sing female directed song and both males and females display high-intensity courtship behaviours to their mates. These results suggest that the environment provided is sufficiently natural to elicit these behavioral responses. Furthermore, as an example of complex behavioral annotation, we developed a fully unsupervised song motif detector and used it to manipulate the virtual social environment of male zebra finches based on the number of motifs sung. Our virtual environment represents a first step in real-time automatic behaviour annotation and animal–computer interaction using higher level behaviours such as song. Our unsupervised acoustic analysis eliminates the need for annotated training data thus reducing labour investment and experimenter bias. |
| first_indexed | 2024-04-13T11:26:53Z |
| format | Article |
| id | doaj.art-d0b2a524d980448e9c358f76746b3488 |
| institution | Directory Open Access Journal |
| issn | 2045-2322 |
| language | English |
| last_indexed | 2024-04-13T11:26:53Z |
| publishDate | 2022-08-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Scientific Reports |
| spelling | doaj.art-d0b2a524d980448e9c358f76746b34882022-12-22T02:48:40ZengNature PortfolioScientific Reports2045-23222022-08-0112111210.1038/s41598-022-16456-0Driving singing behaviour in songbirds using a multi-modal, multi-agent virtual environmentLeon Bonde Larsen0Iris Adam1Gordon J. Berman2John Hallam3Coen P. H. Elemans4University of Southern Denmark, SDU-BioroboticsDepartment of Biology, University of Southern DenmarkDepartment of Biology, Emory UniversityUniversity of Southern Denmark, SDU-BioroboticsDepartment of Biology, University of Southern DenmarkAbstract Interactive biorobotics provides unique experimental potential to study the mechanisms underlying social communication but is limited by our ability to build expressive robots that exhibit the complex behaviours of birds and small mammals. An alternative to physical robots is to use virtual environments. Here, we designed and built a modular, audio-visual 2D virtual environment that allows multi-modal, multi-agent interaction to study mechanisms underlying social communication. The strength of the system is an implementation based on event processing that allows for complex computation. We tested this system in songbirds, which provide an exceptionally powerful and tractable model system to study social communication. We show that pair-bonded zebra finches (Taeniopygia guttata) communicating through the virtual environment exhibit normal call timing behaviour, males sing female directed song and both males and females display high-intensity courtship behaviours to their mates. These results suggest that the environment provided is sufficiently natural to elicit these behavioral responses. Furthermore, as an example of complex behavioral annotation, we developed a fully unsupervised song motif detector and used it to manipulate the virtual social environment of male zebra finches based on the number of motifs sung. Our virtual environment represents a first step in real-time automatic behaviour annotation and animal–computer interaction using higher level behaviours such as song. Our unsupervised acoustic analysis eliminates the need for annotated training data thus reducing labour investment and experimenter bias.https://doi.org/10.1038/s41598-022-16456-0 |
| spellingShingle | Leon Bonde Larsen Iris Adam Gordon J. Berman John Hallam Coen P. H. Elemans Driving singing behaviour in songbirds using a multi-modal, multi-agent virtual environment Scientific Reports |
| title | Driving singing behaviour in songbirds using a multi-modal, multi-agent virtual environment |
| title_full | Driving singing behaviour in songbirds using a multi-modal, multi-agent virtual environment |
| title_fullStr | Driving singing behaviour in songbirds using a multi-modal, multi-agent virtual environment |
| title_full_unstemmed | Driving singing behaviour in songbirds using a multi-modal, multi-agent virtual environment |
| title_short | Driving singing behaviour in songbirds using a multi-modal, multi-agent virtual environment |
| title_sort | driving singing behaviour in songbirds using a multi modal multi agent virtual environment |
| url | https://doi.org/10.1038/s41598-022-16456-0 |
| work_keys_str_mv | AT leonbondelarsen drivingsingingbehaviourinsongbirdsusingamultimodalmultiagentvirtualenvironment AT irisadam drivingsingingbehaviourinsongbirdsusingamultimodalmultiagentvirtualenvironment AT gordonjberman drivingsingingbehaviourinsongbirdsusingamultimodalmultiagentvirtualenvironment AT johnhallam drivingsingingbehaviourinsongbirdsusingamultimodalmultiagentvirtualenvironment AT coenphelemans drivingsingingbehaviourinsongbirdsusingamultimodalmultiagentvirtualenvironment |