Into the wild – Musical communication in ensemble playing. Discerning mutual and solitary gaze events in musical duos using mobile eye-tracking

摘要

1 Introduction/Related WorkThe communicative behaviour of musicians is multimodal: it involves gazing behaviour, gesture and musical sound. The current study focuses specifically on the function and timing of interactive gazing behaviour. Thus far, only little systematic research has been dedicated to this specific aspect of musical interaction, with the exception of Kawase (2014a, 2014b). Using external cameras, Kawase investigated two communicative aspects: the impact of leader- follower roles on gazing behaviour (Kawase 2014a) and the influence of gaze events on coordination (Kawase 2014b). Eye-tracking technology has found its way into the musical research area in studies on music reading (Drai- Zerbib et al. 2012, Penttinen & Huovinen 2011, Wurtz et al. 2009). By implementing mobile eye trackers into the study of musical ensemble interaction this study ventures into a relatively new and unknown territory.2 Our ContributionThe present study focuses on solitary and mutual gaze events in musical duos. Solitary gaze events are defined in this study as the occasions where one musician looks at the other. Mutual gaze events refer to those occasions where both musicians look at each other. The research questions are:(i) Do solitary and mutual gaze events tend to reoccur at the same places in the musical piece? (ii) Do these events correlate with specific musical characteristics? (iii) Do they correlate with specific problems in the rehearsal process? Five duos (bachelor and master level students of the Lemmens Institute, Leuven) were recorded while playing and working on a piece of their choice. The members of each duo had previously played together and had already performed or were planning to perform the piece of their choice. The instrumentation of each duo was unique, ranging from relatively unchallenging (two flutes, two guitars) over moderately challenging (harp-violin, clarinet-piano) to challenging (two percussionists) as regards the implementation of mobile eye-tracking. Each duo was recorded during two or three rehearsal sessions of around 30 minutes and was asked to work for each session on the same piece. At the beginning and end of each rehearsal the participants were asked to play through the entire piece, which resulted in a total amount of 26 run-throughs.Two ‘Pupil Pro’ eye trackers recorded the eye movements of both players during the entire rehearsal session. The rehearsal was interrupted twice for the purpose of checking and adjusting the calibration: after the first and before the second run-through. External cameras captured the full body of each player and an audio recorder guaranteed a reasonable sound quality. After each session the musicians completed a questionnaire asking them to specify difficulties in the musical score. The synchronized data (from the eye trackers, external cameras and audio recorder) are currently being annotated for gaze distribution using the annotation software ELAN.3 DiscussionWe aim to present preliminary results drawn from segments of about 1 minute out of each run-through, during which the same musical fragment is heard. The annotated solitary and mutual gaze events will be compared across several run-throughs within the same duo as well as across different duos. One general finding so far has been the fact that mutual gaze events tend to be reserved for the beginnings and endings of pieces. The causes for solitary gaze events seem to be rather diverse and related to unexpected events in the performance (e.g. a mistake by the partner), important structural moments in the music, the reversal of leader-follower roles, etc. However, analysis of a more substantial amount of data is needed in order to confirm and further specify these results.References1. Drai-Zerbib V, Baccino T, Bigand E (2012). Sight-reading expertise: Cross-modality integration investigated using eye tracking. Psychology of Music 40:216-235.2. Kawase S (2014a). Assignment of Leadership Role Changes Performers' Gaze During Piano Duo Performances. Ecological Psychology 26:198-215.3. Kawase S (2014b). Gazing behavior and coordination during piano duo performance. Attention, Perception, & Psychophysics 76:527- 540.4. Keller PE (2008). Joint Action in Music Performance. In: Morganti F, Carassa A, Riva G. (ed), Enacting intersubjectivity: A cognitive and social perspective to the study of interactions (pp. 205-221), Amsterdam: IOS Press.5. Penttinen M, Huovinen E (2011). The Early Development of Sight-Reading Skills in Adulthood: A Study of Eye Movements. Journal of Research in Music Education 59:196-220.6. Williamon A, Davidson J (2002). Exploring co-performer communication. Musicae Scientiae 6:53-72.7. Wurtz P, Mueri RM, Wiesendanger M (2009). Sight-reading of violinists: eye movements anticipate the musical flow. Experimental Brain Research 194:445-450.