Performance – the hidden power of the visual

Hello Dear Reader,

One of the things I enjoy about the world of music psychology is the sense of community. We are \ a relatively small discipline so you tend to see similar faces at conferences and social events, and I very often bump into colleagues and ex-students either in person or when perusing the literature.

Recently I came across a fellow music psychology academic who I have known since we completed our MAs in the Psychology of Music at the University of Sheffield in 2005. Dr Noola Griffiths is a charming and bubbly person, always smiling and full of enthusiastic energy for her work.

I once interviewed Noola for this blog about her life in research and she gave a detailed and helpful response. You can see that interview here.

Noola’s MA project looked into the impact of concert dress on people’s opinions of a music performance. I remember seeing her stimuli in a class discussion and thinking – ‘wow, okay, so you can do publishable stuff at this level’. It was inspiring. This project turned into a great paper that got a mention (in a really positive way!!) in Ben Goldacre’s famous ‘Bad Science’ column in the Guardian.

“Keeping up appearances” in Bad Science – Noola’s paper is referred to here as ‘cracking’! (That’s a good thing for anyone unfamiliar with the English slang). I quote from that article on the results:

For technical proficiency, performers in a concert dress were rated higher than if they were in jeans or a clubbing dress, even though the actual audio performance was exactly the same every time (and played by a separate musician who was never filmed). The results for musicality were similar: musicians in a clubbing dress were rated worst.

New paper

Noola has continued her interest in this area of research and recently published a paper with Jonathon Reay called “The Relative Importance of Aural and Visual Information in the Evaluation of Western Canon Music Performance by Musicians and Nonmusicians” for the Music Perception.

The new paper aims to differentiate the effects of audio and visual information on the evaluation of a musical performance. In the Western Classical genre tradition, it is often taken for granted that auditory events are critical for the way we evaluate the merits of a music performance. We care about whether a performance SOUNDS good … right?

Noola’s paper above and others have chipped away at this idea, showing that visual elements of a performance can indeed impact on how we evaluate a musician’s performance, whether we like that idea in theory or not. At a basic level it seems we prefer a performance when we have some visuals. Broughton and Stevens (2009) found that audience members gave higher ratings when they experienced performances in audiovisual format compared to audio alone. This result is linked to the strong relationship we have between observed movement and perceived emotional expression and intention. The visuals add to a performance, there is no doubt.

But can visuals really impact on how well we evaluate the quality of a performance?

And let’s go further (why not?!) – might the relative importance of visual information be even higher than that of audio information when evaluating a music performance?

And finally, does the musical experience (minimum grade 8 musician vs. no formal music training) of the evaluator matter?

Design

In a fantastic piece of simple design, Noola and her colleague found video piano performances (Debussy’s Clare de lune) that were rated as ‘amateur’ (sub-elite) or ‘professional’ (elite).

They paired these seamlessly alongside two levels of music audio performance, ‘amateur’ and ‘professional’, in all possible combinations.

Both performers were female and of white ethnicity; the pianists’ facial expressions were clearly visible but their hands were not in sight. The four videos were rated on a 7 point scale for technical proficiency, musicality, and overall performance quality.

So do professional videos influence how well we rate the sound of a music performance?

Results

The congruent professional performance (both audio and visual was professional) was rated significantly higher than the congruent amateur performance and both incongruent performances.

The key results was that more importance was placed on the visual element of performance than the audio element when evaluating performances on all three measures. Across all three evaluative measures of performance, Amateur Audio + Professional Video, was rated significantly higher than Professional Audio + Amateur Video.

There was no impact of musicality in the evaluator upon these results.

Conclusions

As hypothesised, visual information has a greater impact than audio information on observers’ judgements of technical proficiency, musicality, and overall performance quality.

Both audio and visual information are important, which explains why the professional + professional example was consistently rated the best and the amateur + amateur was the worst. However, when the audio and visual are tested against each other in this clean and simple way, it was the quality of the visual information that more strongly swayed opinion towards the positive ends of the evaluation scales.

So why is the visual so important?

The authors offer an explanation in terms of Daniel Kahneman’s theory published in his 2011 best selling book Thinking, Fast and Slow.

He argues that we have two cognitive systems: System 1 is automatic and quick, offering intuitions, impressions and feelings; System 2 is slower and requires attention, providing us with effortful mental activities. This second system comes into play if there is a reason to doubt System 1’s evaluation.

In a short evaluation paradigm such as the present musical example, the authors argue that visual information is used as a primary heuristic in System 1 to minimize cognitive effort and load, as would be required by System 2.

One important note is that this study focuses on the performance outcomes and not the skills of the performer. There is nothing to say that visual information is more important than auditory when it comes to deciding on the merits of a musician’s inherent skill. But who knows?

Also, this result is clearly confined to it’s genre of Western tonal classical music – it would be interesting to see if the pattern was consistent in other kinds of music performance, or hold out when the music being evaluated is unfamiliar to the audience.