SMPC Day 3: Session 4 (Multisensory)
The final session of talks on Saturday at SMPC had me almost as anxious as the session that contained my talk on earworms.
These nerves were because my lovely masters student, Guang Yi Chua, was due to present the data from his masters project on tone colour synaesthesia and absolute pitch. Because of my nerves I’m not sure I even smiled though I am sure he was looking to me for reassurance. He did not need to though as he did a great job.
Before Guang Yi however, there were other talks on multisensory perception, starting with a talk from a lovely chap, Matts Küssner, who did his masters on the MSc MMB program back in the day. Matts is coming towards the end of his PhD at King’s College where he has been working on the music and shape project with Professor Daniel Leech Wilkinson.
Matts has been working in a really interesting area; how ideas about music are influenced by the concept of shape, and vice versa. He pointed out that there are many traditional associations in the realm, such as the pairing of spatial ascent with increased pitch and increasing size of shapes with increasing loudness. Matts’ work extended such traditional pairings into the world of gestures.
He used clever wii technology to capture musicians (composers, pianists and other instrumentalists – no percussionists) and nonmusicans movements as they moved in space to represent the sounds that they were hearing. They were asked to complete two conditions, one with and one without visual feedback
Matts found lots of interesting patterns connecting gestures with musical form; pitch was represented in the Y axis, with a marked asymmetry but only in nonmusicians. Loudness was also represented it in the Y axis. Matts measured muscular energy used in movements but this only correlated with loudness in the nontrained people. And only musicians sped up their movements in response to changes in tempo.
There were other relationships as well, but they key message from the talk is that Matts’ paradigm provides a neat method for measuring how real time body movements in space are inspired by musical sound, and that these movements appear to be fairly consistent across different people (though differentiated by musical training)
The next talk was a duet by Jason Silveira ( Oregon State, US) and Frank Diaz (University of Oregon, US) who discussed the role of expressivity in musical performance. They showed clips and asked us to consider the expressive behaviours. One clip was of B.B King, so I was a very happy audience member!
The gentlemen demonstrated how perceptions of expressivity can be enhanced but not always in a systematic fashion. They raised the question of whether expressivity and performance gesture should be more explicitly taught in music schools. They also presented an interesting study that showed how the reading of subtitles (for example during opera) can reduce continuous perceptions of expressivity in a performance – though of course this must go hand in hand with the question of comprehension.
The third talk was by Konstantina Menouti (Goldsmiths, University of London) who, like Guang Yi, was presenting the sum of her present MMB masters project (see the link for a video of the talk). She has been studying timbre-colour synaesthesia, an anecdotal but never formally studied condition whereby people report sensations of colour when hearing music that varies in its timbre.
Konstantina has developed a melody and a single tone timbre test, and so far has assessed the experiences of 3 synaesthetes who show both consistent timbre colour synaesthetic experiences (as in, they experience the same associations reliably)
She told us about some of her interesting observations that may inform future work. For example, some of her participants reported more than one colour to the same timbre and one reported that they only experienced colour in response to changing timbres (i.e. not single tones).
Another issue is semantic mediation, how much recognition of instruments plays a role in the experience. One solution would be present pictures of instruments of their names during the test. Konstantina also plans to run a Stroop like task to determine the automaticity of the timbre-colour experiences of her participants, since automaticity is one of the key constructs in general synaesthetic experiences.
Finally, we came to Guang Yi. He has been interested in possible links between absolute pitch (AP) and tone-colour synaesthesia (TCS). Recent evdience has suggested that they may have a common underlying antecedent, that of hyperconnectivity in the brain. At the end of the day they are both forms of ‘labelling’ for musical sound (whether with colour or a tone name) so we wondered whether one may facilitate the appearance of the other.
His first study was an epidemiological investigation of the population prevalence of TCS within people who have AP. Only about 0.35% of the population have TCS – is the prevelance higher in a group of AP? Turns out that yes, it is, quite a bit higher. We have a rate of around 7% and it could be as high as 20%, depending on borderline people who we would need to assess in the lab.
Guang Yi’s second study then looked at the question of whether having an extra label for tones, as in TCS, facilitates pitch perception or pitch memory.
He has so far tested 3 TCSs and compared them to a control group of 5 people with matched age, gender and training. And so far there is no evidence to support the idea that TCS facilitates pitch processing in either of our tests but further work would need to be done to address issues of sample size and sensitivity of the tests.
So that was it, a great, packed Saturday. After giving Guang Yi a hug of congratulations I headed out with a group of my friends.We investigated Five Guys burgers (excellent!), the Elephant and Castle pub (very friendly and comfortable with a good selection) and our skills at table hand-slaps game with tequila shots (a slight mistake). A very good time was had by all.