Sp7book copy

2.5. Perceptual Evaluation 
An additional 30 female participants were recruited to 
participate in a perceptual evaluation task. None had 
participated in the production experiment. They were 
randomly assigned to rate either the expressivity or the 
rhythmicity of the verse couplets. 
Half of the participants (n= 15) rated their perception of the 
each couplet’s rhythmicity on a scale of 1-4 where 1 was 
“very rhythmic” and 4 was “not at all rhythmic.” Instructions 
were adapted from [5] such that raters were asked to determine 
how well they could find a steady beat for the couplet. Each of 
these 15 participants was assigned to one of the three lists, so 
that five participants rated each couplet. They were first 
presented with 8 practice couplets; four were categorized by 
the authors as “very rhythmic”; four were categorized as “not 
at all rhythmic.” We observed agreement on these ratings 
which was significantly greater than chance, Fleiss’ κ = .09, z 
= 10.5, p = 0 [25], indicating that raters were sensitive to 
couplet rhythm. Ratings from all five raters were averaged, 
resulting in an average Rhythmicity score for each couplet. 
Half of the participants (n=15) rated their perception of the 
each couplet’s expressivity on a scale of 1-4 where 1 was 
“very expressive” and 4 was “not at all expressive.” Each of 
these 15 participants was assigned to one of the three lists, so 
that five participants rated each couplet. They were first 
presented with 8 practice couplets; four were categorized by 
the authors as “very expressive”; four were categorized as “not 
at all expressive.” We observed agreement on these ratings 
which was significantly greater than chance, Fleiss’ κ = .29, z 
= 34.5, p = 0, indicating that raters were sensitive to couplet 
expressivity. Ratings from all five raters were averaged, 
resulting in an Expressivity score for each couplet.
SP-7
Conference Programme
Campbell, Gibbon, and Hirst (eds.)
Speech Prosody 7, 2014
473

---