LoboVault Home

Listener reliability and agreement of a brief intelligibility rating task


Please use this identifier to cite or link to this item: http://hdl.handle.net/1928/11089

Listener reliability and agreement of a brief intelligibility rating task

Show simple item record

dc.contributor.author Ewing-Buck, Cai
dc.date.accessioned 2010-09-03T20:49:08Z
dc.date.available 2010-09-03T20:49:08Z
dc.date.issued 2010-09-03
dc.date.submitted July 2010
dc.identifier.uri http://hdl.handle.net/1928/11089
dc.description.abstract PURPOSE: This study addresses the development of the I-RAVN Test of Speech Intelligibility, an assessment instrument designed to identify which speech components most affect speech intelligibility. The I-RAVN consists of ratings of overall intelligibility, and ratings of four speech components: rate/rhythm/prosody, articulation, voice quality/breath support, and nasality using a rating scale technique adapted from the CAPE-V instrument for voice. This study seeks to establish that listeners can reliably rate overall intelligibility and the four speech components in speakers with dysarthria. METHODS: Twenty-two graduate students listened to recordings from 24 talkers (7 normal, 6 with Parkinson Disease, 11 with oculopharyngeal muscular dystrophy) producing 3 sentences. The listeners rated each talker using the I-RAVN tool, which uses a visual analog scale (100 mm lines) to evaluate the following speech dimensions: overall impression of intelligibility; rate/rhythm/prosody; articulatory precision; voice quality/breath support; and nasality. RESULTS: To assess intra-rater reliability and agreement, listeners rated sentences from five of the speakers chosen at random a second time, and Pearson product-moment correlations, t-tests, and percent close agreement calculations were performed for all pairs of 22 listeners. Pearson t-tests showed that there were no significant differences between the first and second ratings of the repeated talkers, though percent close agreement calculations demonstrated that nasality, intelligibility, and articulation were more likely to be rated consistently than rate and voice. Overall, intra-rater reliability was high for intelligibility, articulation and voice, and lower for rate and nasality. To assess inter-rater reliability and agreement, Pearson product-moment correlations, factor analysis, intraclass correlation coefficients (ICCs), rater bias one way analyses of variance (ANOVAs), and percent close agreement calculations were performed. The Pearson correlations demonstrated that more than 85% of the ratings were consistent for intelligibility and articulation, and less than 50% for rate. The ICCs showed that listeners had high consistency when rating intelligibility, moderate consistency when rating articulation, voice, and nasality, and lower consistency when rating rate. Inter-rater reliability and agreement across measures were high for intelligibility, somewhat lower for articulation, voice, and nasality, and consistently lower for rate. Overall, good reliability and agreement were noted for intelligibility and articulation, with moderate values for voice quality and nasality. Lower levels of reliability and agreement were obtained for the rate/rhythm/prosody scale on both intra- and inter-rater tests. CONCLUSIONS: Preliminary results indicate adequate inter- and intra-rater reliability and agreement for the I-RAVN Test of Speech Intelligibility for dysarthric speech. Further research will determine if the I-RAVN can be used as an explanatory, streamlined assessment technique to determine treatment targets for individuals with speech intelligibility deficits. en_US
dc.description.sponsorship University of New Mexico Department of Speech and Hearing Sciences en_US
dc.language.iso en_US en_US
dc.subject dysarthria en_US
dc.subject intelligibility en_US
dc.subject.lcsh Speech, Intelligibility of--Testing--Evaluation.
dc.subject.lcsh Articulation disorders--Diagnosis--Evaluation.
dc.title Listener reliability and agreement of a brief intelligibility rating task en_US
dc.type Thesis en_US
dc.description.degree Speech-Language Pathology en_US
dc.description.level Masters en_US
dc.description.department University of New Mexico. Dept. of Speech and Hearing Sciences en_US
dc.description.advisor Neel, Amy
dc.description.committee-member Dale, Philip
dc.description.committee-member Palmer, Phyllis

Files in this item

Files Size Format View
Thesis 7-12-2010.pdf 1.645Mb PDF View/Open

This item appears in the following Collection(s)

Show simple item record

UNM Libraries

Search LoboVault


My Account