LoboVault Home

Listener reliability and agreement of a brief intelligibility rating task


Please use this identifier to cite or link to this item: http://hdl.handle.net/1928/11089

Listener reliability and agreement of a brief intelligibility rating task

Show full item record

Title: Listener reliability and agreement of a brief intelligibility rating task
Author: Ewing-Buck, Cai
Advisor(s): Neel, Amy
Committee Member(s): Dale, Philip
Palmer, Phyllis
Department: University of New Mexico. Dept. of Speech and Hearing Sciences
Subject: dysarthria
LC Subject(s): Speech, Intelligibility of--Testing--Evaluation.
Articulation disorders--Diagnosis--Evaluation.
Degree Level: Masters
Abstract: PURPOSE: This study addresses the development of the I-RAVN Test of Speech Intelligibility, an assessment instrument designed to identify which speech components most affect speech intelligibility. The I-RAVN consists of ratings of overall intelligibility, and ratings of four speech components: rate/rhythm/prosody, articulation, voice quality/breath support, and nasality using a rating scale technique adapted from the CAPE-V instrument for voice. This study seeks to establish that listeners can reliably rate overall intelligibility and the four speech components in speakers with dysarthria. METHODS: Twenty-two graduate students listened to recordings from 24 talkers (7 normal, 6 with Parkinson Disease, 11 with oculopharyngeal muscular dystrophy) producing 3 sentences. The listeners rated each talker using the I-RAVN tool, which uses a visual analog scale (100 mm lines) to evaluate the following speech dimensions: overall impression of intelligibility; rate/rhythm/prosody; articulatory precision; voice quality/breath support; and nasality. RESULTS: To assess intra-rater reliability and agreement, listeners rated sentences from five of the speakers chosen at random a second time, and Pearson product-moment correlations, t-tests, and percent close agreement calculations were performed for all pairs of 22 listeners. Pearson t-tests showed that there were no significant differences between the first and second ratings of the repeated talkers, though percent close agreement calculations demonstrated that nasality, intelligibility, and articulation were more likely to be rated consistently than rate and voice. Overall, intra-rater reliability was high for intelligibility, articulation and voice, and lower for rate and nasality. To assess inter-rater reliability and agreement, Pearson product-moment correlations, factor analysis, intraclass correlation coefficients (ICCs), rater bias one way analyses of variance (ANOVAs), and percent close agreement calculations were performed. The Pearson correlations demonstrated that more than 85% of the ratings were consistent for intelligibility and articulation, and less than 50% for rate. The ICCs showed that listeners had high consistency when rating intelligibility, moderate consistency when rating articulation, voice, and nasality, and lower consistency when rating rate. Inter-rater reliability and agreement across measures were high for intelligibility, somewhat lower for articulation, voice, and nasality, and consistently lower for rate. Overall, good reliability and agreement were noted for intelligibility and articulation, with moderate values for voice quality and nasality. Lower levels of reliability and agreement were obtained for the rate/rhythm/prosody scale on both intra- and inter-rater tests. CONCLUSIONS: Preliminary results indicate adequate inter- and intra-rater reliability and agreement for the I-RAVN Test of Speech Intelligibility for dysarthric speech. Further research will determine if the I-RAVN can be used as an explanatory, streamlined assessment technique to determine treatment targets for individuals with speech intelligibility deficits.
Graduation Date: July 2010
URI: http://hdl.handle.net/1928/11089

Files in this item

Files Size Format View
Thesis 7-12-2010.pdf 1.645Mb PDF View/Open

This item appears in the following Collection(s)

Show full item record

UNM Libraries

Search LoboVault


My Account