Is a subjective measure of intelligibility as good as an objective measure of intelligibility?

August Quotes, website.png

It is 4:45 PM. You’re frantically trying to finish up a report so you can be out the door by 5:00. The evidence-based practitioner in you is determined to provide an objective measure of intelligibility in your comprehensive speech and voice evaluation for your client with Parkinson’s Disease (PD). So, you obtain a 3-minute audio recording of the ‘Caterpillar Passage.’ You listen and compare your client’s performance to the transcribed version, carefully marking words understood. Ten minutes later, you’re only 50% of the way through the recording because you had to stop multiple times to transcribe what you actually heard. And, you still need to count the total number of words understood so you can calculate the percent intelligibility score… And, it’s 5:15.  

Isn’t there a quicker way of measuring intelligibility? 

The good news is, yes, there is! Abur et al. found that the use of a visual analog scale (VAS) was a feasible way to measure intelligibility. The relationship between orthographic transcription (OT) and VAS was strong regardless of whether raters were rating a short or long speech sample, or if the speech samples were produced by speakers with or without PD.

This relationship, though, was dependent upon having at least two listeners/raters use the VAS. Let’s break this down: If only the treating SLP uses VAS, the average difference between OT and VAS is 8%— MORE than the 7% average treatment effect of Lee Silverman Voice Treatment (LSVT LOUD). If the SLP grabs a colleague to also rate, the difference between VAS and OT drops to a much more acceptable range, 4.5%. Figure 4 captures this effect very nicely.  

A notable fact was that all the listeners were naïve—they were neither trained to assess speech intelligibility, nor did they have exposure to speech-impaired individuals prior to this study. This allows flexibility for SLPs to choose even non-SLP colleagues to be the 2nd rater.  

This study builds on an emerging body of work that also found objective and subjective measures of intelligibility to be highly correlated with listeners who are unfamiliar with evaluating disordered speech. Along with being an accurate predictor of intelligibility, VAS is also feasible given the time constraints of clinical settings. The authors found that listeners took an average of 5 minutes to rate 28 sentences using VAS as opposed to 9 minutes for the OT condition – short enough to convince a colleague to do you a solid and provide a rating!

So, how do we use VAS? Here’s one way:


Or, use software!  

Office scene, now: It’s 4:45 PM and you’re about to rate a client’s speech intelligibility after listening to an audio recorded sample of him reading. You pull out your VAS, grab the physical therapist (PT) (who owes you a favor anyway), and done by 4:55. Out the door you go! Remember to grab an extra coffee for that PT who helped you in the morning!  

Abur, D., Enos, N.M, & Stepp, C.E. (2019). Visual Analog Scale Ratings and Orthographic Transcription Measures of Sentence Intelligibility in Parkinson’s Disease With Variable Listener Exposure. American Journal of Speech–Language Pathology. doi:10.1044/2019_AJSLP-18-0275.