Assessing language with diverse preschoolers? Go for dynamic assessment

2.png

Making the right call when assessing language skills of children with cultural or language backgrounds that don’t match our own is hard. Using our go-to assessment methods, we risk labeling normal language variation as signs of a disorder. Standardized test norms may over-identify children from non-mainstream language backgrounds as having language impairment.  

Enter dynamic assessment, which involves testing a child, providing teaching and support, and then retesting to see what the child can do with help. In a new study, Henderson et al. used dynamic assessment to assess language skills of Navajo preschoolers with narrative retell tasks from the Predictive Early Assessment of Reading and Language (PEARL, from the same acronym aficionados that brought us the DYMOND).

Dynamic assessment takes longer than static (one-time) assessment. The PEARL accounts for this—you give the pretest, look at the score, and then administer the teaching and retest only if it’s below a cutoff. Henderson et al. found that the reported cutoff score for the PEARL pretest didn’t work well for Navajo children; sensitivity and specificity were better with a cutoff score of 7 rather than 9. Looking at the whole test, scores on the retest (following teaching) were even better at diagnosing children, and examiners’ “modifiability” ratings (how the child responded to teaching) diagnosed children with 100% accuracy. These findings suggest that the PEARL is a valid test for assessing language in children from non-mainstream language or cultural backgrounds.   

 

Henderson, D. E., Restrepo, M. A., & Aiken, L. S. (2018). Dynamic assessment of narratives among Navajo preschoolers. Journal of Speech, Language, and Hearing Research, 61(10), 2547–2560.

Teacher ratings as a language screening for dialect speakers

3.png

In the last review, we shared research on a potentially valid tool to screen Mainstream English-speaking kindergarteners for language disorders. But what about our kiddos who speak other dialects of English, like African American English (AAE) or Southern White English (SWE)? In this study, researchers gave a group of AAE- and SWE-speaking kindergarteners a handful of language and literacy screeners, to see which one(s) could best identify possible language disorders, while avoiding “dialect effects.”

Their most successful screener (and TISLP’s winner for best acronym of the month) was the TROLL, or Teacher Rating of Oral Language and Literacy—available here for free. And yes, that’s a teacher questionnaire, rather than another individually-administered assessment for our students who spend so much time testing already. Importantly, the teachers completed the ratings and the end of the kindergarten year, not the beginning, so they had time to really get to know the students and their abilities.

The researchers calculated a new cut score of 89 for this population, since the TROLL itself only suggests cut scores through age 5. This resulted in sensitivity of 77% for identification of language disorders. Now, 77% isn’t really high enough—we want a minimum of 80 for a good screener. But it may be a starting place until better tools come our way.

Gregory, K. D., & Oetting, J. B. (2018). Classification Accuracy of Teacher Ratings When Screening Nonmainstream English-Speaking Kindergartners for Language Impairment in the Rural South. Language, Speech, and Hearing Services in Schools, 49(2), 218–231. https://doi.org/10.1044/2017_LSHSS-17-0045.

Modification to standardized tests for speakers of nonmainstream dialects

The authors of this paper discuss how, when an SLP evaluates a young speaker of a nonmainstream American English dialect (NMAE), s/he is faced with two tasks: first, to determine if the child is a speaker of a nonmainstream dialect, and then to determine if that child does or does not have a language disorder.

Though the task may seem straightforward at first glance, it can be incredibly challenging. One major barrier is that children use NMAE variably. Conversational contexts are more likely to elicit NMAE use, then use can also change per communication partner. Dialect use also changes with age; the authors state, “… the general trend is that use of NMAE features drops during the first few years of elementary school as students master code-switching strategies, and then increases during adolescence as students begin using NMAE dialect for more social reasons (N.P. Terry et al., 2010; Van Hofwegen & Wolfram, 2010).” This variability is challenging. Then, the overlap between what’s considered ungrammatical in mainstream American English and grammatical in NMAE makes it all the more challenging.

As part of the evaluation process, SLPs may choose to use a combination of language sample analysis (LSA) with standardized testing. An adjustment that is often made to the standardized test in order to account for the child’s dialect is to apply scoring modifications—that is, count an item on a test as “accurate” if it’s accurate per the child’s dialect. And this is in-line with what is recommended within testing manuals, e.g. per the CELF-4 and CELF-5.

In this study, the researchers examined what happens when you try using scoring modifications on the CELF-4 with a sample of 299 2nd-grade students. They found that:

  • without scoring modifications, NMAE speakers were over-identified as having a language disorder
  • but with scoring modifications, the over-identification of children as having a language disorder was improved, but the under-identification of NMAE speakers who do truly have a language disorder also increased

Yikes. It’s well-known that using a standardized language assessment for a speaker of a nonmainstream dialect, when the test wasn’t designed with speakers of that dialect in mind, can provide inaccurate diagnostic results (see article for review). However, this study also provides clear data that the scoring modifications don’t exactly work well, either.

Currently, there isn’t a perfect solution. For now, it’s important for SLPs to simply understand the potential pitfalls they may encounter during assessment. The authors suggest that good options to add to the assessment protocol include: detailed case histories of the child’s abilities at both home and school, peer comparisons, LSA, and dynamic assessment. The authors acknowledge the huge need for more research on how to streamline this process, because even with some of the strategies that look promising (like dynamic assessment), we still don’t have adequate research to fully guide diagnostic decision-making.

Hendricks, A.E., & Adlof, S.M. (2017). Language Assessment With Children Who Speak Nonmainstream Dialects: Examining the Effects of Scoring Modifications in Norm-Referenced Assessment. Language, Speech, and Hearing Services in Schools. Advance online publication. doi:10.1044/2017_LSHSS-16-0060

Dialect awareness for school-age children

Children who enter school speaking a non-mainstream dialect must quickly learn to dialect shift (a.k.a “code-switch”). Similar to bilingual children, they have two sets of syntactic, semantic, morphologic, and phonological rules, to be applied in different settings and with different communicative partners.

Mainstream American English (MAE) is used in American schools, in the workplace, and in classroom literature. Most children who enter Kindergarten speaking a non-mainstream American English (NMAE) dialect, such as African American English (AAE), “…change their dialect use spontaneously and without explicit instruction,” with the 1st grade being critical as a time of the most rapid growth in dialect shifting. Importantly, children who don’t learn how to dialect shift (e.g. continue to use NMAE in their writing, when MAE is the expectation) struggle; they “…tend to demonstrate weaker literacy achievement and less growth in reading skills during the school year,” and “…research findings over the last 15 years suggest a strong, predictive relationship between young children’s spoken NMAE use and various language and literacy skills, including vocabulary, word reading, spelling, phonological awareness, reading comprehension, and composition” (see article for thorough literature review).

Who are these children who aren’t dialect shifting spontaneously? Data point toward language skill—“…oral language skills, such as vocabulary and morphosyntax, appear to be associated with… dialect shifting ability.”

In this study, the researchers aim to reduce the achievement gap observed in children who don’t spontaneously shift dialects by providing a Dialect Awareness program (DAWS). This program is built upon decades of evidence, thoroughly reviewed in the paper. This paper actually covers two studies—Part 1 with 116 children, and Part 2 with 374 children. For our purposes, we’ll focus only on Part 2, because it was built upon findings from Part 1. Participants were 2nd­–4th grade students (45% African American, 33% White, 4% Hispanic, 4% Asian, 7% multiracial) from four different schools in the southeastern U.S. Children were eligible to participate in DAWS if NMAE features were present in their writing.

The DAWS program was provided to half the students in 15 minute sessions, 4 days per week, for 8 weeks. The other half of the students served as controls. DAWS targeted the following forms: copula/auxiliaries, plurals, past tense, subject–verb agreement, possessives, and preterite had.

First, it’s imperative to note that, “… the instructional program was designed to be respectful of both dialects…” Instruction not only highlighted differences between MAE and NMAE grammar and vocabulary, but taught that it was good and normal to use both dialects, and that there are contexts for using each dialect. Instructors used analogies to things like clothing—just like outfits differ per situation, so does dialect. Language activities within the program included listening tasks, sentence cloze tasks, editing tasks, sentence sorts, and plenty of games with vocabulary and grammar tasks built in. There was a lot of writing, as well, which was taught as a primary context for MAE use. Instructors provided both reminders and corrective feedback, such as: “Remember that we are using school language so we have to include –s/-es for plurals and –d/-ed for past tense.” To demonstrate to students how to use both dialects in their writing, “…students learned to put quotes around sentences where characters in their narratives were using home English…”

Overall, DAWS was found to be very effective, with post-program gains in language, reading, and writing skills. Also, though effective for the group as a whole, it was found to elicit the greatest gains for children who entered the program with the heaviest NMAE use, and in children who started with somewhat weaker language scores.

Johnson, L., Terry, N.P., Connor, C.M., Thomas-Tate, S. (2017). The effects of dialect awareness instruction on nonmainstream American English speakers. Reading and Writing. Advance online publication. doi:10.1007/s11145-017-9764-y