By Sarah Forbes
K-1 ELL Teacher at John F. Kennedy Elementary, Winooski School District, Vermont
In early March, I had the amazing opportunity to go to Washington D.C. to work with representatives from WIDA, the Center for Applied Linguistics (CAL), and other ELL professionals to learn a bit more about the process of developing content for the ACCESS for ELLs 2.0 English language proficiency assessment. We worked in grade cluster teams to review drafts of new test items and to share ideas and outlines for potential future ACCESS 2.0 topics. It is nice to know that CAL includes practicing teachers in this process, relying heavily on our input to decide appropriate topics and tasks for students.
The goal of this large scale standardized test is to isolate language from content knowledge in order to get an accurate measurement of students’ proficiency in English listening, reading, writing and speaking. The results are used to inform classroom teachers of students’ strengths and areas for growth, to communicate progress to parents and schools, and ultimately to determine whether English learners continue to receive language support services. This means that test item creators must ensure that all of the content students need to complete a language task is available and accessible within the test item text, visuals and verbal directions.
In the 6 stage timeline of test development, this workshop fit into stages 3 and 4, Item Refinement and Initial Item Generation. As we worked in our teams, we began to see the careful consideration required to create test items within the confines of a restrictive online programming environment and to avoid of a long list of topics due to bias and sensitivity issues. No wonder CAL calls in teachers for help! Our teams were provided with check-lists of questions to ask ourselves as we reviewed draft items. Then we began brainstorming new topics and drafting potential new tasks for future tests.
During the content review, as our team worked through test items, we were able to voice concerns and give feedback on many aspects of the student task, including how visuals supported text, whether text was appropriate for our grade level, how wording supported student understanding, and whether the topic was applicable to all of our diverse student populations around the U.S. We found our CAL facilitator to be incredibly open and excited about our feedback. Another CAL participant took extensive notes on our discussion; later the CAL staff will comb through our transcript to find major themes and areas for improvement.
The item writing workshop asked us to create test item idea boards using assigned MPIs, or Model Performance Indicators. We generated a large number of topic ideas that avoided bias and sensitivity issues and were not repeats of past topics. These themes were then grouped together based on similarity. From these, we chose three main topics to develop into tiered speaking tasks. The new online ACCESS test is an adaptive system, and test tasks under one topic must aim to elicit language that matches proficiency levels within 3 tiers. If students are placed in tier A, beginner, in the reading and listening portions of the test, then they will be presented with tasks at at the beginner level (1) and intermediate level (3) in the speaking section. If they are placed in Tier B/C then they will be presented with intermediate (3) to advanced (5) level speaking tasks. For example, a Tier 1 task may involve simply naming objects in a visual shown on the test screen, a Tier B task might push students to make comparisons between things in the picture, and a Tier C task might encourage students to add detail and complexity to discourse by making justifications for choices based on a provided narration. WIDA’s website provides more information on the test format and development process: https://www.wida.us/assessment/access/.
The experience of working with test developers and teachers from across the country was an incredibly valuable one. As we voiced our concerns and questions, they were met with either open consideration or detailed explanations of why things are done a certain way, or why certain adjustments might be challenging. I also learned through discussion with different teachers that while our students may be different demographically and geographically, they are quite similar in their development and ability. I do believe that WIDA’s English Language Development standards provide us a cohesive and consistent definition of student proficiency and that WIDA’s work to align those standards with ACCESS 2.0, and to include educators in the process, is commendable. Despite all the technical bumps with the first year roll out of the online test, I’m confident WIDA will continue to improve on their work. I’m proud to be in a WIDA state.