Oklahoma school districts received some shocking but encouraging news this month when the state released student testing results from the last school year.
Student achievement, particularly in English, appears to be skyrocketing. One highlight: An impressive 51% of third-graders scored a passing grade or better, up from 29% last year. The reported jump came a full eight years before the majority of Oklahoma students were expected to score a passing grade under the state’s plan to meet federal accountability laws.
But the excitement quickly turned to disbelief when local authorities took a closer look at the data.
Get stories like this delivered straight to your inbox. Sign up for the 74 newsletter
“Nobody makes leaps of this magnitude,” said a testing director at a school system near Oklahoma City. The official asked to remain anonymous because she did not want to impose a “target” on her district.
To put the enormous success into perspective, The 74 asked Andrew Ho, a leading testing expert at Harvard University, to review the results.
Ryan Walters: How a popular teacher became Oklahoma’s greatest culture warrior
In Oklahoma, where student achievement has long lagged behind the national average, progress in math is two to 10 times faster than in the fastest-growing states, depending on the grade level, he said. In reading, gains are 10 to 20 times higher.
“If that’s true, … the reading and writing skills of a fourth-grader will, on average, be as good as those of last year’s sixth-grader,” said Ho, a former member of the National Assessment Governing Board, which sets the guidelines for the nationwide test commonly known as “the nation’s report card.”
As Ho suspected, Oklahoma’s supposed successes are largely illusory.
Interviews with people familiar with the state’s testing process, as well as emails and other documents obtained by The 74, indicate that the results do not reflect actual performance but rather a decision by the state, under the auspices of Superintendent Ryan Walters, to lower performance standards.
“Last year, you had to know more to be competent,” said a source familiar with the work of a technical advisory committee the state convened this summer to review competency goals. But the source, who asked not to be identified because he is still working with the state, said, “This year, you didn’t have to know as much on the same topics and you were still considered competent.”
It’s not uncommon for states to manipulate the results of large-scale tests, especially after they implement new standards. Last spring, Oklahoma students took tests for the first time that reflected a 2021 update to language standards and a 2022 math revision. But states often accompany such complex changes with attempts to communicate first to districts and then to parents how they were made and what they mean.
“It is my understanding that (the department) has handled these types of changes in the past with media events where the department has invited news organizations to help communicate the system changes,” a member of the Technical Advisory Committee wrote in May to Catherine Boomer, the department’s director of evaluation, according to an email obtained by The 74.
Boomer referred questions to the department. As of Wednesday evening, a department spokesperson had not responded to calls or emails from The 74.
A more recent email to the technical panel from Julie DiBona, vice president of program management at Cognia, the state testing provider, shows that representatives from the Department of Education requested a meeting to discuss the results. On the agenda: “How to lead the public discourse on comparing the results of the new tests to the old tests?”
A Cognia spokesman declined to comment.
But that meeting never happened. In fact, Walters recently praised early results from the state’s largest school district, Tulsa Public Schools, but made no mention of the internal machinations of the assessment. The school district saw a remarkable 16 percentage point increase in the number of students grading 3-5 with proficient or advanced grades.
“The numbers are enormous,” Walters said at a state education committee meeting last month.
Many local officials have been left perplexed by this separation.
“I’m not alone in believing that the successes achieved on the state test would be nearly, if not completely, statistically impossible in a normal year,” Tulsa County Board President Stacey Woolley told The 74.
In Moore Public Schools, for example, nearly two-thirds of third-graders scored proficient or advanced in reading, up from 38% last year. In Stillwater, about an hour north, the share of fourth-graders in those upper grades rose 21 points.
Ryan Walters calls on Oklahoma House of Representatives to take action against him “immediately”
Some blame the communication deficit on Walters, who spent the summer embroiled in dirty political wrangling and controversial academic initiatives. In June, Walters garnered national attention when he demanded that all public schools teach the Bible. Last week, at least two dozen Republicans said they would seek impeachment against him for his lack of transparency and failure to distribute funds to districts. And on Tuesday, a federal audit of the U.S. Department of Education found that his department needed to improve its financial management, complaining that it was not compliant in areas such as auditing and handling of Title I funds. Walters blamed the previous administration for some of the problems.
“There are warning signs everywhere,” said Erika Wright, head of the Oklahoma Rural Schools Coalition, calling the state’s lack of communication part of the superintendent’s “dismal record of honesty and transparency.”
“Moving the goalposts”
Parents rely on test data to understand what their children are learning, and the resulting achievement rates help determine school grades in the state accountability system. Officials use these numbers to determine which students and schools receive additional academic support.
That’s one reason why calibrations like the one in Oklahoma are common.
In May, Clare Halloran, a researcher at Brown University, confirmed the updates to the state standards with Alyssa Tyra, who is the English language arts director at the Oklahoma Department of Education. Halloran is working on a project to track state assessment data.
“They confirmed that 2024 is a new baseline and is not comparable to previous years,” Halloran said. When states change their standards, “often you’ll see either a big drop or a bigger increase than normal. That’s essentially because they’re shifting the targets a little bit.”
After teachers reviewed the test questions, the advisory committee made recommendations on how much content students need to learn to be placed in the four performance areas from below average to advanced. On a scale of 200 to 400, 300 is the minimum score for proficiency, but this year the state decided that students don’t need to get as many questions right to reach that level.
The Oklahoma City area school district official said the data made her question how much progress students really were making.
“It would have been nice to celebrate our progress instead of just feeling like this isn’t true,” she said. “It’s not representative of the hard work we’ve done.”
