When the financial services website WalletHub announced its annual list of states’ education systems, Oklahoma’s ranking became instantly quotable by politicians on all sides.
The site listed Oklahoma nearly dead last at 50th in the nation, including the District of Columbia; only New Mexico ranked lower at 51st.
WalletHub’s ranking system synthesizes 15 education-related metrics into a quality score worth 80 points using test scores, graduation rates, teacher certification, and more. An additional 20 points assess the safety of states’ education systems.
Data experts say some of their measures are spot-on. Some are misleading. Others are flat-out wrong.
But the end result mirrors others’ findings, including a recent analysis of national test score data by the Oklahoma Center for Education Policy and U.S. News and World Report, which both placed Oklahoma at 48th.
Oklahoma’s 50th-place position seemed to draw more attention and spark more policy conversations, which WalletHub’s Chip Lupo said is the purpose of their list.
“We’re a personal finance website, but we publish these to give consumers choices if they’re looking to relocate, and give them ammunition to lobby state and local leaders,” Lupo said.
And lobby they have. Sen. Ally Seifried, R-Claremore, said the ranking elevated the conversation about how to improve education, bringing in business leaders and folks without children in schools who now want answers.
“I’m glad that people are upset, because we were 47th, 48th for my previous three years, and I guess 50th really woke people up,” Seifried said.
WalletHub’s Methodology
WalletHub is best known for its credit monitoring and budgeting services. The site publishes about 100 lists annually, everything from the best states to retire in — where Oklahoma ranked 49th despite a nod for having the lowest adjusted cost of living — to the most gambling-addicted states, where Oklahoma landed at number seven.
It’s not exactly in the business of education, or assessing school quality.
WalletHub’s education ranking attempts to simplify a complex dynamic — learning — into a simple, digestible score using 15 metrics. Each is worth 3.64 points, but some are doubled to 7.27 points.
More than half the points come from students’ test scores. There’s math and reading, each a double-weight, from The Nation’s Report Card. And there are six categories related to Advanced Placement exams, SAT scores and ACT scores, worth a total of 36.36 points.
Gary Anderson, a professor at New York University and one of WalletHub’s education experts, was troubled by the site’s lack of inputs related to states’ financial support of public schools.
“Testing has come front and center as the way to rank schools,” Anderson said. “I think our rankings are too narrow.”
In the early 2000s, with the passage of the federal No Child Left Behind Act, people shifted from thinking in terms of inputs like teacher preparation and school funding to outputs like test results, he said.
“We need to bring some balance to that,” he added.
One-third of WalletHub’s formula depends on ACT and SAT scores, which can punish states that test all students, such as Oklahoma. All Oklahoma high school students take the ACT in 11th grade as part of the state’s accountability system. The state’s median score was 16 for the class of 2024, which ranked 49th in WalletHub’s analysis. ACT scores range from 1 to 36.
States with fewer testers tend to score higher because the pool is mostly high-achieving students who plan to attend college.
WalletHub did include a metric that attempts to address the issue: the share of high school graduates who complete the ACT or SAT. Oklahoma ranked 13th on that measure with more than 100% of students taking the college entrance exams.
However, Oklahoma didn’t fare well on the SAT either. With just 2% of Oklahoma graduates taking the SAT, the state still landed in 45th place.
A Crime Agaist Data
On WalletHub’s graduation metrics, things get a little squishy. WalletHub couldn’t access up-to-date dropout rate data, so its analysts subtracted each state’s graduation rate from 100. That’s different, but probably not different enough to cause a substantial ranking shift.
Then, it factored in what it called a projected graduation rate increase, using data from the Western Interstate Commission for Higher Education’s Knocking at the College Door report. But Patrick Lane, vice president of policy analysis and research at the commission, said their report does not show increases or decreases in graduation rates, which might indicate the quality of a state’s education system. Instead, it reports on the projected number of graduates, which is almost entirely driven by the birthrate.
“This is a crime against data to say this is a high school graduation rate,” Lane said. “They give you just enough information to make it seem like the calculations are robust.”
He reached out to WalletHub, but as of March 5, he hadn’t heard back.
The metric in which Oklahoma ranked highest, according to WalletHub, was its share of certified teachers. WalletHub pulled federal data from 2021 that showed 99.8% of Oklahoma teachers were fully certified, second-highest in the country.
But the state’s own data contradicts that. According to a state report, there were 2,755 educators holding emergency certificates and 2,408 holding provisional certificates that year, representing 25% of the teaching workforce. And federal guidance says teachers with alternative, emergency, temporary or provisional certification shouldn’t be counted as fully certified.
WalletHub spokeswoman Diana Polk said they rely on the data sources to be accurate. The discrepancy may be related to different definitions of fully certified, she added.
At least on math and reading, WalletHub’s findings echo those of other research, including the Oklahoma Center for Education Policy in Tulsa’s recent report, The Fall to 48th. Adam Tyner, who authored the report, reviewed WalletHub’s rankings. He said that while he would have used a different methodology, it likely wouldn’t have made a huge difference.
Like rankings of U.S. colleges, the lists use metrics with a lot of face validity that sound good and produce results people expect.
“I don’t think they made every methodological decision right, but at the end of the day, they got within two of what I came up with,” he said.
Republished in partnership with Oklahoma Watch under a Creative Commons license. Free Press publishes this report as a collaborative effort to provide the best coverage of state issues that affect our readers.
Jennifer Palmer has been a reporter with Oklahoma Watch since 2016 and covers education. Contact her at (405) 761-0093 or jpalmer@oklahomawatch.org. Follow her on Twitter @jpalmerOKC
