Turns Out — Who Knew? — Strong State Accountability Systems Might Even Be Helpful

A day after this year’s disappointing results from the National Assessment of Educational Progress were released, Stanford economist Martin Carnoy published a new analysis of NAEP data that should start some new lines of thinking.

Carnoy and a couple of his colleagues looked at NAEP data from 1992 to 2013, when scores showed significant improvement in both fourth- and eighth-grade mathematics. They found some states improved much more than other states. Texas and Florida, for example, improved at faster rates than other states — and so did Massachusetts.

Carnoy wanted to understand what could explain the results of the different states, so he did a number of analyses on the eighth-grade math data. Like other researchers, he found that poverty can explain a lot of the variation among states scores — not just the poverty of individual students but of concentrated poverty in schools and states.

But there were improvements that still needed explaining, even adjusting for poverty. He found that teacher union strength did not explain the variation, nor did per-pupil spending. Instead, Carnoy came to a conclusion that might surprise some people: Strong accountability for results could explain part of the reason for academic achievement gains in states that improved markedly.

Or, as his report said:

“When we explored possible explanations for the variation that remains in average 8th grade mathematics state test scores after adjustments were made, we found that the strength of state school accountability measures is significantly related to adjusted state test scores over both 1992–2003 and 2003–2013. Students in states with stronger accountability systems do better in the NAEP math test, even though that test is not directly linked to the tests that are used to evaluate students within states.”

There are a lot of caveats that need to go along with Carnoy’s paper. For one, his definition of “strong accountability systems” is open for a lot of debate; for another, his analysis was only on eighth-grade math. He or others might want to see if they can replicate the results with other grade levels and subjects.

Still, he has started an interesting line of inquiry that state policymakers might want to keep an eye on.