Opinion

Op-Ed: Oakland Achieves School Progress Report Misses the Mark

Published

10 years ago

October 6, 2016

By Jan Malvin and Joel Moskowitz

Confusing the public may well be the major achievement of the fourth annual Oakland Achieves Public Education Progress Report prepared by Urban Strategies Council.

This report, deemed “primarily an update on the academic outcomes for the 2014-15 school year,” offers no trends data for Oakland Unified School District-run schools.

Rather, it is the first report in the series to feature student-level data from charter schools.

Without explaining the omission of trends data for district-run schools, the report appears crafted to tell a story that compares charter schools with district-run schools.

Commenting on Oakland Achieves, the Executive Director of GO Public Schools states, “The data is at a high level to spark collaborative learning, not to pit school types against each other and draw tentative conclusions.”

Yet, this report manufactures unfair competition between the charter and district-run school sectors, relying on incomplete or biased data.

For example, out-of-district students enrolled in OUSD-authorized charter schools ranged from 0-56 percent in 2015-16 (average = 14 percent).

How fair would it be to judge performance of a sports team that recruits exclusively from within the city against one that recruits from the whole Bay Area?

Urban Strategies announced, “One important finding revealed that while charter schools did poorly in ELA testing for 3rd-5th graders, by middle school, math testing of 7th & 8th grades were much better than for district schools—a flip in performance.”

Claiming such a “flip in performance” by comparing data from two different cohorts of students at a single point in time, one set of data from English (ELA) and the other from math, is absurd.

If this observation were valid, a more plausible interpretation is that many high achieving students in district-run elementary schools transfer to charters or private schools for middle and high school.

Much higher standards—matched comparison groups, measurement of each indicator at least twice—apply before one can make causal statements about school effects on student performance.

If these standards cannot be met by design, analysts must employ statistical controls for all pre-existing differences that may affect the outcomes.

Since Oakland Achieves failed to address pre-existing student differences, conclusions about charter vs. district-run schools are indefensible.

It is essential to understand the school-site practices and other factors that underlie different outcomes.

This report does not acknowledge this shortcoming and so misleads the reader with false comparisons between types of schools.

Many factors may account for student outcomes, but Oakland Achieves addresses none of these.

For example, selective enrollment and pushout practices, described by the ACLU in Unequal Access, its recent report on charter school enrollment practices, can influence indicators of student progress (see: www.aclusocal.org/unequal-access/).

Oakland Achieves does not consider the impact on student outcomes of teacher turnover and experience; instructional practices; curriculum; parental involvement, education, and occupation; and details about vulnerable subgroups.

For example, are special education students in charter schools less diverse than special education students in district-run schools?

A final shortcoming of Oakland Achieves is that a considerable amount of data was missing for charter schools. Although the report notes the omission, it does not spell out the implications.

Missing data, especially if not random, biases comparisons with district-run schools because charter results do not reflect the entire charter school population.

Despite public claims to the contrary, Oakland Achieves has not demonstrated that charter schools do a better (or worse) job than district-run schools on any of the indicators presented.

The outcomes are different because, from the start, the students are different.

Jan Malvin, Ph.D., is a retired University of California researcher; Joel Moskowitz, Ph.D., is in the School of Public Health, UC Berkeley.