Given the number of parents who have looked at me, stunned, when I uttered those words above to them in the last month, it seems a given that this statement–different DC public middle schools use different PARCC math tests–is not common knowledge.

Nonetheless, it is true: starting at the 6th grade, different DC public schools can elect to offer their students different PARCC tests. None of this is recorded explicitly in official PARCC test results available to the public, which are generally aggregated test scores at each school and, in the case of school equity reports, aggregated results for specific demographic groups at each school. [Update: See comments below on how this extends well beyond middle schools]

In other words, middle school math test scores that are reported via MySchoolDC (which runs the lottery and makes available school profiles for comparison by parents) and LearnDC (run by OSSE, the office of the state superintendent of education) and school equity reports (a collaboration of DC public school agencies, available through LearnDC above) are aggregates of scores on tests that are not the same within, and across, DC public middle schools.

Or, as the saying goes: we are comparing apples to oranges in our PARCC math test score reporting for DC public middle schools.

In DC, where school choice is popular, schools are judged in large part by a reliance on these test scores as proof of educational virtue, resulting in either lauding or closure.

But the only reason I know anything about different PARCC math tests across DC public schools is because Caryn Ernst, a member of the cross sector task force, asked for more detailed PARCC reporting from the deputy mayor for education (who is in charge of the task force). Caryn had noticed that recent PARCC math proficiency scores for Stuart-Hobson Middle School (her and my middle school) were very low, and she wanted to understand why, given that many students at Stuart-Hobson are advanced in math and take advanced math courses.

That is: Were the low test scores at Stuart-Hobson a problem of the curriculum? Of the teachers? Of the students?

In the data provided by the deputy mayor for education (DME), Caryn looked at results for selected charter and DCPS middle schools. (Test results for these schools using OSSE data are here and here. All test scores from OSSE are available here.) Caryn noticed that the charter middle schools appeared to give their students the PARCC pre-algebra test for math, while some DCPS middle schools (including Stuart-Hobson) gave their students more advanced PARCC math tests for geometry or algebra 1.

Citywide, according to OSSE data (see page 22 of the report linked here, from OSSE’s website), there were 993 algebra 1 test takers and 197 geometry test takers in DC public middle schools. None appear to have been students at DC charter schools.

The results for different tests, with different degrees of difficulty, is exactly what you would think: math PARCC scores for middle schools electing to have their students take the more advanced geometry and algebra 1 tests are lower than for those schools electing to take the pre-algebra test.

Yesterday, the head of OSSE, Hanseul Kang, confirmed to both Caryn and me that each DC public school, both charter and DCPS, makes its own decisions about what PARCC test to administer to its students. This is in accord with OSSE guidelines that PARCC tests administered should match each school’s curriculum.

Of the aggregated scoring presented in public data available to parents exercising school choice, Kang noted that “you can draw some conclusions” even with the different tests administered across, and within, schools.

Just what those conclusions are, however, is potentially very misleading at the middle school level.

Right now, for instance, there is no explicit acknowledgement publicly from any city agency that some DC public middle schools take a different PARCC math test than others. The closest you can get is this chart embedded in this OSSE report on PARCC, on page 22.

That chart on page 22 shows aggregated test score data for these “advanced math assessments.”

But exactly which middle schools use what “advanced math assessments,” and what those scores are, remains publicly unavailable, except as what Caryn was able to find out through the data provided to her by the DME and by those spreadsheets above (here and here), which were derived from OSSE data available here. Even Superintendent Kang, when asked, did not appear to know which middle schools used which math PARCC test, though she opined that BASIS appears to be using a math test for its students that is different yet, with “integrated” math combining some geometry and algebra.

Hmm.

Even comparing scores just between schools that clearly are using the advanced math tests is difficult and misleading.

For instance, Hardy students scored 58% proficient in algebra, while Stuart-Hobson students taking the same test were just 24% proficient. But only 24 students at Hardy took that test, while 60 took it at Stuart-Hobson. Because Hardy and Stuart-Hobson have very similar demographics and enrollment, the larger number of test takers at Stuart-Hobson inevitably includes students who would not do as well on that test.

Or, putting it another way: if Stuart-Hobson limited students taking the algebra 1 test to only its top 24 math students, its score would be higher and, presumably, a bit more comparable to the score for that test at Hardy.(Although we have no way of knowing what the criteria were at Hardy for taking that test, either, but I am assuming it was the best math students.)

Kang noted yesterday that the ELA portion of PARCC doesn’t have this same problem as it is more “grade-specific” in its content than the math tests, so there are no differentiation issues with it across or within middle schools. She also noted that there are different tests at the high school level in ELA, but that there is obvious distinction about which schools are using what tests, so the reporting challenge for her is “significantly less” for ELA than for the math test issue at middle school.

Just what will be reported going forward for middle school PARCC math results remains to be seen. As an agency, OSSE has recently promulgated a policy of data transparency as part of its strategic plan for the next 2 years.

But the partners helping develop that strategic plan (see page 18 of that report) did not include the agency in charge of the public schools the majority of kids in DC attend, DCPS–nor any DCPS school.

Typo or omission?