Ranking our schools and giving out grades

What makes one high school better than another? Is it the school's atmosphere - a matter of striking a balance between fun and straightforward learning? Is it the teachers, the core of any educational environment? Or is it just an academic measurement, with test scores and Advanced Placement (AP) enrollment offering quantitative answers?

Everyone has an opinion on the matter, and while there's no exact formula to determining school success, that hasn't stopped education experts from trying to make one. But when high school ranking lists were released recently, principals, teachers, students and parents were understandably puzzled. The number one school on Newsweek's list - BASIS Charter in Tucson, Arizona - ranked 13 on the U.S. News and World Report's list. Newsweek's number two - a magnet-only high school in Dallas, Texas - fell to number 18 by U.S. News' standards.

So why the discrepancy? It's because the two lists see school achievement in different lights, with each publication creating unique standards to determine what qualifies as the cream of the crop.

If you ask the folks at Newsweek, using testing data from 2007, Blair is the 204th best public high school in the nation. While some in the Blair community may take this as a slight, it's far kinder than the verdict from U.S. News, which doesn't count Blair among the top at all. This difference results from the way the two lists are calculated.

Same data, different results

Let's start with Newsweek. Its list was developed 10 years ago by Washington Post education writer Jay Mathews, and it ranks schools by their Challenge Index (CI) rating. The CI is determined by dividing the number of AP or other college-level tests taken by the school's students by the number of graduating seniors. In other words, a school with a 1.000 rating averaged one test taken for every graduating senior, even if all the tests weren't taken by seniors. Any score above 4.000 is through the roof, easily placing in the nation's top 100. By comparison, Blair's 2.795 falls out of the top 200 by just 0.012 points. Schools with extremely high average SAT or ACT scores are excluded, because they do not represent the average public high school, says Mathews.

Mathews argues that the CI measures the school's willingness to allow average students to take challenging college-level courses. And at its core, the Newsweek list is just that - a measure of challenge. But because a cover story on the "Top U.S. High Schools" is likely to fly off the shelves much quicker than the same story about the "Most Academically Rigorous High Schools," few choose to clarify the ranking system.

The main criticism of the CI is the exclusion of passing rate data. The argument is that it's not the number of students who take challenging tests that is important; it's the number who do well on them. But Mathews believes that a high passing rate, such as Blair's 89 percent in 2007, indicates the exclusion of low-income or demographically-disadvantaged students from rigorous courses. And he says the same goes vice-versa - a low passing rate indicates inclusion, which he sees as an admirable quality that should not lower a school's ranking; everyone should be allowed to experience the "trauma" of college-level classes, says Mathews. But not everyone agrees with this logic: Dissatisfied with Mathews' formula, some in the education community turned away from Newsweek's list and, two years ago, came up with their own solution.

Enter Andrew Rotherham, co-founder of the D.C.-based think tank Education Sector, member of the Virginia State School Board of Education and former domestic policy advisor in the Clinton Administration. He says that Mathews' list lacks a modifier - it's actually the "best schools getting poor kids to take AP classes." The same schools that are ranked high by the CI can also have relatively low passing rates and high dropout rates, Rotherham explains. To prevent others from emulating top CI schools, Rotherham joined School Evaluation Services' Paul Gazzerro and education expert Michael Goldstein in developing the current U.S. News methodology. It first compares a school's performance on standardized state tests to statistical expectations, based on socioeconomic make-up and average state performance. Then, for every school that meets certain standards in these areas, a "college readiness" index is calculated using passing rates on college-level exams, and schools are ranked accordingly. The list does not exclude high-scoring schools, as Mathews' does, considering them all valid public schools. The U.S. News' list, although more complex, demonstrates a better understanding of what the public wants to read in a list of the nation's "best" high schools.

A closer look

The difference between the two rankings reflects varying opinions on a "good" high school's goals - striving for greater enrollment in college-level classes versus pushing students to pass college-level exams. By only using participation rates, the CI ignores the importance of a school's passing rate, which is a real indicator of how well the school has prepared its students. U.S. News corrects this particular mistake by including student performance on tests in its determination of the country's top schools.

In defense of his system, Mathews argues that counting passing rates would fill the list with affluent schools, where students have more resources to achieve better test scores. He's got a point. Take a look at local schools that cracked the top 100 on either or both of the two lists - Walt Whitman, Thomas S. Wootton, Winston Churchill, Richard Montgomery, Bethesda-Chevy Chase and Walter Johnson - all schools that draw from wealthier parts of the county. However, as Rotherham asserts, this phenomenon only highlights the necessity of improving the quality of low-income schools, preferably through policy reform.

Right now, both rankings are used by many schools as measures of success and reputation. In an effort to boost CI standings, some schools push their students into college-level classes for which they are not yet ready. Instead, the rankings should encourage schools to improve. Local governments and school systems should look at these numbers as measures of progress, or lack thereof, and work with schools to bring all students on level.

While the CI is effective in measuring how challenging a school is, "most challenging" is not "best." U.S. News provides the better list with inclusion of school performance, though any school's foremost concern should be improving education - not racing to the top.

If you would like to submit a letter to the editor, please send an e-mail to silver.chips.print@gmail.com. Include your name and comments concerning the story; we'd love to hear from you.