Deconstructing the U.S. News high school rankings

U.S. News recently released their first ranking of public elementary schools, a nasty concept nobody really asked for and our preteens don’t need. To place in its proper context the subjugation of our nation’s elementary schools to U.S. News’ version of “Am I hot or not?,” we look at their much-maligned high school rankings and show why alarm is warranted. 

The trouble with absolute performance

Seventy percent of the U.S. News ranking methodology is based on absolute performance metrics:  

• 30% is based on the percentage of students passing an AP / IB tests 
• 10% is based on the percentage of students that pass more than one AP /IB test 
• 20% is based on state standardized tests measuring math and reading  
• 10% is based on graduation rate 

There is a well-documented achievement gap between low- and higher-income student demographics when comparing standardized testing, AP testing, and graduation rate. This trend is amplified if low-income students belong to underserved minorities, namely Black and, to a lesser extent, Hispanic demographics.  For brevity’s sake, we will refer to these 3 demographics as “underserved.” 

See below 2 tables aggregating the income and racial median figures of the top 25 traditional and magnet schools on the U.S. News list. Income levels are based on student participation in free and reduced lunch programs (FRPL), which is offered to families with incomes below 130 and 180 percent of the poverty line respectively.  

All demographic and income figures based on NCES district data. If district details unavailable, census population estimates used.

The schools most highly rated by U.S. News 1) operate in wealthier, more homogenous sub districts, or 2) seem to handpick students that are wealthier and do not belong to underserved minority groups that statistically fare worse on testing. 

We cast no aspersions on the great facilities and teachers these schools likely have, but it is almost certain that their performance is boosted by a student body that is richer and more homogeneous than their districts. 

When asked for comment, Bob Schaeffer, the Executive Director of The National Center for Fair and Open Testing, had this to say:

“By relying almost exclusively on school rating factors that strongly correlate with family wealth, U.S. News guarantees that schools serving relatively affluent student bodies will rank at the top of its lists. In addition, the magazine’s formula assumes that standardized test scores are a fair and accurate measure of academic performance, rather than a narrow snapshot of one set of skills.  It is similar to trying to rank basketball teams based on their players’ free throw shooting percentages. Like most U.S News’ rankings schemes, this is another example of ‘Garbage in, garbage out’.”

And the trouble with predictive performance

U.S. News acknowledges the issues with absolute performance by devoting 30% of their scoring to predictive metrics meant to measure how underserviced groups perform at ranked schools. Their models are flawed and don’t really do that. 

According to U.S. News, 20% of the scoring is based on: 

… “a relative performance indicator that measures how high schools performed on state reading and mathematics assessments relative to the performance that would be expected given their proportion of students from underserved subgroups—defined as Black/African American students, Hispanic/Latinx students, and students who are eligible for FRPL or who are economically disadvantaged as determined by the state.” 

The high absolute performance of their non-underserved students would likely compensate for the scores, good or bad, of their relatively small number of underserved students. 

As described above, the most highly ranked schools will not be impacted much one way or the other.  For example, have a look at the U.S. News’ number 1 rated Thomas Jefferson High School.  TJHS has few minority— 1.7% Black, 3% Hispanic— and low-income students (1.8% FRPL), both in number terms and when compared to their district. The high absolute performance of their non-underserved students would likely compensate for the scores, good or bad, of their relatively small number of underserved students.  

If U.S. News were to instead isolate the performance of underserved groups within and outside of the school, we would still argue that having a small number of underserved students would be a ranking advantage if only because greater resources could be devoted to these pupils.  

The remaining 10% of this score is based on how underserved groups perform against non-underserved groups in each state, otherwise known as the “equity gap.” However, if these schools are not providing service to enough underserved students— less than 10 (not percent, the actual number) in all of the categories— the gap is set at 0, meaning there is no equity gap.  

This is the case with number 3 ranked Davidson Academy of Nevada. Judging from their “tied for first” score, they are granted full points for how they handle their (almost non-existent) underserved students.

“Non-white” as a cloak for diversity

It is outside the scope of this article to discuss the importance of diversity in education. Suffice it to say that many schools highlight their commitment to diversity and present themselves in such a light. Yet, sometime they are hiding the true picture under selective data. 

Suffice it to say that many schools highlight their commitment to diversity and present themselves in such a light. Yet, sometime they are hiding the true picture under selective data.

Degreechoices advisor Michael Nietzel recently detailed in Forbes worrying trends in the student body composition of Big Ten universities. While these universities boast of their growth in racial and ethnic diversity, this has largely been due to the growth in Asian and Hispanic representation. The overall percentage of Black students have been trending downwards over the last 20 years. 

This sleight-of-hand is apparent when examining the demographic makeup of U.S. News’ top high schools. 

Again looking at Thomas Jefferson High School as an example:  

And as represented by U.S. News: 

Student diversity

By defining diversity as non-white, disproportionate Asian representation masks the underrepresentation of Black, Hispanic, and lower–income demographics. 

Negative incentive structure

The rankings are heavily skewed towards schools with superior absolute performance in standardized testing, AP testing, and graduation rates. There is a positive correlation between wealthier demographics and superior performance on these tests, and this is very much borne out when examining the median demographics of the top-rated schools when compared to their districts.  

And, intentionally or not, U.S. News presents “school diversity” in a way that obfuscates the actual diversity, or lack thereof, in many of these schools.

On the other hand, “predictive” metrics meant to measure performance of a school’s underserved students are flawed. And, intentionally or not, U.S. News presents “school diversity” in a way that obfuscates the actual diversity, or lack thereof, in many of these schools. 

Best case, this negatively affects the way that students and their parents view the schools in which they are or should be enrolled, encouraging some to move, transfer, or otherwise act to amplify racial and income gaps. 

In the worst-case scenario, the U.S. News’ methodology incentivizes schools to recruit wealthier, more homogenous demographics to boost their rankings, thus wagging the dog.  

But yeah, go on and do elementary schools too.