U.S. News College Rankings Questioned

Atlanta Journal-Constitution

September 11, 2012

When U.S. News & World Report releases its annual college rankings Wednesday, students and parents will study the lists in an attempt to find the perfect school for them.

Few realize the largest single component of the rankings comes from subjective surveys that ask presidents, other administrators and, in some cases, high school guidance counselors to rate colleges.

Some colleges have long opposed the rankings, calling the surveys the most offensive part because it places too much emphasis on subjective information that can be manipulated. But U.S. News says the surveys are a logical part of the process because it asks those in higher education to analyze their peers.

Since 2007, nearly 65 college presidents, many from liberal arts schools, have signed a letter refusing to fill out the survey and have refused to use the rankings to promote their school. That’s a small portion of the nearly 1,600 colleges ranked, but supporters say other schools stopped completing the surveys, even though they haven’t signed the letter.

William Bloodworth, the former president of Augusta State University, and Ruth Knox, president of Wesleyan College in Macon, GA, signed the letter.

“It is a dubious instrument and just plain silly,” said William Durden, president of Dickinson College in Pennsylvania, who was one of the first to sign the letter. “Lots of colleges rate themselves supreme and put everyone else in the bottom. It’s a ridiculous process.”

Defending its product

The majority of college leaders and those at top-tier schools still complete the surveys. Brian Kelly, U.S. News editor and chief content officer, said it is a strong tool that addresses “intangibles” that can’t be measured by statistics. “Who better to evaluate colleges than other college administrators? They are experts,” he said.

Robert Morse, the magazine’s director of data research, acknowledged the criticism in an April 5 blog post about the questionnaire. “U.S. News knows that peer assessments are subjective, but a school’s reputation for academic quality is important to prospective students since they know it could help them get their first job after graduation and make a good impression if they are planning to apply to graduate school,” he wrote.

The group’s rankings are under additional scrutiny this year after Emory University disclosed last month it intentionally submitted inflated data to rating publications for more than a decade. Kelly said the false information would not have affected the No. 20 score Emory received last year.

U.S. News has published the rankings for nearly 30 years and averages 15 million page views on its website when its new ranking comes out. Many students and families rely on these rankings when deciding where to apply and enroll. For colleges, a strong rank attracts academic prestige and higher achieving students.

Joni Towles, a private college counselor who advises metro Atlanta families, said parents wrongly assume the rankings are just based on hard data. She tells them information can be found elsewhere with little effort.

“Considering how aware people are of the rankings, they know very little about how it’s calculated,” she said. “Parents are surprised when I tell them about the surveys. But, even after that, I still have to convince them to look beyond the ranking.”

The magazine calculates rankings using test scores, faculty salaries, per-pupil spending, alumni giving and other data that is publicly available. Survey results count between 22.5% and 25% of a school’s score.

A Clemson University administrator caused an uproar in 2009, when she said the South Carolina school gave below average marks to competitors to make the college look better. She doubted the college was the only one doing this. Kelly said they weight the results to eliminate excessively high or low scores and prevent people from gaming the system.

Opting out

Bloodworth said the first time he filled out the survey at Augusta State, he gave the college high marks. But it has about 15 years since he filled out the surveys. He questions their accuracy, noting college leaders are too busy running their own campuses to spend time studying other schools.

“Also, I didn’t like the idea of contributing my time to help a magazine make money over rankings that don’t tell people what they need to know,” said Bloodworth, who ended his 18-year presidency June 30.

Lloyd Thacker, founder of the Education Conservancy, pointed to a drop in survey response rates as evidence that people increasingly distrust the ranking. The group discourages families from focusing on the rankings and has called on presidents to boycott the surveys.

Last year’s participation rate was 43%, a drop of five percentage points from the previous year. That’s still considered statistically strong, although at one point the participation rate was more than 20 percentage points higher.

Durden said he expects the rankings to remain popular but that the college won’t change its opinion on the surveys. “If we represent academic integrity, we will not play a game of filling out silly forms and using rankings to advance our institutions,” he said. “We are supposed to represent firm logic, and this is not that.”