Criticism of Newsweek's Challenge Index is misplaced and misdirected. This is due to the limited background of critics who show no knowledge of a valuable analog: evolutionary theory. Consider two of stronger critics of Jay Mathew's work: Sara Mead and Andrew Rotherham. They succinctly write:
"A successful high school should show high levels of student achievement, graduate almost all of its students and not let any demographic subgroup suffer at the expense of others. Most national and local experts and policymakers share these values. To be sure, graduation rates and student achievement are hardly the only indicators of a school's quality. At a minimum, however, America's best high schools should be expected to meet these basic criteria.
"Yet our analysis shows that many schools on Newsweek's list do not meet these minimum standards."
Mead's and Rotherham's words appear well-chosen and difficult with which to disagree, but they are actually irrelevant and misleading for the simple reason that education is an individual achievement. Bureaucratic measures are not only meaningless, but often distorting to individual students. The efforts in education are simply those of multiple single individuals seeking the necessary trade-offs and efficiencies in teaching the many; and there are many types of many. Education is hard, but education is not policy. Forcing education to fit policy is a fool's game. There are many fools.
In 1966, simple Darwinism, which holds that evolution functions primarily at the level of the individual organism, was threatened by opposing concepts such as group selection, a popular idea stating that evolution acts to select entire species rather than individuals. George Williams's famous argument in favor of the Darwinists delivered the decisive response to those in opposing camps. His Adaptation and Natural Selection, now a classic of science literature, is a thorough and convincing essay in defense of Darwinism; its suggestions for developing effective principles for dealing with the evolution debate and its relevance to many fields outside biology ensure the timelessness of this critical work. Almost all Ed wonks ignore this. For example, they think that subgroup success and small "achievement gaps" on tests matter. Well it does for them and it sounds compassionate, but what does it matter to an individual student? Is it realistic to think that someone picks the a school because it has a small achievement gap, without asking "how much achievement or why does that matter to me?" Do Hispanic parents chose a school for their children because the Native-American subgroup did OK, without asking "what's OK or what does that matter to me?" Saying "meeting proficiency in state standards" truly begs many questions. People want a safe school with solid academics to maximize the chances of their children becoming successful. Everything else, outside of athletic considerations, is silly fluff for irrelevant-to-learning, but not to concerned bureaucrats in the education business. This can be nasty sounding. For example, a school with a high dropout rate isn't really an issue to successful students. It may actually be a plus: get rid of low-performing riffraff. This is what sport teams on campuses know also.
The Challenge Index simply and effectively helps parents and students pick schools from their perspective! Other measures may have some utility for education bureaucrats, but not for families. Bureaucratic measures shouldn't be mentioned in public because they lead to confusion like those from advocates of social group selection in the early 1960's.