Latest Research: Tests of General Mental Ability are Flawed

Standardized tests in school have long been criticized as very imperfect measures of achievement. Reviews of actual coursework and grades earned in high school provide much better measures of what a student is capable of doing in college, which is why the University of California system stopped using SAT scores for college admissions. Last year at the American Education Research Association conference this was noted:

Especially troubling, the perception of the SAT as a test of basic intellectual ability had an adverse effect on many students from low-performing schools, tending to diminish academic aspiration and self-esteem. Low scores on the SAT were too often interpreted as meaning that a student lacked the ability to attend UC, notwithstanding his or her record of accomplishment in high school. These concerns prompted Atkinson (a former president of UC—PF) to propose in 2001 dropping the SAT in favor of curriculum-based achievement tests in UC admissions.

A thorough study from Indiana State University  now claims that all such tests of general mental ability, including employment and civil service tests, contain significant bias.

Overturning more than 40 years of accepted practice, new research proves that the tools used to check tests of "general mental ability" for bias are themselves flawed. This key finding from the Indiana University Kelley School of Business challenges reliance on such exams to make objective decisions for employment or academic admissions even in the face of well-documented gaps between mean scores of white and minority populations.

The study, published in the July issue of the Journal of Applied Psychology, investigated an amalgam of scores representing a vast sample of commonly used tests, including civil service or other pre-employment exams and university entrance exams.

..."Test bias" means that two people with different ethnicity or gender, for example, who have the same test score are predicted to have different "scores" on the outcome (e.g., job performance); thus a biased test might benefit certain groups over others. Decades of earlier research consistently found no evidence of test bias against ethnic minorities, but the current study challenges this established belief.

Many homeschoolers and alternative schoolers know that standardized tests don't truly reflect what they learned, but in order to make their nontraditional learning fit into conventional college slots they often take test prep courses, study test taking guides, pay for testing tutors, etc. so they can gain admission. Decades of research by Alfie Kohn and others, wonderful books such as "None of the Above" by David Owen, and the work of organizations like FairTest have fallen on deaf ears in the testing and schooling business. Perhaps, I hope, this will now change in light of this large study. This new research shows how unjust the sham "scientific and objective measurements" of our personal abilities and achievement are that we have been subjected to over the years.