The Case Against The Case Against Education: Pre-Employment Testing

In the continuing saga…wait. Before I dive in, I want to reiterate something. Sending fewer kids to college is an excellent goal. But we need a realistic case to argue, one that understands how we arrived at this point, what the pressures are to keep it this way, and what are realistic alternatives. Caplan’s 0 for 3. That’s irritating, particularly since Caplan is personally in favor of killing all public education funding, which I absolutely do not support. He makes all these wild statements while many reviewers go the chinstroker route, pretending to take him seriously but actually cherrypicking some of his arguments they agree  or disagree with–yes, this is all very interesting and we should think about it–without engaging with the consequences of his proposals.

The most common explanation for the deep emphasis on credentialing in America is  that employers use college degrees as a proxy for cognitive ability for fear that they’ll be sued. In most circles, this is referred to as the Griggs problem, for the Griggs v Duke Power decision. In the past, employers routinely gave cognitive ability tests for jobs not requiring college degrees, to ensure the applicant has a baseline ability level–or even just to hire the smartest candidate. However, the  Griggs decision severely constrained their ability to test employees if it resulted in a disparate impact by race or gender, so employers began using education credentials as a proxy for ability. Caplan calls it “IQ laundering”; take the IQ, stick it in college for a few years, and then hey, presto, that dirty cognitive ability has been converted into a shiny new, entirely legal, credential, since credentials are not held to the disparate impact ruling.

Caplan thinks IQ laundering proponents are wrong. He thinks it’s obvious that employers “fear” high IQ people who don’t go to college–it signals non-conformity and low conscientiousness. He argues that IQ laundering has to face an “awkward fact”:

10-30% of large employers admit they use cognitive ability tests. (page 89)

Then he continues:

“…the total number of employment discrimination cases filed in federal court peaked at about 23,000 in 1998, then gradually declined to about 14,000 in 2007. The average cash award if you win a trial is large–about 1.1 million for 1990-2000. But only 2% of plaintiffs acutally go to trial and win, so annual awards sum to less than $600 million. Most plaintiffs–58%–manage to get an out of court settlement. Settlements are usually confidential, but the average settlement is about 5% as large as the average trail award. Annual settlements therefore sum to less than $800 million. If plaintiffs’ lawyers work for a 40% contingency fee, and defense outpsends them by a factor of three, employers’ legal costs still sum to less than 1.7 billion. Updating these mid-1990s figures for inflation, employers’ total legal burden sums to under $5 billion per year.

Compared to total labor costs, $5 billion is trivial…[But] Only 4% of federal discrimination cases brought between 1987 and 2003 alleged disparate impact. That amounts to under a thousand annual cases against any form of employment testing. If disparate impact cases cost the usual amount, employers’ total test tax is under $200 million a year. (page 89)

So to restate, Caplan thinks employers aren’t interested in finding high IQ people, but only people who have managed to go through college, which presumably signals a decent IQ. Employers don’t have any interest in cognitive ability testing. If they did, they would, because the pittance they’d pay in lawsuits would dwarf the savings they’d find in high IQ workers. They don’t do this. Ergo, they don’t want high IQ workers. They want conforming conscientious folks.

So first, on the 10-30% of corporations testing. I actually heard about this argument several months before, on Twitter,  and called bullshit. I’m amazed no one else noticed. The article, The Benefit of a Degree in I-O Psychology or Human Resources, lists 2 prior surveys and does one of their own:

  • Terpstra, Rozelle, 1993:  201 companies, 20% did cognitive ability testing
  • Drogan, Yancy, 2011: 122 credit unions, 27% did cognitive ability testing
  • Wang, Yancy, 2012: 94 credit unions, 11% did cognitive ability testing.

I do not see how Caplan can use these three papers to assert that 10-30% of all corporations do cognitive ability testing. The papers themselves make no such claims.

Next, Caplan thinks that, since corporations spend billions in labor costs, they should shrug off a few hundred millions in court settlements in exchange for more efficient hiring. But labor costs will be in the billions no matter what. Suppose hiring the perfect employee every time saves employers collectively $1 billion each year.  Tests are expensive. Developing a test that will pass muster in the event someone sues would be extremely expensive. The tradeoff isn’t billions against $200 million, but more like $1 billion against $200 million and the cost of developing a test that passes EEOC in the event of a lawsuit. Morever, $200 million might be the total test tax for all corporations, but it’s not spread out among them evenly. Just ask Target ($2.8 million) or Federal Express ($54.9).

But the gaping hole in Caplan’s case is government hiring. The Civil Service exam was one of the great achievements of the late 19th century governance, but it didn’t last 100 years before the federal government abandoned it under pressure of a consent decree rather than lose at trial because of the test’s disparate impact. Teacher credential tests are routinely challenged for disparate impact and although they’ve been winning for 30 years, every so often a test is rejected for disparate impact and content that can’t be directly linked to the needs of the position.  But teachers have it easy next to  firefighters and cops–in no small part because firefighters and cops get promotions that have to be defensible and racially balanced.

Caplan doesn’t mention the extensive case history on government employment testing and disparate impact, possibly because he is unaware of it, possibly because it interferes with his easy, brief dismissal or, most likely, because he has some glib reason that he’ll use to argue in favor of ignoring it. But I find it difficult to justify his failure to take into account the hundreds of government cases on testing and disparate impact. The cases weren’t cheap, certainly, and it’s quite possible many large employers are scared off testing because of the many times courts have thrown out even carefully calibrated tests for seemingly random reasons. Toss that in with the $200 million “test tax” and the huge expense of developing a test against the likelihood of a loss–which happens to governments all the time, reminding corporations of what they could be wasting–and it’s far more reasonable, contra Caplan, to think that perhaps corporations don’t want the risk of cognitive ability testing.

Caplan occasionally mentions the “defenders of the IQ laundering theory”–those misguided souls who think Griggs had any sort of impact. For those looking for an excellent argument otherwise, see  Griggs vs. Duke Power: Implications for College Credentialing (O’Keefe/Vedder).  As Vedder and O’Keefe point out, employment tests were ubiquitous in this country before the Griggs decision. Now they’re very rare, other than in the EEOC-approved college credential path. In contrast to this history, Caplan’s simplistic, skeletal treatment of Griggs‘ potential impact on the rise of college credentialism undercuts his already weak argument for the employment value of conformity and conscientiousness.

Furthermore,  Caplan erred in saying that Griggs was codified in the Civil Rights Act of 1991. It was codified in the Equal Opportunity Employment Act of 1972, a distinction that matters because the court cases immediately following this codification supported Griggs. But  (Note: The Equal Opportunity Act didn’t codify Griggs, it just expanded the scope. I was misled by wording in the Vedder/O’Keefe piece. Thanks to Robert Verbruggen for pointing this out.)

But those of us blaming Griggs are focusing on the wrong target. In 1989, the Supreme Court threw out key elements of Griggs in a case known as Wards Cove , restoring the original 1964 understanding of the requirement.

Congress was much better at getting things done back then, and President Bush was running for re-election. So Teddy Kennedy proposed an amendment that resulted in the Civil Rights Act of 1991, overruling the Supreme Court’s decision and reinstating disparate impact.

This strikes me as important for a several reasons. First, it shows again that Caplan’s not a reliable narrator. He read the O’Keefe/Vedder article; it’s in his (excellent) bibliography. But he presented the information in such a way that utterly evades the complexity and controversy behind the disparate impact requirement.  Naw, it’s just like the 55 mph speed limit–a formality. Everyone breaks it. And that’s just not true.

Next, the Supreme Court appears to be once again forcing the conversation back towards equity and away from reverse discrimination–and this time, Congress and the President aren’t inclined towards swift action. What happens if disparate impact is ruled discriminatory in some future case?

Because, finally, Congress’s reaction to the Ward’s Cove decision overruled the Supreme Court, which doesn’t happen very often. What made this case special? Similarly, employers flagrantly violate all sorts of laws, but most of them are very leery of taking on the cognitive ability test and disparate impact. It’s almost as if institutionally, there might be a powerful counterforce pushing political and business leaders away from cognitive ability testing.

Hmm. What on earth could that counterforce be? But I’m at 1600 words already, so that’s the next article.

I can’t prove Caplan is wrong about employers and disparate impact. Caplan doesn’t take the long view, and it’s quite possible that today, given the ubiquitous nature of college attendance, employers do see failure to attend college as a sign of either low intellect or low conformity. But because Caplan elides or omits a great deal of importance from his argument, he makes the issue seem simple  when it clearly isn’t. Again, I don’t get the sense he’s making a serious case. Griggs wasn’t decided by stupid people. They had a reason for trying to stop employers from using cognitive ability as a hiring criterion.

I learned a great deal in fact-checking Caplan in this section. Most importantly, I learned that those of us who blame Griggs aren’t telling the whole story. Griggs was declared unconstitutional and then its elements were explicitly forced back into law by Congress and President Bush I. Disparate impact might not be similarly rescued in the future.

About educationrealist


20 responses to “The Case Against The Case Against Education: Pre-Employment Testing

  • rob

    I think this is a super-summary of all the main points.

  • RohanV

    The section on Griggs is excellent. I haven’t seen it detailed like this before.

    However, I don’t understand your argument with the “10-30% of corporations testing” cite. The surveys shows 20%, 27% and 11% which seems to fit Caplan’s statement of 10%-30%. What exactly are you disagreeing with? That the companies surveyed are not representative of industry at large?

    • educationrealist

      Thanks!

      On the second point: he said 10-30% of corporations say they test. I do not think three surveys of approximately 400 companies, 300 of them in the same field (and probably duplicated) comes even close to making that point. He didn’t say “three surveys have shown”. He states it as a fact. Maybe I should edit it to make that clearer, but the three papers he cite do NOT make that claim, so why should he?

      • RohanV

        I think this line of argument is weak and detracts from your case.

        Maybe Caplan is a bit too sweeping, but his point is that there are companies which still use cognitive ability tests, and those surveys back him up. To me, the fact that they’re credit unions is even more supportive. Financial companies usually have greater government oversight than normal. If anything, I would expect them to use less cognitive testing, if cognitive testing has shaky legality.

      • educationrealist

        I spend very little time on it.

  • Mark Roulo

    Something to keep in mind is that “cognitive tests” may NOT be IQ tests.

    Some random Googling turns up a presentation on how different organizations make hiring decisions and this presentation includes this entry:

        “Cognitive Tests (e.g., job knowledge tests) (N=809)”

    Note that asking someone to show very basic job knowledge (e.g. write a short program to do something for a computer programming job, formating a document using MS-Word for a secretary job) is not particularly illegal. An inability to do core parts of the job is still something companies can filter against when hiring.

    But this is NOT the same thing as an IQ test, which would be very much illegal unless enormous hoops were cleared. If a company wants to explicitly hire “smart people” there is still not good way to do this. Favoring Stanford grads is a proxy, but you can’t just test for smarts.

    So … without reading the actual papers (and maybe not even then) we can’t tell if the “cognitive tests” given by the organizations in these three papers are what folks who want to replace college diplomas with IQ tests mean by cognitive testing. There is a good chance that the testing in even these three studies is NOT what folks other than Caplan mean by “cognitive tests.”

  • Alvaro de Menard

    The strongest piece of evidence in Caplan’s favor is looking at other countries. Griggs only applies to the US. But we see similar issues with tertiary education in every developed country. I can tell you that in my field the use of IQ-like tests is routine in entry-level hiring, but that has not influenced the degree requirements one bit.

    • educationrealist

      Every other developed country tracks madly and has much higher standards for college than we do.

      • Alvaro de Menard

        Given the fact that the US isn’t even in the top 10 in terms of % of people with tertiary education I don’t see how that’s possible. Just look at Canada, where 58% of people aged 25-34 have degrees (vs 46% in the US).

      • educationrealist

        It’s not only possible, it’s well documented.

      • Roger Sweeny

        OECD has a site with numbers, but some of them look weird to me. 98.3% of Russian women will have a college degree by the time they die? 72% of Russian men will?

        https://data.oecd.org/eduatt/graduation-rate.htm

      • educationrealist

        Yeah, that’s incredibly weird.

      • Roger Sweeny

        Assuming it is true that “Every other developed country tracks madly and has much higher standards for college than we do”, there would seem to be less need for IQ tests in other developed countries. You could get a good idea of what an applicant can do from what track he graduated from and you would know that a college graduate was substantially smarter than average.

        If IQ tests are common overseas anyway, that suggests a real desire for them overseas. If there is, in some sense, a greater need for them in America but they aren’t used much, that suggests something is keeping companies from using them. That something could be fear of Griggs-like lawsuits, which would be based on the 1991 Civil Rights Act that overruled Wards Cove.

        Shocking how hard it is to determine how many companies violate the letter of a law 🙂

      • educationrealist

        Yes, you’re right. Tracking would obviate the need for IQ tests, if standards were maintained. It’s hard enough to find out what american companies do, never mind them.

        I get very annoyed at the fact that American employers ignore a lot of employment law, so long as they don’t think it puts them at risk of a class action suit.

  • Paul Rain

    If a professed ‘scholar’ like Caplan who believes in the overwhelming power of corporatism to achieve the best possible goals suggests you- as an employer, or as a worker- should do something that noone in their right mind is doing, he probably doesn’t have your best interests at heart.

  • Powerlurker

    There is still one major employer in the US that administers IQ tests for all new hires: the military. The AFQT is basically an IQ test and the military continues to use it because it turns out to be extremely useful for their goals.

  • Roger Sweeny

    I did a law school paper on Griggs not that long after it was decided, well before Ward’s Cove and the CRA of 1991. I’m really glad to see this update. Lots of people should read it (along with the comments of Mark Roulo and Powerlurker).

  • The Case Against The Case Against Education: How Well Are Americans Educated? | educationrealist

    […] of my seemingly endless review of Bryan Caplan’s The Case Against Education. Parts one, two, […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: