Category Archives: testing

Advanced Placement Test Preferences: Asians and Whites

I just finished my AP US History survey course, and a glorious time it was. But I will save the specifics of my three to four hour lectures, and whether or not this is a good way to teach history for another post. I will also, hopefully, weigh in some time on what value add I think I bring to history. (If you’re curious, in public school I taught history of Elizabethan theater and a truly awesome 50s science fiction film course, in which students were to analyze the movie’s foreign policy approach by Walter Russell Mead’s paradigm.)

I always end my AP class by discussing the students’ course selections for next year. APUSH is a junior course, and I have about ten kids in this particular class, and the conversation is always the same.

“What are you taking?”

“Calc BC, AP Physics C, AP Bio, AP Gov.”

“You?

“AP Chem, AP Stats, AP Psych,…”

“So you’ve already taken BC?”

“Yeah, just took the test. Piece of cake. I’m taking intro to MVC.”

“What about AP English?”

All the heads shake. “God, no. Way too hard.”

One kid says “I’m taking AP Gov, I heard it’s easy.”

“I’m taking Macro Econ, one of our teachers has all the info you need to pass the tests.”

I laugh. “Jesus. Embrace the stereotype.”

They all get it and laugh, shamefacedly.

“Who’s taking AP English this year?” Two hands rose. “AP English next year?” No hands.

“So here’s what I don’t understand. You are all trying to get into college, and the reason you are taking these tough classes is to make yourself look good for colleges.”

“Sure.”

“And I see only Chinese, Korean, and Indian Americans in front of me, all either FOB or citizens with parents who lived most of their lives in China, Korea, or India. Moreover, as I imagine you’ve heard, and certainly your parents have heard, universities often engage in some form of discrimination against Asians.”

“Wow,” one of the students laugh-gasped. “I never thought I’d hear an American admit that.”

“An American, or a white person?”

“They aren’t the same?”

“You born here?” Pause, as I see that datapoint register. Yes. She’s an American. (We’ll leave aside the fact that they don’t consider blacks and Hispanics American, either. I’ve written about this before; it’s still weird to see.)

“Anyway. All of you avoid classes that involve reading literature or written analysis because they would be too difficult.”

“Well, yeah.”

“So the stereotype is all wrong.”

“What, the stereotype that says we’re good at science and math?”

“No, the stereotype that says you work hard, that you take on challenges.”

“Oooooh, SNAP.”

I smiled, too. “Look, there’s a serious point here. You’re a college admissions officer, reading through approximately 16 billion Asian resumes that all read exactly the same: 4.2 GPA, BC calculus as a sophomore ( with the occasional underachiever waiting until junior year), several AP science courses, APUSH for those of you who can string a sentence together, AP Chinese for those of you lucky enough to win the language lottery, and so on. What’s going to stand out? Not one more STEM course.”

“Yeah, but I hate reading.”

“You think the universities don’t know that? Oh, look, one more Asian kid who’s a machine at math and can memorize all the facts in AP Bio but uses Cliff notes for Hamlet. College admissions is a numbers game anyway, and I’m not pretending anything is going to make a huge difference,, but…”

“My dad says colleges are reducing Asians born here…American Asians [score!] for Chinese and Koreans.”

“Your dad’s right. So given all the work you’re putting in clearly to just get that last inch of consideration, may I suggest that the path to differentiation lies in showing the admissions reviewer that you take on challenges in all subjects, as opposed to taking classes you know you’ll get an A in.”

***********************************************

I was going to just post this little anecdote, but then I got to wondering just how prevalent the behavior is—it is exclusive to my little corner of the country, or are the recent Asian immigrants showing up in national data?

One of the problems with AP data is that you simply can’t make too many assumptions. For example, much has been written about the fact that the mode AP score for blacks is 1. Not only do most blacks fail the AP test, people wail, but they fail it completely! Twice as many blacks get a failing score as get a passing score! Our teachers are failing black children!

Yeah, no. The black AP population is a combination of at least three different groups. First, the group of genuinely qualified, academically prepared black students. Small group, I know, but each year hundreds of African American students take and pass the BC Calculus test, many with a score of 5 (however, 1 is still the mode for BC Calc). Second, the group of average or higher ability blacks with relatively little interest in academic success, who have nonetheless been put in AP classes by desperate suburban school officians who are under fire from the feds for their “opportunity gap” numbers. These are kids who could, with good teaching, achieve a respectable “3” on a number of tests, and probably do.

The problem, alas, is that a teacher can focus on getting middle achievers over the hump, or on challenging a bunch of smart kids. Can’t do both in the same room, not easily and probably not at all. Thus bringing in more marginal black students and coaxing them to a three occasionally has a depressing effect on suburban AP scores, as the top white kids aren’t being taught at the top of their ability. But I digress.

The third group, and it’s huge, are low income urban and charter schools gaming the GPA and Jay Mathews Challenge Index. These are kids who are barely literate, often aren’t even taught the course material, but boy, by golly if they get the butts in the seats they’ll show up on Jay’s list somewhere. All at taxpayer expense.

While the AP tests results disaggregate Mexicans and Puerto Ricans from the rest of Hispanics, Mexican performance has the same conflation of three groups as black results do, and are equally useless. The Hispanic mode score is also one.

Asian scores aren’t disaggregated, but the Big Three (Chinese, Koreans, and Indians) dominate.

So are Asians showing a preference for science and math over the humanities AP tests?

AP testing populations by race–mostly. It would have been a huge hassle to add up all the URM categories, so I just subtracted whites and Asians from the total. So “Decline to state” is categorized as a URM, when it’s probably mostly white. I checked a couple values, it wasn’t a big difference. These are the top 20 tests by popularity, in order from left to right1.

2013aptablebyrace

The visual display is useful—look for big green, little blue, or a relatively high number of URMs, fewer Asians. See? Asians live the stereotype. Don’t assume that blacks and Hispanics are drawn to the Humanities courses—it’s just easier for schools to shove unprepared kids into English, Geography, and History classes than it is to science and math courses. Fewer prerequisites.

Here’s the same data in table form. I added one column, Asians as a percentage of the Asian/white total, to clear away the URM noise. Then I highlighted the tests for each column that were more than one average deviation away from the mean, both higher and lower (I used average deviation because I don’t think these distribution have a standard one. Could be wrong, but that’s why the choice). I bolded any values that were more than two average deviations away from mean.
2013aptablebyrace

Whites are the most tightly clustered, URMs next. Asians tilt strongly towards and against.

There’s a lot more to explore here, and I hope to do that soon. But for now, I wanted to stay focused on Asian vs. white preferences. So I next compared the top 20 Asian test preferences to those of whites. (Actually, I did 22 for Asians because I thought #22 was revealing.)

AP totals include many multiple testers, so I took the number of testers for any given test as a percentage of the total for that race. This is not a perfect measure, for obvious reasons. Or maybe not so obvious. Say, for example, that an entirely different group of Asians take the English Lit test than take the Calc AB test, but the white students have a significant overlap. In that case, the percentage of testers would be saying something entirely different about each group than if both Asians and whites had overlapping testers.

However, in either case, it would be revealing. If more whites than Asians took both math and English tests, or if one group of Asians took math tests and another group took English (or the same case of whites), the percentages are still showing a preference. I think. I’m sure there’s a way to describe this more technically, but it’s late, the school year’s almost over, so put the correct text in comments and I’ll change it.

Anyway.

whitestop20ap

asiantop22

And here it is graphically, ranked again by test popularity. The blue and green columns are the percentage of white or Asian testers taking that test. The graph above was percent of each test population that was white/Asian/URM. These columns show the percent of white or Asian population taking that particular test (the blue column “% of total” in the tables immediately above). The line graph is the percent of each group that scored a 5 on that test.

apasianwhitepref

(You notice something weird? Spanish is the tenth most popular test–but it barely makes the top 20 for either whites or Asians. How could that be? Who on earth is taking all those Spanish tests?)

So again, I want to write more about these results but I thought I’d put them out there and let people chew on them. Here’s a few preliminary observations:

  • Whites appear to be the utility players, good in a number of subjects and not expressing huge preferences. They are stretching more into STEM than Asians stretch into writing.
  • Asians appear to be avoiding writing-intensive tests relative to whites, no matter how you interpret the data.
  • Asians tend to choose tests that are more likely to yield high scores, and avoid tests that give out fewer 5s. Until recently, AP Bio doled out 5 scores like candy; they clearly changed scoring in some significant way this year (without announcing it, I guess). Environmental Science, which has a deservedly crappy rep, is actually pretty hard to get a high score on, so Asians avoid.
  • The real difference between Asians and whites in both preferences and scores is in the science tests, not math. Asians have higher scores in all tests—and while that’s probably a reflection of cognitive ability, you really can’t understand the difference in preparation and grinding until you see it—but the real gaps are in the sciences. AP Science courses are, in my opinion, pretty horrible to begin with. Yes. It’s the subject I don’t teach. Bias alert.

TL, DR: Asians across the land reflect the same biases. They may or may not be working hard, but they appear to be avoiding subjects that are more difficult for them, and don’t yield as high a score. This may also be why they avoid the ACT. Or not.

More on this later. Let me know what you think and of course, point out any errors.

1I actually did this work from the bottom up. So in the first chart, which was actually the last one I did, there are only 19 tests. Guess which one I left off, and why. The other charts all have 20 tests.


Finding the Bad Old Days

Michael Petrilli wrote an extremely aggravating article suggesting we tell unqualified kids they aren’t ready for college and go to CTE and then a much improved follow up that acknowledges the racial reality of his idea.

In his first piece, Petrilli only mentions race once:

PetrilliCTEquote3

This is a common trope in articles on tracking, a nod to “the bad old days” right after the end of segregation, that time immediately after Brown and ending sometime in the late 70s, or when Jeannie Oakes excoriated the practice in Keeping Track.

In the bad old days, the story goes, evil school districts, eager to keep angry racist white parents from fleeing, sought a means of maintaining segregation despite the Supreme Court decision and the Civil Rights Act. So they pretended to institute ability grouping and curriculum tracks, but in reality, they used race. That way the district could minimize white flight and still pretend to educate the poor and the brown. That’s why so many brown kids were in the low ability classes, and that’s why so many lawsuits happened, because of the evil racist/classist methods of rich whites keeping the little brown people down.

The bad old days are a touchstone for anyone proposing an educational sorting mechanism. So you have Petrilli advocating a return to tracking, who tell us the bad old days are a thing of the past: yeah, we used to track by race and income, pretending to use ability, but we’ve progressed. Districts pretended to use IQ, but they were really using culturally biased tests to commit second-order segregation. Today, we understand that all races and all incomes can achieve. Districts don’t have to distort reality. The bad old days are behind us, and we can group by ability secure that we aren’t discriminating by race.

Before ed school, I accepted the existence of the bad old days, but then I noticed that every reading asserted discrimination but didn’t back it up with data. Since ed school, I’d occasionally randomly google on the point, looking for research that established discriminatory tracking back in the 60s and 70s. And so the Petrilli article got me googling and thinking again. (What, buy books? Pay for research? Cmon, I’m a teacher on a budget. If it’s damning, the web has it.)

I first reviewed Jeannie Oakes, reaffirming that Oakes holds tracking itself, properly applied, as the operative sin. Discriminatory tracking isn’t a main element of Oakes’ argument, although she points out that “some research” suggests it occurred. Oakes’ third assumption, that tracking is largely made on valid decisions (page 4) is accepted at face value. So the grande dame of the anti-tracking movement has completely neglected to mention the bad old days—which, at that time, would have been contemporary.

On I move to Roslyn Mickelson, who does charge Charlotte Mecklenburg schools with discriminatory tracking.

I wasn’t much impressed for a number of reasons–not least of which a big error, which you can see here (lines 8-9):

mickelson5

Not unusual, apparently. In Capacchione v Charlotte-Mecklenburg, Judge Richard Potter eviscerates her expert testimony, finding faults with her credibility, her accuracy, and her logic.

Bottom line, however, Mickelson’s research shows that high achieving scorers in year one are not consistently placed in high achieving classes six years later. While both whites and blacks with high scores end up in low tracks and vice versa, more whites get high placement than blacks. But generally, her data shows something I’ve documented before, that achievement falls off each year because school gets harder.

Both whites and blacks experience the falloff, even though Mickelson seems to think that the pattern should be linear. The achievement scale simply gets larger as kids move up in grade levels, and fewer blacks make the top tier. This is consistent with cognitive realities.

There might be a smoking gun in research. But I couldn’t find it.

Then I suddenly realized duh, what about case law? If districts were tracking by race, there’d be a lawsuit.

I started with three legal articles that discussed tracking case law: 1, 2 and 3. They were all useful, but all failed to mention a significant case in which the district routinely used different standards or sorted directly by race or zip code.

From these articles, I determined that Hobson vs. Hanson was the original tracking case, and that the McNeal standard was for many years (and may still be) the test for ability grouping.

So I created a reading list of cases from the late 60s to the early 90s:

Only two of these cases involved schools directly accused of using race to sort students. In Johnson v. Jackson, the schools were forced to integrate in the middle of a school year. The black kids were ported over to white schools and the classes kept intact. The court ordered them to fix this. From first integration order to the fix order: 4 months.

The second case, Rockford, was decided in the early 90s, and the judge directly accuses the district of intentionally using race to ability group. However, Jeannie Oakes was the expert witness, and the judge drank every bit of Koolaid she had to offer and licked the glass. Oakes is presented as an expert witness, with no mention that she’s an anti-tracking advocate. Her testimony appears to be little more than readings from her book and some data analysis.

The proof of “intentional racism” was pretty weak and largely identical to Mickelson’s described above. Major difference: the judge accepted it.

Leaving aside these two cases, I couldn’t find any case in which the district was found to misuse the results of the test, either by using different racial standards or ignoring the tests entirely. The tests themselves were the issue.

In the south, school systems that weren’t “unitary” (that is, were previously segregated districts) couldn’t use ability testing. Since blacks would have lower scores based on past racial discrimination, the use of tests was discriminatory, an intent to segregate.

For school systems that were found to be unitary, ability testing isn’t in and of itself invalid and racial imbalance isn’t a problem (see Starkville case for example).

In all these cases, I couldn’t find a district that was tracking by race. They were guilty of tracking by test. Everyone knew the tests would reveal that blacks would have lower ability on average, and therefore ability grouping was by definition invalid in previously segregated schools. This was an era in which judges said “The court also finds that a Negro student in a predominantly Negro school gets a formal education inferior to the academic education he would receive, and which white students receive, in a school which is integrated or predominantly white.” (Hobson)

Once the system is declared unitary, or that was never an issue, the record is mixed. When judges did accept the results as valid, they ruled in favor of the school districts (Starkville, Hannon). In Pase v Hannon, the judge actually reviewed the test questions himself and determined they were unbiased with few exceptions, all of which were far above the IQ level in question.

In California, on the other hand, where de jure segregation wasn’t an issue*, the mere existence of racial imbalance was still a problem (Pasadena, Riles). In Riles, Judge Robert Peckham banned all IQ testing of blacks in California for educational purposes. He later extended the ruling even if black parents requested testing, but later withdrew that order. Peckham’s reasoning is much like the other judges who believed in cultural bias:

Even if it is assumed that black children have a 15 percent higher incidence of mild mental retardation than white children, there is still less than a one in a million chance that a color-blind system would have produced this disproportionate enrollment. If it is assumed that black children have a 50 percent greater incidence of this type of mental retardation, there is still less than a one in 100,000 chance that the enrollment could be so skewed towards black children.

Notice the reasoning: of course it’s not possible that blacks have a 50% greater incidence of an IQ below 75. Except it’s worse than that.

This image is from The Bell Curve (borrowed from here) reflecting the frequency of black/white IQ distribution:

BCFreqblkwhiteIQ

As many blacks as whites populate the sub 75 IQ space, but the population distribution being what it is, blacks are far more likely to have low IQs.

When Charles Murray researched this for The Bell Curve:

In the NLSY-79 cohort, 16.8 percent of the black sample scored below 75, using the conversion of AFQT scores reported in the appendix of TBC and applying sample weights. The comparable figure for non-Latino whites was 2.2 percent. In the NLSY-97 cohort, the comparable figures were 13.8 percent for blacks and 2.7 percent for non-Latino whites.

(Charles Murray, personal communication)

So at the time of Peckham’s decision, blacks didn’t have a 50% higher chance of an IQ below 75, but rather a several hundred percent higher chance, a chance that is still in the triple digits today.1 Peckham couldn’t even begin to envision such a possibility, and so no IQ testing for blacks in California.

(As for the lower frequency of blacks in the “trainable” mentally retarded division, as it was called then, an interesting but rarely discussed fact: Low IQ blacks are often higher functioning that low IQ whites. They are less likely to be organically retarded, and more likely to be capable of independent living. This despite the fact that their IQ tests and academic outcomes are identical. Arthur Jensen discovered this phenomenon, and I highly recommend that article; it’s fascinating. I wonder if the difference is somehow related to crystallized vs. fluid intelligence, but haven’t read up enough on it.)

So there it is. Obviously, if I missed a key case in which a major district was found to have deliberately tracked kids by race, please let me know.

But despite extensive efforts, I couldn’t find the bad old days of discriminatory sorting. What I found, instead, was a judicial rejection of IQ and other ability tests, coupled with an inability to conceive of the actual distribution patterns of cognitive ability.

Please understand my limited objective. Many Southern districts did everything they could to avoid integration. See, for example, US v Tunica, where the school tried to assign students based on test scores, but were denied because of the achievement testing ban and required to reassign students and teachers to achieve integration. The teachers refused assignment to integrated schools and resigned, white parents withdrew their kids, then the white schools set up shop at local churches, classes largely intact. Money? Not an issue. They used taxpayer dollars, since the district paid the teachers who resigned and the kids took all their school books with them.

But believe it or not, there’s no mention that the district was only pretending to use test scores, actually assigning students by race. And this is a place where I’d expect to find it. Opposition to integration, absolutely. Achievement testing used as a way to minimize racially mixed classes? Sure.

In many other cases, schools or districts instituted tracking as a genuine attempt to educate a much wider range of abilities, or even had a tracking system in place before integration.

The inconvenient realities of cognitive ability distribution being what they are, the test scores would be depressingly indifferent to intent.

Then there’s the messy middle, the one that Mickelson probably found in Charlotte and Oakes found in Rockford and any one looking at my classrooms would find as well. All tracked classrooms are going to have inconsistencies, whether the schools use tests, teacher recommendations, or student choice. The honors classes fill up or a teacher suddenly dies or all sorts of other unforeseen situations mean some kids get moved around and it’s a safe bet high income parents bitch more about wrong assignments than poor parents. Go through each high score in a “regular” class and each low score in a tracked, and each one of those test scores will have a story—a story usually doesn’t involve race or malign intent. The story occasionally does involve bad teachers or district bureaucracy, but not as often as you might think.

Teacher recommendations are supposed to mitigate the testing achievement gap but teachers are moralists, particularly in math, as I’ve written before. It doesn’t surprise me that new study shows that controlling for performance, blacks are less likely to be assigned to algebra as 8th graders by teacher recommendation. I can’t tell you the number of bright Hispanic and black kids I’ve run into (as well as huge number of white boys, including my son) who don’t bother with homework and have great test scores. So their GPA is 2.7, but their test scores are higher than the kids who got As–and the teacher recommendations.

Parents: some parents insist that their kids need to be in the top group to be challenged. Others feel that their kids do better when they feel secure, able to manage the challenge. Then there are the parents who don’t give a damn about their kids’ abilities but don’t want them in a noisy classroom with kids who don’t give a damn about education. White and Asian parents are disproportionately represented in the first group, black and Hispanic parents take up more than their share in the second, and all parents of all races worry about the last.

So let’s stop using teacher recommendation, stop allowing parents or students to ask for different placement. Test scores are destiny.

But test scores today still reflect the same reality that the judges assumed, back then, could only be caused by racism or bias.

The tests haven’t changed. The kids haven’t changed much.

The judges are another story.

Richard Posner, in a much-quoted 1997 decision on an appeal to the People Who Care v Rockford did what he has done before–made my point with much greater efficiency:

Tracking is a controversial educational policy, although just grouping students by age, something no one questions, is a form of “tracking.” Lawyers and judges are not competent to resolve the controversy. The conceit that they are belongs to a myth of the legal profession’s omnicompetence that was exploded long ago. To abolish tracking is to say to bright kids, whether white or black, that they have to go at a slower pace than they’re capable of; it is to say to the parents of the brighter kids that their children don’t really belong in the public school system; and it is to say to the slower kids, of whatever race, that they may have difficulty keeping up, because the brighter kids may force the pace of the class. …

Tracking might be adopted in order to segregate the races. The well-known correlation between race and academic performance makes tracking, even when implemented in accordance with strictly objective criteria, a pretty effective segregator. If tracking were adopted for this purpose, then enjoining tracking would be a proper as well as the natural remedy for this form of intentional discrimination, at least if there were no compelling evidence that it improves the academic performance of minority children and if the possible benefits to the better students and the social interest in retaining them in the public schools were given little weight. The general view is that tracking does not benefit minority students…although there is evidence that some of them do benefit… All this is neither here nor there. The plaintiffs’ argument is not that the school district adopted tracking way back when in order to segregate the schools. It is that it misused tracking, twisting the criteria to achieve greater segregation than objective tracking alone would have done. The school district should be enjoined from doing this not, on this record, enjoined from tracking.

The Charlotte-Mecklenburg case mentioned above cited Posner’s reasoning. The third of my case law articles discusses Holton v Thomasville II, which doesn’t mention Posner but does say that racial imbalance in ability grouping isn’t of itself evidence of discrimination, and points out that the time for judicial interference in educational decisions is probably over:

holtoncase

Most districts ended tracking out of fear of lawsuits. It may be time for parents to demand more honors classes, test the limits.

So what does this have to do with Petrilli? Well, less than it once did, now that Petrilli has acknowledged the profound racial implications of his suggestion.

But if the bad old days of racial tracking never really existed, then Petrilli can’t pretend things will be better. Yes, we must stop devaluing college degrees, stop fooling kids who have interest but no ability in taking on massive loans that they can never pay off. And with luck even Petrilli will eventually realize as well that we have to stop forcing kids with neither interest nor ability to sit in four years of “college preparation” courses feeling useless.

So what comes next? Well, that’s the question, isn’t it?

*************************
*Commenter Mark Roulo points out that California did commit de jure segregation against Hispanics and was ordered to stop in Mendez v. Westminster. See comments for my response.

1See Steve Sailer’s comment for why black IQs might have been biased against lower IQ blacks and the 97 data more representative.


NAEP TUDA Scores—Detroit isn’t Boston

So everyone is a-twitter over NAEP TUDA (Trial Urban District Assessment) scores. For those who aren’t familiar with The Nation’s Report Card, the “gold standard” of academic achievement metrics, it samples performance rather than test every student. For most of its history, NAEP only provided data at the state level. But some number of years ago, NAEP began sampling at the district level, first by invitation and then accepting some volunteers.

I don’t know that anyone has ever stated this directly, but the cities selected suggest that NAEP and its owners are awfully interested in better tracking “urban” achievement, and by “urban” I mean black or Hispanic.

I’m not a big fan of NAEP but everyone else is, so I try to read up, which is how I came across Andy Smarick‘s condemnation of Detroit, Milwaukee, and Cleveland: “we should all hang our heads in shame if we don’t dramatically intervene in these districts.”

Yeah, yeah. But I was pleased that Smarick presented total black proficiency, rather than overall proficiency levels. Alas, my takeaway was all wrong: where Smarick saw grounds for a federal takeover, I was largely encouraged. Once you control for race, Detroit looks a lot better. Bad, sure, but only a seventh as bad as Boston.

So I tweeted this to Andy Smarick, but told him that he couldn’t really wring his hands until he sorted for race AND poverty.

He responded “you’re wrong. I sorted by race and Detroit still looks appalling.”

He just scooted right by the second attribute, didn’t he?

Once I’d pointed this out, I got curious about the impact that poverty had on black test scores. Ironic, really, given my never-ending emphasis on low ability, as opposed to low income. But hey, I never said low income doesn’t matter, particularly when evaluating an economically diverse group.

But I began to wonder: how much does poverty matter, once you control for race? For that matter, how do you find the poverty levels for a school district?

Well, it’s been a while since I did data. I like other people to do it and then pick holes. But I was curious, and so went off and did data.

Seventeen days later, I emerged, blinking, with an answer to the second question, at least.

It’s hard to know how to describe what I did during those days, much less put it into an essay. I don’t want to attempt any sophisticated analysis—I’m not a social scientist, and I’m not trying to establish anything certain about the impact of poverty on test scores, an area that’s been studied by people with far better grades than I ever managed. But at the same time, I don’t think most of the educational policy folk dig down into poverty or race statistics at the district level. So it seemed like it might be worthwhile to describe what I did, and what the data looks like. If nothing else, the layperson might not know what’s involved.

If my experience is any guide, it’s hard finding poverty rates for children by race. You can get children in poverty, race in poverty, but not children by race in poverty. And then it appears to be impossible to find enrolled children in a school district—not just who live in it, which is tough enough—by poverty. And then, of course, poverty by enrollment by race.

First, I looked up the poverty data here (can’t provide direct links to each city).

But this is overall poverty by race, not child poverty by race, and it’s not at the district level, which is particularly important for some of the county data. However, I’m grateful to that site because it led me to American Community Survey Factfinder, which organizes data by all kinds of geographic entities—including school districts—and all kinds of topics–including poverty—on all sorts of groups and individuals—including race. Not that this is news to data geeks, which I am not, so I had to wander around for a while before I stumbled on it.

Anyway. I ran report 1701 for the districts in question. If I understand googledocs, you can save yourself the trouble of running it yourself. But since the report is hard to read, I’ll translate. Here are the overall district black poverty rates for the NAEP testing regions:

ACSdistrictblkpoverty

Again, these are for the districts, not the cities.

(Am I the only one who’s surprised at how relatively low the poverty rates are for New York and DC? Call me naïve for not realizing that the Post and the Times are provincial papers. Here I thought they focused on their local schools because of their inordinately high poverty rates, not their convenient locations. Kidding. Kind of.)

But these rates are for all blacks in the district, not black children. Happily, the ACS also provides data on poverty by age and race, although you have to add and divide in order to get a rate. But I did that so you don’t have to–although lord knows, my attention to detail isn’t great so it should probably be double or triple checked. So here, for each district, are the poverty rates for black children from 5-17:

ACSblk517poverty

In both cases, Boston and New York have poverty rates a little over half those of the cities with the highest poverty rates—and isn’t it coincidental that the four cities with the lowest black NAEP scores have the highest black poverty rates? Weird how that works.

But the NAEP scores and the district data don’t include charter or private schools in the zone, and this impacts enrollment rates differently. So back to ACS to find data on age and gender, and more combining and calculating, with the same caveats about my lamentable attention to detail. This gave me the total number of school age kids in the district. Then I had to find the actual district enrollment data, most of which is in another census report (relevant page here) for the largest school districts. The smaller districts, I just went to the website.

Results:

naepdistenrollrate

Another caveat–some of these data points are from different years so again, some fuzziness. All within the last three or four years, though.

So this leads into another interesting question: the districts don’t report poverty anywhere I can find (although I think some of them have the data as part of their Title I metrics) and in any event, they never report it by race. I have the number and percent of poor black children in the region, but how many of them attend district schools?

So to take Cleveland, for example, the total 5-17 district population was 67,284. But the enrolled population was 40871, or 60.7% of the district population.

According to ACS, 22,445 poor black children age 5-17 live in the district, and I want an approximation of the black and overall poverty rates for the district schools. How do I apportion poverty? I do not know the actual poverty rate for the district’s black kids. I saw three possibilities:

  1. I could use the black child poverty rate for the residents of the Cleveland district (ACS ratio of poor black children to ACS total black children). That would assume (I think) that the poor black children were evenly distributed over district and non-district schools.
  2. I could have take the enrollment rate and multiplied that by the poor black children in ACS—and then use that to calculate the percentage of poor kids from blacks enrolled.
  3. I could assign all the black children in poverty (according to ACS) to the black children enrolled in the district (using district given percentage of black children enrolled).

Well, the middle method is way too complicated and hurts my head. Plus, it didn’t really seem all that different from the first method; both assume poor black kids would be just as likely to attend a charter or private school than they would their local district school. The third method assumes the opposite—that kids in poverty would never attend private or charter schools. This method would probably overstate the poverty rates.

So here are poverty levels calculated by methods 1 and 3–ACS vs assigning all the poor black students to the district. In most cases, the differences were minor. I highlight the districts that have greater than 10 percentage points difference.

naepweightingpov

Again, is it just a coincidence that the schools with the lowest enrollment rates and the widest range of potential poverty rates have some of the lowest NAEP scores?

Finally, after all this massaging, I had some data to run regression analysis on. But I want to do that in a later post. Here, I want to focus on the fact that gathering this data was ridiculously complicated and required a fair amount of manual entry and calculations.

If I didn’t take the long way round, I suspect this effort is why researchers use the National Student Lunch Program (“free and reduced lunch”) as a poverty proxy.

The problem is that the poverty proxy sucks, and we need to stop using it.

Schools and districts have noticed that researchers use National School Lunch enrollment numbers as a proxy for poverty, and it’s also a primary criterion for Title I allocations. So it’s hard not to wonder about Boston’s motives when the district decides to give all kids free lunches regardless of income level, and whether it’s really about “awkward socio-economic divides” and “invasive questions”. The higher the average income of a district’s “poor” kids, the easier it is to game the NCLB requirements, for example.

Others use the poverty proxy to compare academic outcomes and argue for their preferred policy, particularly on the reform side of things. For example, charter school research uses the proxy when “proving” they do a “great job educating poor kids” when in fact they might just be skimming the not-quite-as-poor kids and patting themselves on the back. We can’t really tell. And of course, the NAEP uses the poverty proxy as well, and then everyone uses it to compare the performance of “poor” kids. See for example, this analysis by Jill Barshlay, highlighted by Alexander Russo (with Paul Bruno chiming in to object to FRL as poverty proxy). Bruce Baker does a lot of work with this.

To see exactly how untrustworthy the “poverty proxy is”, consider the NAEP TUDA results broken down by participation in the NSLP.

naepfrlelig

Look at all the cities that have no scores for blacks who aren’t eligible for free or reduced lunch: Boston, Cleveland, Dallas, Fresno, Hillsborough County, Los Angeles, Philadelphia, and San Diego. These cities apparently have no blacks with income levels higher than 180% of poverty. Detroit can drum up non-poor blacks, but Hillsborough County, Boston, Dallas, and Philadelphia can’t? That seems highly unlikely, given the poverty levels outlined above. Far more likely that the near-universal poverty proxy includes a whole bunch of kids who aren’t actually poor.

In any event, the feds, after giving free lunches to everyone, decided that NSLP participation levels are pretty meaningless for deciding income levels “…because many schools now automatically enroll everyone”.

I find this news slightly cheering, as it suggests that I’m not the only one having a hard time identifying the actually poor. Surely this article would have mentioned any easier source?

So. If someone can come back and say “Ed, you moron. This is all in a table, which I will now conveniently link in to show you how thoroughly you wasted seventeen days”, I will feel silly, but less cynical about education policy wonks hyping their notions. Maybe they do know more than I do. But it’s at least pretty likely that no one is looking at actual district poverty rates by race when fulminating about academic achievement, because what I did wasn’t easy.

Andy Smarick, at any rate, wasn’t paying any attention to poverty rates. And he should be. Because Detroit isn’t Boston.

This post is long enough, so I’ll save my actual analysis data for a later post. Not too much later, I hope, since I put a whole bunch of work into it.


Algebra 1 Growth in Geometry and Algebra II, Spring 2013

This is part of an ongoing series on my Algebra II and Geometry classes. By definition, students in these classes should have some level of competence in Algebra I. I’ve been tracking their progress on an algebra I pre-assessment test. The test assesses student ability to evaluate and substitute, use PEMDAS, solve simple equations, operate with negative integers, combine like terms. It tiptoes into first semester algebra—linear equations, simple systems, basic quadratic factoring—but the bulk of the 50 questions involve pre-algebra. While I used the test at my last school, I only thought of tracking student progress this year. My school is on a full-block schedule, which means we teach a year’s content in a semester, then repeat the whole cycle with another group of students. A usual teacher schedule is three daily 90-minute classes, with a fourth period prep. I taught one algebra II and one geometry class first semester (the third class prepared low ability students for a math graduation test), their results are here.

So in round two, I taught two Algebra 2 courses and one Geometry 10-12 (as well as a precalc class not part of this analysis). My first geometry class was freshmen only. In my last school, only freshmen who scored advanced or proficient on their 8th grade algebra test were put into geometry, while the rest take another year of algebra. In this school, all a kid has to do is pass algebra to be put into geometry, but we offer both honors and regular geometry. So my first semester class, Geometry 9, was filled with well-behaved kids with extremely poor algebra skills, as well as a quarter or so kids who had stronger skills but weren’t interested in taking honors.

I was originally expecting my Geometry 10-12 class to be extremely low ability and so wasn’t surprised to see they had a lower average incoming score. However, the class contained 6 kids who had taken Honors Geometry as freshmen—and failed. Why? They didn’t do their homework. “Plus, proofs. Hated proofs. Boring,” said one. These kids knew the entire geometry fact base, whether or not they grokked proofs, which they will never use again. I can’t figure out how to look up their state test scores yet, but I’m betting they got basic or higher in geometry last year. But because they were put into Honors, they have to take geometry twice. Couldn’t they have been given a C in regular geometry and moved on?

But I digress. Remember that I focus on number wrong, not number right, so a decrease is good.

Alg2GeomAlg1Progress

Again, I offer up as evidence that my students may or may not have learned geometry and second year algebra, but they know a whole lot more basic algebra than they did when they entered my class. Fortunately, my test scores weren’t obliterated this semester, so I have individual student progress to offer.

I wasn’t sure the best way to do this, so I did a scatter plot with data labels to easily show student before/after scores. The data labels aren’t reliably above or below the point, but you shouldn’t have to guess which label belongs to which point.

So in case you’re like me and have a horrible time reading these graphs, scores far over to the right on the x-axis are those who did poorly the first time. Scores low on the y-axis are those who did well the second time. So high right corner are the weak students at both beginning and end. The low left corner are the strong students who did well on both.

Geometry first. Thirty one students took both tests.

Spring2013GeomIndImprovement

Four students saw no improvement, another four actually got more wrong, although just 1 or 2 more. Another 3 students saw just one point improvement. But notice that through the middle range, almost all the students saw enormous improvement: twelve students, over a third, got from five to sixteen more correct answers, that is, improved from 10% to over 30%.

Now Algebra 2. Forty eight students took both tests; I had more testers at the end than the beginning; about ten students started a few days late.

Spring2013A2IndImprovement

Seven got exactly the same score both times, but only three declined (one of them a surprising 5 points—she was a good student. Must not have been feeling well). Eighteen (also a third) saw improvements of 5 to 16 points.

The average improvement was larger for the Algebra 2 classes than the Geometry classes, but not by much. Odd, considering that I’m actually teaching algebra, directly covering some of the topics in the test. In another sense, not so surprising, given that I am actually tasked to teach an entirely different topic in both cases. I ain’t teaching to this test. Still, I am puzzled that my algebra II students consistently show similar progress to my geometry students, even though they are soaked in the subject and my geometry students aren’t (although they are taught far more algebra than is usual for a geometry class).

I have two possible answers. Algebra 2 is insanely complex compared to geometry, particularly given I teach a very slimmed-down version of geometry. The kids have more to keep track of. This may lead to greater confusion and difficulty retaining what they’ve learned.

The other possibility is one I am reminded of by a beer-drinking buddy, a serious mathematician who is also teaches math: namely, that I’m a kickass geometry teacher. He bases this assertion on a few short observations of my classes and extensive discussions, fueled by many tankards of ale, of my methods and conceptual approaches (eg: Real-life coordinate Geometry, Geometry: Starting Off, Teaching Geometry,Teaching Congruence or Are You Happy, Professor Wu?, Kicking Off Triangles, Teaching Trig).

This possibility is a tad painful to contemplate. Fully half the classes I’ve taught in my four years of teaching—twelve out of twenty four—have been some form of Algebra, either actual Algebra I or Algebra I pretending to be Algebra II. I spend hours thinking about teaching algebra, about making it more understandable, and I believe I’ve had some success (see my various posts on modeling).

Six of those 24 classes have been geometry. Now, I spend time thinking about geometry, too, but not nearly as much, and here’s the terrible truth: when I come up with a new method to teach geometry, whether it be an explanation or a model, it works for a whole lot longer than my methods in algebra.

For example, I have used all the old standbys for identifying slope direction, as well as devising a few of my own, and the kids are STILL doing the mental equivalent of tossing a coin to determine if it’s positive or negative. But when I teach my kids how to find the opposite and adjacent legs of an angle (see “teaching Trig” above), the kids are still remembering it months later.

It is to weep.

I comfort myself with a few thoughts. First, it’s kind of cool being a kickass geometry teacher, if that is my fate. It’s a fun class that I can sculpt to my own design, unlike algebra, which has a billion moving parts everyone needs again.

Second, my algebra II kids say without exception that they understand more algebra than they ever did in the past, that they are willing to try when before they just gave up. Even the top kids who should be in a different class tell me they’ve learned more concepts than before, when they tended to just plug and play. My algebra 2 kids are often taking math placement tests as they go off to college, and I track their results. Few of them are ending up in more than one class out of the hunt, which would be my goal for them, and the best are placing out of remediation altogether. So I am doing something right.

And suddenly, I am reminded of my year teaching all algebra, all the time, and the results. My results look mediocre, yet the school has a stunningly successful year based on algebra growth in Hispanic and ELL students—and I taught the most algebra students and the most of those particular categories.

Maybe what I get is what growth looks like for the bottom 75% of the ability/incentive curve.

Eh. I’ll keep mulling that one. And, as always, spend countless hours trying to think up conceptual and procedural explanations that sticks.

I almost titled this post “Why Merit Pay and Value Added Assessment Won’t Work, Part IA” because if you are paying attention, that conclusion is obvious. But after starting a rant, I decided to leave it for another post.

Also glaringly on display to anyone not ignorant, willfully obtuse, or deliberately lying: Common Core standards are irrelevant. I’d be cynically neutral on them because hell, I’m not going to change what I do, except the tests will cost a fortune, so go forth ye Tea Partiers, ye anti-test progressives, and kill them standards daid.


Why Merit Pay and Value Added Assessment Won’t Work, Part I

The year I taught Algebra I, I did a lot of data collection, some of which I discussed in an earlier post. Since I’ve been away from that school for a while, I thought it’d be a good time to finish the discussion.

I’m not a super stats person. I’m not even a mathematician. To the extent I know math, it’s applied math, with the application being “high school math problems”. This is not meant to be a statistically sound analysis, comparing Treatment A to Treatment B. But it does reveal some interesting big picture information.

This data wasn’t just sitting around. A genuine DBA could have probably whipped up the report in a few hours. I know enough SQL to get what I want, but not enough to get it quickly. I had to run reports for both years, figure out how to get the right fields, link tables, blah blah blah. I’m more comfortable with Excel than SQL, so I dumped both years to Excel files and then linked them with student id. Unfortunately, the state data did not include the subject name of each test. So I could get 2010 and 2011 math scores, but it took me a while to figure out how to get the 2010 test taken—and that was a big deal, because some of the kids whose transcripts said algebra had, in fact, taken the pre-algebra (general math) test. Not that I’m bitter, or anything.

Teachers can’t get this data easily. I haven’t yet figured out how to get the data for my current school, or if it’s even possible. I don’t know what my kids’ incoming scores are, and I still haven’t figured out how my kids did on their graduation tests.

So the data you’re about to see is not something teachers or the general public generally has access to.

At last school, in the 2010-11 school year, four teachers taught algebra to all but 25 of over 400 students. I had the previous year’s test scores for about 75% of the kids, 90% of whom had taken algebra the year before, the other 10% or so having taken pre-algebra. This is a slightly modified version of my original graph; I put in translations of the scores and percentages.

algallocdist

You should definitely read the original post to see all the issues, but the main takeaway is this: Teacher 4 has a noticeably stronger population than the other three teachers, with over 40% of her class having scored Basic or Higher the year before, usually in Algebra. I’m Teacher 3, with by far the lowest average incoming scores.

The graph includes students for who I had 2010 school year math scores in any subject. Each teacher has from 8-12 pre-algebra student scores included in their averages. Some pre-algebra kids are very strong; they just hadn’t been put in algebra as 8th graders due to an oversight. Most are extremely weak. Teachers are assessed on the growth of kids repeating algebra as well as the kids who are taking it for the first time. Again, 80% of the kids in our classes had taken algebra once. 10-20% had taken it twice (our sophomores and juniors).

Remember that at the time of these counts, I had 125 students. Two of the other teachers (T1 and T4) had just under 100, the third (T2) had 85 or so. The kids not in the counts didn’t have 2010 test scores. Our state reports student growth for those with previous years’ scores and ignores the rest. The reports imply, however, that the growth is for all students. Thanks, reports! In my case, three or four of my strongest students were missing 2010 scores, but the bulk of my students without scores were below average.

So how’d we do?

I limited the main comparison to the 230 students who took algebra for both years and had scores for both years and had one of 4 teachers.

scoreimpalg

Here are the pre-algebra and algebra intervention growth–pre-algebra is not part of the above scores, but the algebra intervention is a sub-group. These are tiny groups, but illustrative:

scoreimpother

The individual teacher category gains/slides/pushes are above; here they are in total:
myschooltotcatchg

(Arrrggh, I just realized I left off the years. Vertical is 2010, horizontal is 2011.)

Of the 230 students who took algebra two years in a row, the point gain/loss categories went like this:

Score change > + 50 points 57
Score change > -20 points 27
-20 points < score change < + 50 points 146

Why the Slice and Dice?

As I wrote in the original post, Teacher 1 and I were positive that Teacher 4 had much stronger student population than we did—and the data supports that belief. Consequently I suspected that no matter how I sliced the data, Teacher 4 would have the best numbers. But I wanted a much better idea of how I’d done, based on the student population.

Because one unshakeable fact kept niggling at me: our school had a tremendous year in 2010-2011, based largely on our algebra scores. We knew this all throughout the year—benchmark tests, graduation tests—and our end of year tests confirmed it, giving us a huge boost in the metrics that principals and districts cared about. And I’d taught far more algebra students than any other teacher. Yet my numbers based on the district report looked mediocre or worse. I wanted to square that circle.

The district reports the data on the right. We were never given average score increase. A kid who had a big bump in average score was irrelevant if he or she didn’t change categories, while a kid who increases 5 points from the top of one category to the bottom of another was a big win. All that matters were category bumps. From this perspective, my scores look terrible.

I wanted to know about the data on the left. For example Teacher 1 had far better “gain” category numbers than I did. But we had the same mean improvement overall, of 5%, with comparable increases in each category. Broken down further, Teacher 4’s spectacular numbers are accompanied by a huge standard deviation—she improved some kids a lot. The other three teachers might not have had as dramatic a percentage increase, but the kids moved up more consistently. In three cases, the average score declined, but was accompanied by a big increase in standard deviation, suggesting many of the kids in that category improved a bit, while a few had huge drops. Teacher 2 and I had much tighter achievement numbers—I may have moved my students less far, but I moved a lot of them a little bit. None of this is to argue for one teacher’s superiority over another.

Of course, once I broke the data down by initial ability, group size became relevant but I don’t have the overall numbers for each teacher, each category, to calculate the confidence interval or a good sample size. I like 10. Eleven of the 18 categories hit that mark.

How many kids have scores for both years?

The 2011 scores for our school show that just over 400 students took the algebra test. My fall 2010 graph above show 307 students with 2010 scores (in any subject) who began the year. Kick in another 25 for the teacher I didn’t include and we had about 330 kids with 2010 scores. My results show 230 kids with algebra scores for both years, and the missing teacher had 18, making 248. Another 19 kids had pre-algebra scores for the first year, although the state’s reports wouldn’t have cared about that. So 257 of the kids had scores for both years, or about 63% of the students tested.

Notice that I had the biggest fall off in student count. I think five of my kids were expelled before the tests, another four or so left to alternative campuses. I remember that two went back to Mexico; one moved to his grandparents’ in Iowa. Three of my intervention students were so disruptive during the tests that they were ejected, so their test results were not scored (the next year our school had a better method of dealing with disruptive students). Many of the rest finished the year and took the tests, but they left the district over the summer (not sure if they are included in the state reports, but I couldn’t get their data). I think I had the biggest fall-off over the year in the actual student counts; I went from 125 to 95 by year-end.

What about the teachers?

Teacher 1: TFA, early-mid 20s, Asian, first year teacher. Had a first class honors masters degree in Economics from one of the top ten universities in Europe. She did her two, then left teaching and is now doing analytics for a fashion firm in a city where “fashion firm” is a big deal. She was the best TFAer I’ve met, and an excellent new teacher.

Teacher 2: About 60. White. A 20-year teacher who started in English, took time off to be a mom, then came back and got a supplemental math credential. She is only qualified to teach algebra. She is the prototype for the Teacher A I described in my last post, an algebra specialist widely regarded as one of the finest teachers in the district, a regard I find completely warranted.

Teacher 3: Me. 48 at the time, white. Second career, second year teacher, English major originally but a 15-year techie. Went to one of the top-rated ed schools in the country.

Teacher 4: Asian, mid-late 30s. Math degree from a solid local university, teaches both advanced math and algebra. She became the department head the next year. The reason her classes are top-loaded with good students: the parents request her. Very much the favorite of administration and district officials.

And so, a Title I school, predominantly Hispanic population (my classes were 80% Hispanic), teachers that run the full gamut of desirability—second career techie from a good ed school, experienced pro math major, experienced pro without demonstrated higher math ability, top-tier recent college grad.

Where was the improvement? Case 1: Educational Policy Objectives

So what is “improvement”? Well, there’s a bunch of different answers. There’s “significant” improvement as researchers would define it. Can’t answer that with this data. But then, that’s not really the point. Our entire educational policy is premised on proficiency. So what improvement does it take to reach “proficiency”, or at least to change categories entirely?

Some context: In our state, fifty points is usually enough to move a student from the bottom of one category to the bottom of another. So a student who was at the tip top of Below Basic could increase 51 points and make it to the bottom of Proficient, which would be a bump of two categories. An increase of 50 points is, roughly, a 17% increase. Getting from the bottom of Far Below Basic to Below Basic requires an increase of 70%, but since the kids were all taking Algebra for the second time, the boost needed to get them from FBB to BB was a more reasonable 15-20%. To get from the top of the Far Below Basic category to Proficient—the goal that we are supposed to aim for—would require a 32% improvement. Improving from top of Basic to bottom of Advanced requires a 23% improvement.

Given that context, only two of the teachers in one category each moved the needle enough to even think about those kind of gains—and both categories had 6-8 students. Looking at categories with at least ten students, none of the teachers had average gains that would achieve our educational policy goals. In fact, from that perspective, the teachers are all doing roughly the same.

I looked up our state reports. Our total population scoring Proficient or Advanced increased 1%.

Then there’s this chart again:

myschooltotcatchg

32 students moved from “not proficient” to “proficient/advanced”. 9 students moved from “proficient” to “advanced”. I’ll throw them in. 18% of our students were improved to the extent that, officially, 100% are supposed to achieve.

So educational policy-wise, not so good.

Where was the improvement? Case 2: Absolute Improvement

How about at the individual level? The chart helps with that, too:

myschooltotcatchg

Only 18 students were “double gainers” moving up two categories, instead of 1. Twelve of those students belonged to Teacher 4; 4 belonged to Teachers 1 , while Teacher 2 and I only had 1 (although I had two more that just missed by under 3 points). Teachers 1, 2, and 3 had one “double slider” each, who dropped two categories.

(I interviewed all the teachers on the double gainers; in all cases, the gains were unique to the students. The teachers all shrugged—who knew why this student improved? It wasn’t some brilliant aha moment unique to that teacher’s methods, nor was it due to the teacher’s inspiring belief and/or enthusiasm. Two of the three echoed my own opinion: the students’ cognitive abilities had just developed over the past year. Or maybe for some reason they’d blown off the test the year before. I taught two of the three “double sliders”—one was mine, one I taught the following year in geometry, so I had the opportunity to ask them about their scores. Both said “Oh, yeah, I totally blew off the test.” )

So a quarter of the students had gains sufficient to move from the middle of one category to the middle of another. The largest improvement was 170 points, with about 10 students seeing >100 point improvement. The largest decline was 169 points, with 2 students seeing over 100 point decline. Another oddity: only one of these two students was a “double slider”. The other two “double sliders” had less than 100 point declines. My double slider had a 60 point decline; my largest point decline was 89 points, but only dropped one category.

However, the primary takeaway from our data is that 63% of the students forced to take algebra twice were, score-wise if not category-wise, a “push”. They dropped or gained slightly, may have moved from the bottom of one category to the middle of the same, or maybe from the top of one category to the bottom of another.

One might argue that we wasted a year of their lives.

State reports say our average algebra score from 2010 to 2011 nudged up half a point.

So it’s hard to find evidence that we made much of a difference to student achievement as a whole.

I know this is a long post, so I’ll remind the reader that all of the students in my study have already taken algebra once. Chew on that for a while, will you?

Where was the improvement? Case 3: Achievement Gap

I had found no answer to my conundrum in my above numbers, although I had found some comfort. Broken down by category, it’s clear I’m in the hunt. But the breakdown doesn’t explain how we had such a stupendous year.

But when I thought of comparing our state scores from year to year, I got a hint. The other way that schools can achieve educational policy objectives is by closing the achievement gap.

All of this data comes from the state reports for our school, and since I don’t want to discuss who I am on this blog, I can’t provide links. You’ll have to take my word for it—but then, this entire post is based on data that no one else has, so I guess the whole post involves taking my word for it.

2010-11 Change
Overall + 0.5
Whites - 7.2
Hispanics + 4
EcDis Hisp - 1
ELL + 7

Wow. Whites dropped by seven points, Hispanics overall increased by 4, and non-native speakers (almost entirely Hispanic and economically disadvantaged), increased by 7 points.

So clearly, when our administrator was talking about our great year, she was talking about our cleverness in depressing white scores whilst boosting Hispanics.

Don’t read too much into the decline. For example, I personally booted 12 students, most of them white, out of my algebra classes because they’d scored advanced or proficient in algebra the previous year. Why on earth would they be taking the subject again? No other teacher did this, but I know that these students told their friends that they could get out of repeating Algebra I simply by demanding to be put in geometry. So it’s quite possible that much of the loss is due to fewer white advanced or proficient students taking algebra in the first place.

So who was teaching Hispanics and English Language Learners? While I can’t run reports anymore, I did have my original file of 2010 scores. So this data is incoming students with 2010 scores, not the final 2011 students. Also, in the file I had, the ED and ELL overlap was 100%, and I didn’t care about white or black EDs for this count. Disadvantaged non-ELL Asians in algebra is a tiny number (hell, even with ELL). So I kept ED out of it.

  Hisp ELL
t1 30 21
t2 32 38
t3 48 37
t4 39 12

Well, now. While Teacher 4 has a hefty number of Hispanics, very few of them are poor or ELLs. Teacher 2 seems to have Asian ELLs in addition to Hispanic ELLs. I have a whole bunch of Hispanics, most of them poor and ELL.

So I had the most mediocre numbers, but we had a great year for Hispanic and ELL scores, and I had the most Hispanic and ELL students. So maybe I was inadvertently responsible for depressing white scores by booting all those kids to geometry, but I had to have something to do with raising scores.

Or did I? Matthew DiCarlo is always warning against confusing comparing year to year scores, which are a cross-section of data at a point in time, with comparing student progress at two different points in time. In fact, he would probably say that I don’t have a conundrum, that it’s quite possible for me to have been a crappy teacher who had minimal impact on student achievement compared point to point, while the school’s “cross-section” data, which doesn’t compare students directly, could have some other reason for the dramatic changes.

Fair enough. In that case, we didn’t have a great year, right? It was just random happenstance.

This essay is long enough. So I’ll leave any one interested to explain why this data shows that merit pay and value added scores are pointless. I’m not sure when I’ll get back to it, as I’ve got grades to do.


Spring 2013: These students aren’t really prepared, either.

I’m teaching Geometry and Algebra II again, so I gave the same assessment and got these results, with the beginning scores from the previous semester:

AlgAssessspr13

I’m teaching two algebra II classes, but their numbers were pretty close to identical—one class had the larger range and a lower mode—so I combined them.

The geometry averages are significantly lower than the fall freshmen only class, which isn’t surprising. Kids who move onto geometry from 8th grade algebra are more likely to be stronger math students, although (key plot point) in many schools, the difference between moving on and staying back in algebra come down to behavior, not math ability. At my last school, kids who didn’t score Proficient or Advanced had to take Algebra in 9th grade. I’d have included Basic kids in the “move-on” list as well. But sophomores who not only can’t factor or graph a line, but struggle with simple substition ought not to be in second year algebra. They should repeat algebra I freshman year, go onto geometry, and then take algebra II in junior year—at which point, they’d still be very weak in algebra, of course, but some would have benefited from that second year of first year.

Wait, what was my point? Oh, yeah–this geometry class class is 10-12, so the students took one or more years of high school algebra. Some of them will have just goofed around and flunked algebra despite perfectly adequate to good skills, but a good number will also be genuinely weak at math.

On the other hand, a number of them really enjoyed my first activity: visualizing intersecting planes, graphing 3-D points. I got far more samples from this class. I’ll put those in another post, also the precalc assessment.

I don’t know if my readers (I have an audience! whoo!) understand my intent in publishing these assessment results. In no way am I complaining about my students.

My point in a huge nutshell: how can math teachers be assessed on “value-added” when the testing instrument will not measure what the students needed to learn? Last semester, my students made tremendous gains in first year algebra knowledge. They also learned geometry and second year algebra, but over half my students in both classes will test Below Basic or Far Below Basic–just as they did the year before. My evaluation will faithfully record that my students made no progress—that they tested BB or FBB the year before, and test the same (or worse) now. I will get no credit for the huge gains they made in pre-algebra and algebra competency, because educational policy doesn’t recognize the existence of kids taking second year algebra despite being barely functional in pre-algebra.

The reformers’ response:

1) These kids just had bad teachers who didn’t teach them anything, and in the Brave New World of Reform, these bad teachers won’t be able to ruin students’ lives;

2) These bad teachers just shuffled students who hadn’t learned onto the next class, and in the Brave New World of Reform, kids who can’t do the work won’t pass the class.

My response:

1) Well, truthfully, I think this response is moronic. But more politely, this answer requires willful belief in a delusional myth.

2) Fail 50-60% of kids who are forced to take math classes against their will? Seriously? This answer requires a willful refusal to think things through. Most high schools require a student to take and pass three years of math for graduation. Fail a kid just once, and the margin for error disappears. Fail twice and the kid can’t graduate. And in many states, the sequence must start with algebra—pre-algebra at best. So we are supposed to teach all students, regardless of ability, three years of increasingly abstract math and fail them if they don’t achieve basic proficiency. If, god save us, the country was ever stupid enough to go down this reformer path, the resulting bloodbath would end the policy in a year. We’re not talking the occasional malcontent, but over half of a graduating class in some schools—overwhelmingly, this policy impacts black and Hispanic students. But it’s okay. We’re just doing it for their own good, right? Await the disparate impact lawsuits—or, more likely, federal investigation and oversight.

Reformers faithfully hold out this hope: bad teachers are creating lazy students who could do the work but just don’t want to. Oh, yeah, and if we catch them in elementary school, they’ll be fine in high school.

It is to weep.

Hey, under 1000 words!


Algebra 1 Growth in Geometry and Algebra II

Last September, I wrote about my classes and the pre-algebra/Algebra 1 assessment results.

My school covers a year of instruction in a semester, so we just finished the first “year” of courses. I start with new students and four preps on Monday. Last week, I gave them the same assessment to see if they’d improved.

Unfortunately, the hard drive on my school computer got wiped in a re-imaging. This shouldn’t have been a problem, because I shouldn’t have had any data on the hard drive, except I never got put on the network. Happily, I use Dropbox for all my curriculum development, so an entire year’s worth of intellectual property wasn’t obliterated. I only lost the original assessment results, which I had accidentally stored on the school hard drive. I should have entered the scores in the school grading system (with a 0 weight, since they don’t count towards the grade) but only did that for geometry, the only class I can directly compare results with.

My algebra II class, though, was incredibly stable. I only lost three students, one of whom got a perfect score—which the only new addition to the class also got, so balance maintained. The other two students who left got around 10-15 wrong, so were squarely in the average at the time. I feel pretty comfortable that the original scores didn’t change substantially. My geometry class did have some major additions and removals, but since I had their scores I could recalculate.

Mean Median Mode Range
Original just above 10 9.5 7 22
Recalculated just below 10 (9.8) 8 7 22

I didn’t have the Math Support scores, and enough students didn’t take the second test that comparisons would be pointless.

One confession: Two Algebra II students, the weakest two in the class, who did no work, scored 23 and 24 wrong, which was 11 more than the next lowest score. Their scores added an entire point to the average wrong, increased the range by 14 points, and you know, I just said bye and stopped them from distorting the results the other 32 kids. (I don’t remember exactly, but the original A2 tests had five or six 20+ wrong scores.)

So here’s the original September graph and the new graph of January:

AlgtestAlgAssessyrend

The geometry class was bimodal: 0 and 10. Excel refused to acknowledge this and I wasn’t sure how to force it. The 10s, as a group, were pretty consistent—only one of them improved by more than a point. The perfect scores ranged from 8 wrong to 2 wrong on the first test.

geoalgclassgrowth

In short, they learned a lot of first year algebra, and that’s because I spent quite a bit of time teaching them first year algebra. In Algebra II, I did it with data modeling, which was a much more sophisticated approach than what they’d had before, but it was still first year algebra. In geometry, I minimize certain standards (proofs, circles, solid shapes) in favor of applied geometry problems with lots of algebra.

And for all that improvement, a still distressing number of students answered x2 + 12 when asked what the product of (x+3) and (x+4) was, including two students who got an A in the class. I beat this into their heads, and STILL some of them forget that.

Some folks are going to draw exactly the wrong impression. “See?” these misguided souls will say, nodding wisely. “Our kids just aren’t being taught properly in early grades. Better standards, better teachers, this problems’s fixed! Until then, this poor teacher has to make up the slack.” In short, these poor fools still believe in the myth that they’ve never been taught.

When in fact, they were taught. Including by me—and I don’t mean the “hey, by the way, don’t forget the middle term in binomial multiplication”, but “you are clubbing orphan seals and making baby Jesus cry when you forget the middle term” while banging myself on the head with a whiteboard. And some of them just forgot anyway.

I don’t know how my kids will do on their state tests, but it’s safe to say that the geometry and second year algebra I exposed them to was considerably less than it would have been had their assessment scores at the beginning of class been the ones they got at the end of class. And because no one wants to acknowledge the huge deficit half or more of each class has in advanced high school math, high schools won’t be able to teach the kids the skills they need in the classes they need—namely, prealgebra for a year, “first year” algebra for two years, and then maybe some geometry and second year algebra. If they do okay on the earlier stuff.

Instead, high schools are forced to pretend that transcripts reflect reality, that all kids in geometry classes are capable of passing a pre-algebra test, much less an algebra one test. Meanwhile, reformers won’t know that I improved my kids’ basic algebra skills whilst still teaching them a lot of geometry/algebra II, because the tests they’ll insist on judging me with will assume a) that the kids had that earlier material mastered or b) that I could just catch them up quickly because after all, the only problem was the kids’ earlier teachers had never taught them.


Teaching Students with Utilitarian Spectacles

In my last post, commenter AllaninPortland said, of my Math Support students, “Their brains are wired a little too literally for modern life.”

James Flynn, of the Flynn Effect:

A century ago, people mostly used their minds to manipulate the concrete world for advantage. They wore what I call “utilitarian spectacles.” Our minds now tend toward logical analysis of abstract symbols—what I call “scientific spectacles.” Today we tend to classify things rather than to be obsessed with their differences. We take the hypothetical seriously and easily discern symbolic relationships.

Yesterday I gave my math support kids a handout on single step equations similar to the one in the link.

“Oh, I know how to do this,” said Dewayne. “Just subtract six from both sides.”

“You could do that,” I said. “But here’s what I want people to try. I want everyone to read the first equation as a sentence. What is it saying?”

“Some number added to six gets fourteen,” came from Andy.

“Excellent!”

“You mean, you don’t want us to subtract, add, do things to get x by itself?” asked Jose.

“That’s called ‘isolation’. You are ‘isolating’ x, getting it all by itself as you put it. Who knows how to do that?” Over half the class raised their hands. “Great. You can do that if you want to, but I’d like you to try seeing each equation just as Andy described it. Put the equation you see into words. This will help make it real, and will often give you the answer right away. For example, what number do I add to six to get 14?”

“Eight.” chorused most of the room.

“There you go. Now, remember, what did I say a fraction was?”

“Division.”

“So instead of saying ‘x over 5′, you’re going to say….”

“X divided by 5″ came back a number of students.

“Off you go.”

This worked for most of the students, but one student, Gerry, sat at the back of the room drawing, as he often does. After watching him do no work for 10 minutes, I called him up front. (Normally, I am wandering the room, but every so often I call them up for conversations instead.)

“So you aren’t working.”

“Yeah. I can’t do this.”

“Remember yesterday, when we were doing those PEMDAS problems? You were on fire!”

“Yeah, but it didn’t have the letters in it. I can do math when it doesn’t have letters. And yesterday, when you showed us how to just draw pictures for the word problems? That was cool. I think I can do those now.”

“You need to look at these problems from a different part of your brain.”

“A different what?”

“This is a really, really easy problem. Way easier than the math problems you solved in your head yesterday. But you don’t see this as the same kind of problem, so we have to fool your brain.”

“How do we do that?”

“Read the first problem aloud.”

“X + 6 = 14. This is when you have to do stuff to both sides, right? I can’t do that.”

“Read it again. But instead of saying x, say ‘what’.”

“Say ‘what’?”

“Yep.”

“You crazy.”

“Definitely. Try it.”

“What plus 6 = 14? 8.”

“There you go.”

He was sitting in one of my wheeled chairs, pushing it back and forth with his feet. This stopped him cold.

“Eight’s the answer? Holy sh**.”

“Try another. Without the language.”

“What minus 3 = 7. That’s nine…no, 10. Ten? Really? No f**k….no way.”

“And this one?”

“Oh, that’s a fraction. I can’t do those.”

“What did I tell you fractions were?”

“Division. Oh. What divided by 5 is 9? Forty five? No way?”

“So. I want to see you do this whole handout, 1-26, and every time you see an x, call it ‘what’. Remember to sketch out subtraction questions on a numberline and think about direction.”

“Okay. Man, I can’t believe this.”

Fifteen minutes later, Gerry was done with the entire set. Only three minor errors, all involving negative numbers.

“I feel like a math genius,” he said with a wry grin.

I sat down next to him. “It’s like I said. We have to ask your brain a different question. So instead of tuning me out, next time I come up with some goofy idea using pictures or tiles or different words, give it a shot. And tell me if it works to give your brain the right question. Some of my ideas will work, some won’t. And some things, we won’t be able to fool your brain to answer a different way. But you know a lot more math than you think you do. You just have to figure out how to ask the question in a way your brain understands.”

Back to Flynn:

A greater pool of those capable of understanding abstractions, more contact with people who enjoy playing with ideas, the enhancement of leisure—all of these developments have benefited society. And they have come about without upgrading the human brain genetically or physiologically. Our mental abilities have grown, simply enough, through a wider acquaintance with the world’s possibilities.

But not everyone is capable of understanding abstractions to the same degree. Some people do better learning the names of capitals and Presidents and the planets in the solar system. They’d learn confidence and competence through interesting, concrete math word problems and situations, and enjoy reading and writing about specific historic events, news, or scientific inventions that helped society. Instead, we shovel them into algebra, chemistry, and literature analysis and make them feel stupid.

Students’ names have been changed. They are all awesome kids. Do not say mean things about them in the comments, which I can control, or other blogs, which I cannot.


The Sinister Assumption Fueling KIPP Skeptics?

Stuart Buck on KIPP critics:

It’s unwitting, to be sure; most of the critics haven’t thought through the logical implications of what they’re saying, and they would sincerely deny being racist in their thoughts or intentions. But even granting their personal good will, what they are saying is full of racially problematic implications. These KIPP critics are effectively saying that poor minority children are incapable of genuinely learning anything more than they already do. If poor minority children seem to be learning more, it can’t really be true; there must be some more sinister explanation for what’s going on.
…..
Now here’s the key point: If selection and attrition is what explains KIPP’s good results, then that logically means that several hundred extra hours a year being instructed in reading, math, music, art, etc. do NOT explain KIPP’s good results. But wait a minute: what does that really mean?
….
Nothing less than this: several hundred hours a years instructing kids doesn’t actually make much difference. Recall that KIPP’s critics say that if KIPP’s students seem to be learning more, it must be an artifact of how KIPP selects kids and then pushes out the low-performers. In saying that, KIPP’s critics are implying, however unwittingly, that no amount of effort or study could possibly get poor urban minorities to learn anything more.

Okay, let me be clear that I am not speaking for any other KIPP critic. While I don’t talk much about KIPP, I am certainly one who thinks their results are due to attrition, creaming, and the benefits that accrue from a homogenous and motivated population.

But yeah. In a nutshell, I’m saying this:

IF you take low ability kids (of any race or income) and IF you select for motivation in the parents, at least, and IF you remove the misbehaving or otherwise highly dysfunctional kids who don’t share their parents’ motivation, and IF you enforce strict behavioral indoctrination in middle class mores and IF you give them hundreds of hours more education a year and IF they are in middle school and IF they are simply being asked to catch up with the material that middle to high ability kids learned fairly effortlessly—that is, elementary reading and math skills…..

…then they will have a slightly better test scores than similarly motivated low ability kids stuck in classes with the misbehavers and highly dysfunctional kids and fewer hours of seat time and less behavioral indoctrination into middle class mores, but their underlying abilities will still be weak and just as far behind their higher ability peers as they were before KIPP.

I’ve written before, improving elementary school or middle school scores is a false god when it comes to improving actual high school outcomes. Children who need tons of hours to get up to grade level fundamentally differ from those reading at or above grade level from kindergarten on, and this difference matters increasingly as school gets harder. High school isn’t the linear steps through increased difficulty that occurs in grades K-8, but a much different and far more difficult animal, now that we make everyone take college prep classes. There’s no evidence that KIPP students are learning more or closing the gap in high school, and call me cynical but I’m really, really sure we’d be hearing about it if they were. KIPP is not transforming low ability kids into high ability kids, or even mid-level ability kids.

I am comfortable asserting that hours and hours of additional education time does nothing to change underlying ability. I’m not a racist, nor am I a nihilist who believes outcomes are set from birth. I do, however, hold the view that academic outcomes are determined in large part by cognitive ability. The reason scores are low in high poverty, high minority schools is primarily due to the fact that the students’ abilities are low to begin with, not because they enter school with a fixable deficit that just needs time to fill, and not because they fall behind thanks to poor teachers or misbehaving peers.

That doesn’t mean we can’t improve outcomes, particularly in high school, when we do a great deal of harm by trying to teach kids what they can’t learn and refusing to teach them what they can learn. And it doesn’t mean we couldn’t tremendously improve elementary school outcomes in numbers, if not individual demonstrated ability, by allowing public schools to do what KIPP does—namely, limit classes to motivated kids of similar ability.

Paul Bruno, another KIPP skeptic (whose views in no way should be confused with mine), thinks it’s wrong to dismiss KIPP achievements, because they show that public schools for low income kids simply need much more money. I disagree. What KIPP “success” shows is the importance of well-behaved, homogeneous classes.

So here’s my preferred takeaway from KIPP and other successful charter schools:

Since it’s evident that much of these schools’ success stories come from their ability to control and limit the population, why are we still hamstringing public schools? Here’s a thought: how about KIPP schools take those really, really tough kids and only those kids? Misbehave too often in public schools and off you go to a KIPP bootcamp, where they will drill you with slogans and do their best to indoctrinate you into middle class behavior and after a while you’ll behave because please, god, anything to get back to the nicer public schools! You could also create KIPP schools for special ed kids–put the special ed kids with cognitive issues and learning disabilities in their own, smaller schools. Meanwhile, public schools could extend the school day a bit, help the kids catch up as much as possible while still making school fun. While the average test score might not improve much, this approach would keep a lot of kids engaged in school through elementary school instead of lost, bored, or acting out in chaotic classes disrupted by a few unmanageable or extremely low ability kids.

See, that would scale a lot better. Instead, we set up small schools for what is actually the majority of all low income students—reasonably well-behaved, of low to middle ability and, with no one around to lead them astray, willing to give school a shot. Only a few kids get into these schools, while the rest of them are stuck in schools where just a few misbehavers make class impossible and really low ability kids take up a lot of addtional teacher time. Crazy, that’s what it is. But what I just laid out is completely unworkable from an ideological standpoint, and as I just explained in an earlier post, school policy is set by ideology and politics, not educational validity. To say nothing of the fact that KIPP doesn’t want to teach “those” kids.

Anyway. The reality is that yes, a low ability kid, regardless of income or race, will not, on average, become a high or mid ability kid simply because he spends a lot of seat time working his butt off in a KIPP school. Sorry Stuart.


SAT Prep for the Ultra-Rich, And Everyone Else

Whenever I read about SAT tutors charging in the hundreds of dollars, I’m curious. I know they exist, but I also know that I’m pretty damn good, and I’m not charging three figures per hour (close, though!). So I always read them closely to see if, in fact, these test prep tutors are super fab in a way that I’m not.

At the heart of all test prep stories lies the reporter’s implicit rebuke: See what rich people are doing for their kids? See the disadvantage that the regular folks operate under? You can’t afford those rates! You’re stuck with Kaplan or cheaper, cut-rate tutors! And that’s if you’re white. Blacks and Hispanics can’t even get that much. Privilege. It sucks.

And so the emphasis on the cost of the tutors, rather than any clear-eyed assessment of what, exactly, these tutors are doing that justifies an hourly rate usually reserved for low-end lawyers, never mind the fact that these stories are always about the SAT, when in fact the ACT is taken by as many kids as the SAT. The stories serve up propaganda more than they provide an accurate picture of test prep.

I’ve written before about the persistence of test prep delusions. Reality, summarized: blacks and Hispanics use test prep more than whites, Asians use it more than anyone. Rich parents are better off buying their kids’ way into college than obsessing about the last few points. Test prep doesn’t artificially inflate ability.

So what, in fact, is the difference between Lisa Rattray, test prep coach charging $300/hour; me, charging just short of 3 figures; and a class at Kaplan/Princeton/other SAT test prep schools?

Nothing much. Test prep coaches can work for a company or on their own. The only difference is their own preferences for customer acquisition. Tutors and instructors with a low risk tolerance just sign on with a company. Independent operators, comfortable with generating their own business, then pick their markets based on their own tolerance. My customers sit comfortably in the high income bracket, say $500K to $5 million yearly income, although I’ve worked with a couple Fortune 500 families. Lisa Rattray and Joshua Brown, the featured tutors, clearly work with families a couple notches up the income ladder from mine.

None of this has anything to do with quality of instruction. Test prep is a sales and marketing game. The research is clear: most kids improve at least a little, quite a few kids improve a lot, a very few kids stay put or, heaven forfend, get worse.

Obviously, instructor quality influences results a bit, but only rarely change a kid from one category (mild improvement) to another (major improvement). Remember, all test prep instructors have high test scores, and they’re all excellent at understanding how the test works. So they make career decisions based on their tolerance for sales and marketing, not the quality of their services. I know of some amazingly god-awful tutors who charge more than I do, having learned of them from their furious ex-clients who assumed a relationship between price and quality. These tutors have websites, business cards, offered their own prepared test materials, saw students in their rented space, and often accepted credit card deposits. I have none of these accoutrements, show up at my clients’ houses, usually but not always on time, and take checks. Every so often I get a client who whips out a wad of bills and pays me $500 in cash, which I find a tad unnerving.

I’m just as good now as I was at Kaplan (in fact, I privately tutored my own students while at Kaplan, tutoring theirs), but I only got paid $24/hour for Kaplan work, which charged about $125/hour for my services. Kaplan will (at least, when I worked there) boost a teacher’s hourly rate to $50/hour if they get 80% or more “perfect” customer ratings. Instructors who convinced their students that to respond to the online survey and give them excellent ratings got more money. This is independent of actual improvement. A customer who doesn’t improve at all but felt reassured and valued by her instructor could give straight 5s (or 1s, whatever the highest rating is). A customer who sees a 300 point improvement might not fill in the survey at all. Their research showed that customers who give their instructors perfect ratings gave awesome word of mouth and that was worth rewarding. Nothing else was. Asian cram schools pay instructors based on the students who sign up, with a premium for those who sign up specifically for that instructor. See? Sales and marketing.

Test prep companies, long castigated as the luxury option of the wealthy, have been the first choice of the middle class for a decade or more. For the reasons I’ve outlined, any parent can find excellent instructors in all the test prep companies: Kaplan, Princeton Review, Asian cram schools. They won’t brag about it, though, because these companies are about the brand. Kaplan doesn’t want word getting out that Joe Dokes is a great Kaplan instructor; it wants everyone to be happy with Kaplan. No one is “Princeton Review’s star tutor” for very long, because Princeton doesn’t like it and at that point, the most risk-averse instructor probably has enough word of mouth fame to go independent.

I’ve often advised my students to consider a class. The structure helps. Some of my kids don’t do any work unless I’m there, so what I end up doing is sitting there playing Spider on my android on my client’s dime while the kid works problems, rather than reviewing a bunch of work to move forward. I’m pretty sure Lisa and Joshua would celebrate this, going to the parent and pointing out how much they are helping. I have better things to do and other clients to see. So I tell the parents to fork out an extra thousand for a class, make sure the kid goes, and then we review the completed work. The student gets more hours, more focus and, usually, higher scores, regardless of the quality of the second instructor.

I’m not saying Lisa and Joshua are wrong, mercenary, or irresponsible. They just play to a different clientele, and a huge chunk of their ability to do so rests on their desire to sell an image. That’s fine. That’s just not me. Besides, Josh forks out $15K of his profit for a rental each summer. Lisa gets constant text messages from anxious parents. Also not me.

So you’re a white, middle class or higher parent with a teenager, worried about SAT scores. What do you do? Here are some guidelines. Recognize that GPA or parental income smacks down test scores without breaking a sweat. If Johnny doesn’t have a GPA of 3.8 or higher, elite universities are out of the question unless his parents are alumni or rich/connected enough to make it worth the school’s while.

If Sally qualifies on GPA, has a top-tier transcript (5 or more AP classes) and wants to go to a top 10 school, test scores should be 700 or higher per section. If they’re at that point, don’t waste your time or money or stress. At that point, the deciding factors aren’t scores but other intangibles, including the possibility that the admissions directors toss a pile of applications in the air and see which ones travel the farthest.

If Jesse is looking for a top 20 or 30 school, the GPA/transcript requirements are the same, but looking at the CDS of these schools, realistically a 650 or higher per section will do the trick. It might be worth boosting the test scores to low 700s, but if Jesse is a terrible tester, then don’t break the bank. One of the schools will probably come through.

If Sammy has a lower GPA (3.3 to 3.8) but excellent test scores (high 600s or higher per section) , then look to the schools in the middle–say, from 40 to 60. It’s actually worth spending money to maximize Sammy’s scores, because these mid-tier schools often get a lot of high effort hard workers with mediocre test scores. Not only will Sammy look good, but he might get some money. (By the way, if you’ve got a Sammy whose grades are much lower than his abilities, you should still push him into the hardest classes, even if he and the counsellors cavil. If your Sammy is like most of them, he’s going to get Bs and Cs regardless, so he may as well get them in AP classes and get some college credit from the AP tests. And the transcript will signal better, as well.)

The biggest bang for the test prep buck lies not in making kids competitive for admissions, but to help them test out of remediation at local universities. So if Austin has a 3.0 GPA, works hard but tests poorly, then find out the SAT cut score at his university. If he’s not above that point, then spend the money to get him there, and emphasize the importance of this effort to his college goals.

If your kid is already testing at 650 or higher, either send her to an Asian cram school (they will be the only white kid there, for the most part, but the instruction will be excellent) or invest in a tutor. The average white kid class at Kaplan or Princeton might have an instructor who can finetune for their issues, but probably won’t.

Otherwise, start with a class and supplement with a tutor if you can afford it. Ask around for good instructors, or ask the test prep company how long the instructor has been teaching. Turnover in test prep instructors is something like 75%; the 25% who stay long term do so because they’re good. As for the tutor, I hope I’ve convinced everyone that price isn’t an issue in determining quality. I would ask around for someone like me, because our ability to get a high rate without the sales and marketing suggests we must be, in fact, pretty good. And there’s always someone like me around. Otherwise, I’d go with the private tutoring options at a test prep company, with interviews.

As I said, these rules are for middle class or higher white kids. Only 6% of blacks and Hispanics get above 600 on any section of the SAT–in fact, the emphasis on GPA came about in large part to bypass the unpleasant reality of the score gap. There are only around 300 black students that get higher than 700 on two sections of the SAT. That’s barely enough blacks for one top ten school. Rules are very different. The main reason for blacks and Hispanics to take test prep is to get their scores above the remediation number. Middle class or higher Asians face much higher standards because universities know their (or their parents’) dedication to getting good grades and good test scores is more than a tad unnatural and probably overstates their value to the campus. Athletes and artists of note play by different rules. Poor whites and poor Asians have it really, really tough.

What this means, of course, is that the kids in the Hamptons are probably already scoring 700 or higher per section and are, consequently, wasting their time. But what the hell, they’re doing the economy some good. Or maybe some of them are Asian.

Note: I wrote this focusing on the SAT but it all applies to the ACT as well, and the ACT is a much better test. I wrote about the ACT here.


Follow

Get every new post delivered to your Inbox.

Join 889 other followers