Assessing Math Understanding: Max, Homer, and Wesley

This is only tangentially a “math zombies” post, but I did come up with the idea because of the conversation.

I agree with Garelick and Beals that asking kids to “explain math” is most often a waste of time. Templates and diagrams and “flow maps” aren’t going to cut it, either. Assessing understanding is a complicated process that requires several different solutions methods and an interpretive dance. Plus a poster or three. No, not really.

As I mentioned earlier, I don’t usually ask kids to “explain their answer” because too many kids confuse “I wrote some words” with “I explained”. I grade their responses in the spirit given, a few points for effort. “Explain your answer” test questions are sometimes handy to see if top students are just going through the motions, or how much of my efforts have sunk through to the students. But I don’t rely on them much and apart from top students, don’t care much if the kids can’t articulate their thinking.

It’s still important to determine whether kids actually understand the math, and not just because some kids know the algorithm only. Other kids struggle with the algorithm but understand the concepts, Still others don’t understand the algorithm because they don’t grok the concepts. Finally, many kids get overwhelmed or can’t be bothered to work out the problem but will indicate their understanding if they can just read and answer true/false points.

If you are thinking “Good lord, you fail the kids who can’t be bothered or get overwhelmed by the algorithms!” then you do not understand the vast range of abilities many high school teachers face, and you don’t normally read this blog. These are easily remediable shortcomings. I’m not going to cover that ground again.

So how to ascertain understanding without the deadening “explain your answer” or the often insufficient “show your work”?

My task became much easier once I turned to multiple answer assessments. I can design questions that test algorithm knowledge, including interim steps, while also ascertaining conceptual knowledge.

I captured some student test results to illustrate, choosing two students for direct comparison, and one student for additional range. None of these students are my strongest. One of the comparison students, Max, would be doing much better if he were taught by Mr. Singh, a pure lecture & set teacher; the other, Homer, would be struggling to pass. The third, Wesley, would have quit attending class long ago with most other teachers.

To start: a pure factoring problem. The first is Max, the second Homer.

Both students got full credit for the factoring and for identifying all the correct responses. Max at first appears to be the superior math student; his work is neat, precise, efficient. He doesn’t need any factoring aids, doing it all in his head. Homer’s work is sloppier; he makes full use of my trinomial factoring technique. He factored out the 3 much lower on the page (out of sight), and only after I pointed out he’d have an easier time doing that first.

Now two questions that test conceptual knowledge:

Max guessed on the “product of two lines” question entirely, and has no idea how to convert a quadratic in vertex form to standard or factored. Yet he could expand the square in his head, which is why he knew that c=-8. He was unable to relate the questions to the needed algorithms.

Homer aced it. In that same big, slightly childish handwriting, he used the (h,k) parameters to determine the vertex. Then he carefully expanded the vertex form to standard form, which he factored. This after he correctly identified the fact that two lines always multiply to form a quadratic, no matter the orientation.

Here’s more of Homer’s work, although I can’t find (or didn’t take a picture of) Max’s test.

This question tests students’ understanding of the parameters of three forms of the quadratic: standard, vertex, factored. I graded this generously. Students got full credit if they correctly identified just one quadratic by parameter, even if they missed or misidentified another. Kids don’t intuitively think of shapes by their parameter attributes, so I wanted to reward any right answers. Full credit for this question was 18 points. A few kids scored 22 points; another ten scored between 15 and 18. A third got ten or fewer points.

Homer did pretty well. He was clearly guessing at times, but he was logical and consistent in his approach. Max got six points. He got a wrong, got b, c, & d correct, then left the rest blank. It wasn’t time; I pointed out the empty responses during the test, pointing out some common elements as a hint. He still left it blank.

On the same test, I returned to an earlier topic, linear inequalities. I give them a graph with several “true” points. Their task: identify the inequalities that would include all of these solutions.

(Ack: I just realized I flipped the order when building this image. Homer’s is the first.)

Note the typo that you can see both kids have corrected (My test typos are fewer each year, but they still happen.) I just told them to fix it; the kids had to figure out if the “fix” made the boundary true or false. (This question was designed to test their understanding of linear concepts–that is, I didn’t want them plugging in points but rather visualizing or drawing the boundary lines.)

Both Max and Homer aced the question, applying previous knowledge to an unfamiliar question. Max converted the standard form equation to linear form, while Homer just graphed the lines he wasn’t sure of. Homer also went through the effort of testing regions as “true”, as I teach them, while Max just visualized them (and probably would have been made a mistake had I been more aggressive on testing regions).

Here I threw something they should have learned in a previous year, but hadn’t covered in class:

Most students were confused or uncertain; I told them that when in doubt, given a point….and they all chorused “PLUG IT IN.”

This was all Max needed to work the problem correctly. Homer, who had been trying to solve for y, then started plugging it in, but not as fluently as Max. He has a health problem forcing him to leave slightly early for lunch, so didn’t finish. For the next four days, I reminded students in class that they could come in after school or during lunch to finish their tests, if they needed time. Homer didn’t bother.

So despite the fact that Homer had much stronger conceptual understanding of quadratics than Max, and roughly equal fluency in both lines and quadratics, he only got a C+ to Max’s C because Homer doesn’t really care about his grade so long as he’s passing.

Arrgghhh.

I called in both boys for a brief chat.

For Max, I reiterated my concern that he’s not doing as well as he could be. He constantly stares off into space, not paying attention to class discussions. Then he finishes work, often very early, often not using the method discussed in class. It’s fine; he’s not required to use my method, but the fact that he has another method means he has an outside tutor, that he’s tuning me out because “he knows this already”. He rips through practice sheets if he’s familiar with the method, otherwise he zones out, trying to fake it when I stop by. I told him he’s absolutely got the ability to get an A in class, but at this point, he’s at a B and dropping.

Max asked for extra credit. He knew the answer, because he asks me almost weekly. I told him that if he wanted to spend more time improving his grade, he should pay attention in class and ask questions, particularly on tests.

We’ve had this conversation before. He hasn’t changed his behavior. I suspect he’s just going to take his B and hope he gets a different teacher next year who’ll make the tutor worth the trouble. At least he’s not trying to force a failing grade to get to summer school for an easy A.

I was extremely direct with Homer,  expressing (snarling) my disappointment that he wouldn’t make the effort to be excellent, when he was so clearly capable of more. What was he doing that was so important he couldn’t take 20 minutes or so away to finish a test, given the gift of extra time? Homer stood looking a bit abashed. Next test, he came in during lunch to complete his work. And got an A.

Max got a B- on the same test, with no change in behavior.

I haven’t included any of the top students’ work because it’s rather boring; revelations only come with error patterns. But here, in a later test, is an actual “weak student”, who I shall dub Wesley.

Wesley had been forced into Algebra 2, against his wishes, since it took him five attempts to pass algebra I and geometry. He was furious and determined to fail. I told him all he had to do was work and I’d pass him. Didn’t help. I insisted he work. He’d often demand to get a referral instead. Finally, his mother emailed about his grade and I passed on our conversations. I don’t know how, but she convinced him to at least pick up a pencil. And, to Wesley’s astonishment, he actually did start to understand the material. Not all of it, not always.

This systems of equations question (on which many students did poorly) was also previous material. But look at Wesley! He creates a table! Just like I told him to do! It’s almost as if he listened to me!

He originally got the first equation as 20x + 2y = 210 (using table values); when I stopped by and saw his table, I reminded him to use it to find the slope–or, he could remember the tacos and burritos problem, which spurred his memory. You can’t really see the rest of the questions, but he did not get all the selections correct. He circled two correctly, but missed two, including one asking about the slope, which he could have found using his table. He also graphed a parabola almost correctly, above (you can see he’s marked the vertex point but then ignored it for the y-intercept).

He got a 69, a stupendous grade and effort, and actually grinned with amazement when I handed it back.

Clearly, I’m much better at motivating underachieving boys than I am “math zombies”. Unsurprising, since motivating the former is my peculiar expertise going back to my earliest days in test prep, and I’ve only recently had to contend with the latter. However, I’ve successfully reached out and intervened with similar students using this approach, so it’s not a complete failure. I will continue to work on my approach.

None of the boys have anything approaching a coherent, unified understanding of the math involved. In order to give them all credit for what they know and can do, while still challenging my strongest students, I have to test the subject from every angle. Assessing all students, scoring the range of abilities accurately, is difficult work.

As you can see, the challenges I face have little to do with Asperger’s kids who can’t explain what they think or frustrated parents dealing with number lines or boxes of 10. Nor is it anything solved by lectures or complex instruction. My task is complicated. But hell, it’s fun.

Who I Am as a Teacher

As I thought about writing specific disagreements I have with reformers, I realized that time and again I’d be having to break off and explain how my values and priorities differed. So I thought I’d do that first.

At my last school’s Christmas party (Year 1 at that school, Year 2 of teaching), the popular, widely respected “teacher at large” showed up an hour late. A PE teacher whose credential had been disallowed by NCLB-wrought changes, he was at that point responsible for coming up with plans to help “at-risk” kids.

“Yeah, I was having all sorts of fun reviewing the Lists.”

“The Lists?” asked another teacher.

“Top 25 Discipline Problems, Top 25 Kids On Probation for Felonies, Top 25 Absentee/Truancy Students, Lowest 25 GPAs, you name it. I look for the kids who aren’t on more than one or two lists and try to reach them before they qualify for more lists.”

“I think I have a few of those kids.” groused a teacher.

“A few? Pity Ed here.” He nodded at me. “Half the kids on each list are in one of your classes.”

I think he made up the felonies list. I hope.

Fall of year two was about as tough a time as I’ve ever had as a teacher.

Getting quiet for teaching was job one. I’d separate inveterate chatters, then I’d move the worst offenders to the front groups, and then, if one of them still didn’t shut up, I’d pull the desk forward all the way to a wall (with the kid in it). The rest of the class would snicker at the talker—at, not with.

“It’s not like I’m going to pay attention to you up here. I’ll just go to sleep,” one of them said, defiantly.

“You say that as if it were a bad thing.”

He or she often did go to sleep, which gave me some quiet from that corner, anyway. Otherwise, I wrote a referral. I also wrote referrals when they called me a f***ing [noun of your choice, profane or not], a sh**ty f**ing boring teacher (boring! I ask you), when they threw things, when they got up and wandered around the room refusing to sit, when they texted in open view and refused to give over the phone, when they left the room without permission, when they howled I HAVE TO PISS at the top of their voices (usually one at a time), and so on—all during the time that I was trying to teach the lesson “up front”. Once I released them for work it got easier, as I wasn’t trying to maintain order and some notion of what I’d been doing before the last interruption, but rather walking around the room helping students and telling others to shut up.

As bad as I make it sound, every senior teacher I worked with was astonished at how well I did, given the pressure; all the previous teachers stuck with all algebra all the time had routinely lost control of the classes and had supervisors posted. Administrators didn’t approve of my approach, alas; since my kids were mostly Hispanic, my referrals were, too. So I was caught between an administration who would really rather I’d have flailed ineffectually than kick kids out for order, and the bulk of my students, who opined frequently that I should boot students more often and earlier.

The beginning of the way back up that year began in second period when I’d thrown out the third kid of the day, and Kiley said “Could you toss out Elijah, while you’re at it?” and much of the class laughed. Elijah stood up and said “Yeah, send me, too! I don’t want to be here! Let me go!”

I tend to stay pretty focused on teaching; rarely do I give A Talk. Today, I have no idea why I made an exception.

“Why don’t you want to be here, again?”

“Because I hate math? F***ing duh.”

“What is it you think I want?”

“You want me to shut up.”

“Well, yeah. But why?”

“So you can teach!”

“Why?”

“Because it’s your job!”

“Because I want everyone to pass this class.” And to this day, I thank all that’s holy that I caught the class’s sudden silence and realized that my remark had an impact.

“Maybe I need to make that clearer. I want every single person in here to pass algebra and move onto geometry. Remind me again, how many people have taken algebra more than once?” Almost everyone in the class raised a hand, including Elijah.

“Yeah. Don’t raise your hand, but I know at least ten students in here are taking it for the third time, including some people who get tossed out of class regularly. I don’t kick kids out for fun. I kick them out because I need to teach everyone. I have kids who want to excel in algebra. I have kids who would like to get better at algebra. I have kids who simply would like to survive algebra, although many days they think that’s a pipe dream. And I have kids who don’t want to be here at all. I figure, I kick kids out from the last group, I’m meeting everyone’s goals but mine.” I actually get a couple laughs; they’re listening.

“But make no mistake, that’s my goal. I want everyone in here to pass.” I looked at Elijah, who’d slipped back into his chair, his eyes fixed on me.

“You could tell me about your troubles, and I’ll give you an ear, but here’s a basic truth: there’s not a single situation in your life that gets worse if you pass algebra. And there’s a whole bunch of things that improve.”

“I could get a work permit, for one thing,” Eduardo muttered.

“Get back on the football team,” said DeWayne.

“And now I know some of you are thinking sure, there’s a catch. No. I didn’t say I want you to like algebra. I didn’t even say I want you to understand algebra, although I guarantee that trying will improve your understanding. I’m making a simple commitment: show up and try. You will get a passing grade. No catch.”

The rest of second period, the toughest class, went so well that I decided to repeat that little speech for every class, and in every class, I got utter quiet. I don’t say that all the problems were solved that day, but from that point on far more of the kids “had my back”. Psychologically, their support made it much easier for me to develop a strategy to teach algebra in the face of these challenges.

Here’s how I taught it, and here’s how they did. I only failed 10 kids out of the final 90, or 11%. (Elijah had left. Eduardo got his permit, and DeWayne made it back onto the football team.) That’s the highest failure rate I’ve ever had, but then it’s the last time I taught algebra I. It’s easier to work with kids in geometry and algebra II—they’ve got skin in the game, and graduation becomes a real objective as opposed to the remote possibility it presents to a sophomore taking algebra I for the third time.

The wise reader can infer much about my students and a great deal, although certainly not all, about my values and priorities as a teacher from that tale.

First, I mostly teach kids from the lower third to the middle of the cognitive ability spectrum, with a few outliers on each end. That’s who takes algebra in high school. No more than 10% of my students in any year are capable of genuinely comprehending an actual formal math course in geometry or algebra (I or II). Another 30-50% of the rest are perfectly capable of understanding geometry, algebra and even more advanced topics in applied math, even if they couldn’t really master a formal math course, but they’d have to try a lot harder and want it much more. About a quarter of my students each year are barely capable of learning basic algebra and geometry well enough to apply it in simple, rote situations. A much smaller number can’t even manage that much.

For other teachers, the percentages are skewed heavily to the first and second categories; some of them don’t even know there’s a third and fourth category. A teacher covering precalc and honors algebra II/trig in high-income or Asian suburb, teaching mostly freshmen and sophomores, would have a much higher percentage of students who could master a formal course; their notion of “struggling kids” would be those who aren’t working hard enough. But that’s not my universe—and it’s not the universe I signed up for, although I wouldn’t mind visiting occasionally.

Until this year, my assignments weren’t deliberate. I was just an unimportant teacher who schools didn’t care about losing. In fact, the following year at that same school the administration assigned Algebra II/Trig classes to a teacher who was not qualified to teach the subject while I, who was qualified, was given the lower level Algebra II classes. The administration knew full well about the distinction, which necessitated a “your teacher is not highly qualified” letter to some 90 kids, but that teacher was more valuable than than I was, and so it goes. I’m not bitter, and I’m not marking time until I get “better” kids. I’m doing exactly what I want to do. But every teaching decision I make must be considered in light of my students’ cognitive abilities and, related to that ability, their motivation.

Second, I am a teacher who doesn’t overvalue any individual student at the expense of the class, which means I have no compunction about kicking kids out for the day. You run into these teachers philosophically opposed to removing kids from class; how can these students learn if they aren’t in class, they bleat. These teachers never seem to worry about how all the other kids learn with a disruptive hellion wreaking havoc because, they strongly hint (or outright assert), the right curriculum and caring teachers would eliminate the need to disrupt.

I ask these teachers, politely, do you have kids with tracking bracelets and/or probation officers? Do you have students who have fathered two kids while wearing that tracking bracelet, or gave birth to one? Do you have students who have been suspended or expelled for putting other students in the hospital, or for having a knife in their backpack? Do you have students who routinely tell you to f*** off and don’t bother me? Do you have all of these students plus twelve more who have just enough motivation that, given no distractions, would be able to learn some math but with a distraction will readily jump over to the side telling you to f*** off? And with all that, are you math teachers trying to help students with a four-year range in skills figure out second year algebra? Because otherwise, you can go sing your smug little songs of no student left behind to someone with kids who really shouldn’t be kicked out of the classroom. Okay, maybe not politely.

Come back the next day or even the same day, hat in hand, and no harm, no foul. I don’t only act like it didn’t happen, I have completely forgotten it happened. But get out of my class if you won’t shut up or can’t consider the day a success unless you’ve sucked in three other kids with your distractions.

The biggest pressure on teachers like me these days is the huge pushback they get from administration, district, and state/federal education agencies when they try to maintain an orderly classroom. And charter schools’ ability to a) have none of these kids to start with and b) kick moderately ill-behaved kids back to public school when they act out can’t be overstated as factor in their “success”.

That’s a shame. Because invariably, the bulk of my unmotivated rabble-rousers realize that I really mean it about that whole “passing” thing, if they would just shut up and give the class a shot. And so they do.

Next, I am a teacher who explains. I don’t mean lecture; my explanations always take the form of a semi-Socratic discussion, leading the kids through a process. But when I start to talk, the conversation has a direction and that directed conversation, to me, is the heart of teaching. One of my favorite memories of an ed school classmate came about as we were driving to our placement school.

“I’m really enjoying working on aspects of my teaching that I don’t like. For example, explanations. I hate doing that.”

“Um, what? You hate explanations?”

“Yeah. I’d rather never explain anything.”

Pause.

“What is teaching, if it’s not explaining things?”

I thought it was a rhetorical question. I was wrong. He went on and on about other aspects of teaching: curriculum, motivation, role modeling, assessing students, and so on. Huh. Interesting. Eye-opening. It’s not that I disagreed, but how can you be a teacher if you don’t like to explain things?

And as I began to develop, I realized that teaching is not synonymous with explaining. Still. It’s my go-to skill, it’s what I do best, it’s a big part of my success with low ability students, and it’s why I prioritize getting my students to shut up while I’m teaching up front.

Next, the story reveals that I adopt my students’ values and goals, rather than insist they adopt mine. The kids were shocked into silence when they realize that my most heartfelt goal was to pass everyone in the class.

I learned a key lesson I still use every time I meet a new class, and make it clear I want to help them achieve their goals, which usually involve surviving the class. I do not understand why so many teachers set out objectives based on the assumption that they will successfully re-align their students’ value systems.

And in a related revelation, you can see how I frame my task. In his TED talk, Dan Meyer asks the audience to imagine:

“you really loved something…and you recommended it wholeheartedly to someone you really liked…and the person hated it. By way of introduction, that is the exact same state in which I spent every working day of the last six years. I teach high school math. I sell a product to a market that doesn’t want it but is forced by law to buy it.”

All math teachers can relate to this statement; it’s clever, funny, and does a good job of introducing the fundamental dilemma of high school math teachers: most kids hate math and are required to take it. Many dedicated math teachers would not only relate, but agree with Dan’s framing of his task as a sales job, regardless of their teaching ideology. When I say I disagree, it’s not because Meyer is wrong but because we approach our jobs in fundamentally different ways. I don’t love math, and I’m not selling a product.

Victoria: I’m terrible. I know I’m terrible. I look at the mirror and I’m ashamed. Maybe I should quit. I just can’t seem to do anything right.

Joe Gideon: Listen. I can’t make you a great dancer. I don’t even know if I can make you a good dancer. But, if you keep trying and don’t quit, I know I can make you a better dancer. I’d like very much to do that. Stay?

Were it not for the unfortunate plot point about Joe Gideon’s motives for hiring Victoria for the show (he was a hounddog who had her in bed an hour after they first met), I suspect more math teachers would reveal that they can quote this scene from All That Jazz verbatim.

And those math teachers mostly would agree with me. Teaching math, for us, isn’t about creating mathematicians. It’s only occasionally about working with kids who want to be engineers, doctors, or architects. Mostly, it’s about giving kids enough math skills to pass a college placement test so they won’t end up spending a fortune on remedial math classes and never get any further—or at least enough skills so they’ll pass a remedial math class and move on. Or giving kids enough math so they look at a trade school placement test and think, “Hey, I can do this.” Or just giving kids the will to pass the class and keep them out of mindless credit recovery in alternative institutions, letting them feel part of the educational system, not a failure who couldn’t cut it at normal high school.

We don’t promise miracles. We do promise “better”.

Finally, though, the story indicates that I am acutely aware of all my students’ motivations, that not all my students just want to pass. I have bright kids in almost every class, I have highly motivated kids, I have kids with specific objectives, most of whom want to learn as much as they can. I never forget them, and if I can’t dedicate my entire teaching agenda to meeting their goals, it’s only because I owe allegiance to all my students. I never stop looking for better ways to give these kids what they need while still ensuring I meet my overall responsibility. Many other teachers say these kids should come first. I always worry they might be right. But as I said above, I do not overvalue any individual kid over the needs of the entire class.

This tale doesn’t tell much about how I teach, but that particular topic gets plenty of coverage in other essays.

Anyone who is familiar with reform math can probably infer not only my teaching values and priorities, but also a lot of reasons why I’m not crazy about reform math. But I’ll go into details in the next post.