Assessing Math Understanding: Max, Homer, and Wesley

This is only tangentially a “math zombies” post, but I did come up with the idea because of the conversation.

I agree with Garelick and Beals that asking kids to “explain math” is most often a waste of time. Templates and diagrams and “flow maps” aren’t going to cut it, either. Assessing understanding is a complicated process that requires several different solutions methods and an interpretive dance. Plus a poster or three. No, not really.

As I mentioned earlier, I don’t usually ask kids to “explain their answer” because too many kids confuse “I wrote some words” with “I explained”. I grade their responses in the spirit given, a few points for effort. “Explain your answer” test questions are sometimes handy to see if top students are just going through the motions, or how much of my efforts have sunk through to the students. But I don’t rely on them much and apart from top students, don’t care much if the kids can’t articulate their thinking.

It’s still important to determine whether kids actually understand the math, and not just because some kids know the algorithm only. Other kids struggle with the algorithm but understand the concepts, Still others don’t understand the algorithm because they don’t grok the concepts. Finally, many kids get overwhelmed or can’t be bothered to work out the problem but will indicate their understanding if they can just read and answer true/false points.

If you are thinking “Good lord, you fail the kids who can’t be bothered or get overwhelmed by the algorithms!” then you do not understand the vast range of abilities many high school teachers face, and you don’t normally read this blog. These are easily remediable shortcomings. I’m not going to cover that ground again.

So how to ascertain understanding without the deadening “explain your answer” or the often insufficient “show your work”?

My task became much easier once I turned to multiple answer assessments. I can design questions that test algorithm knowledge, including interim steps, while also ascertaining conceptual knowledge.

I captured some student test results to illustrate, choosing two students for direct comparison, and one student for additional range. None of these students are my strongest. One of the comparison students, Max, would be doing much better if he were taught by Mr. Singh, a pure lecture & set teacher; the other, Homer, would be struggling to pass. The third, Wesley, would have quit attending class long ago with most other teachers.

To start: a pure factoring problem. The first is Max, the second Homer.

Both students got full credit for the factoring and for identifying all the correct responses. Max at first appears to be the superior math student; his work is neat, precise, efficient. He doesn’t need any factoring aids, doing it all in his head. Homer’s work is sloppier; he makes full use of my trinomial factoring technique. He factored out the 3 much lower on the page (out of sight), and only after I pointed out he’d have an easier time doing that first.

Now two questions that test conceptual knowledge:

Max guessed on the “product of two lines” question entirely, and has no idea how to convert a quadratic in vertex form to standard or factored. Yet he could expand the square in his head, which is why he knew that c=-8. He was unable to relate the questions to the needed algorithms.

Homer aced it. In that same big, slightly childish handwriting, he used the (h,k) parameters to determine the vertex. Then he carefully expanded the vertex form to standard form, which he factored. This after he correctly identified the fact that two lines always multiply to form a quadratic, no matter the orientation.

Here’s more of Homer’s work, although I can’t find (or didn’t take a picture of) Max’s test.

This question tests students’ understanding of the parameters of three forms of the quadratic: standard, vertex, factored. I graded this generously. Students got full credit if they correctly identified just one quadratic by parameter, even if they missed or misidentified another. Kids don’t intuitively think of shapes by their parameter attributes, so I wanted to reward any right answers. Full credit for this question was 18 points. A few kids scored 22 points; another ten scored between 15 and 18. A third got ten or fewer points.

Homer did pretty well. He was clearly guessing at times, but he was logical and consistent in his approach. Max got six points. He got a wrong, got b, c, & d correct, then left the rest blank. It wasn’t time; I pointed out the empty responses during the test, pointing out some common elements as a hint. He still left it blank.

On the same test, I returned to an earlier topic, linear inequalities. I give them a graph with several “true” points. Their task: identify the inequalities that would include all of these solutions.

(Ack: I just realized I flipped the order when building this image. Homer’s is the first.)

Note the typo that you can see both kids have corrected (My test typos are fewer each year, but they still happen.) I just told them to fix it; the kids had to figure out if the “fix” made the boundary true or false. (This question was designed to test their understanding of linear concepts–that is, I didn’t want them plugging in points but rather visualizing or drawing the boundary lines.)

Both Max and Homer aced the question, applying previous knowledge to an unfamiliar question. Max converted the standard form equation to linear form, while Homer just graphed the lines he wasn’t sure of. Homer also went through the effort of testing regions as “true”, as I teach them, while Max just visualized them (and probably would have been made a mistake had I been more aggressive on testing regions).

Here I threw something they should have learned in a previous year, but hadn’t covered in class:

Most students were confused or uncertain; I told them that when in doubt, given a point….and they all chorused “PLUG IT IN.”

This was all Max needed to work the problem correctly. Homer, who had been trying to solve for y, then started plugging it in, but not as fluently as Max. He has a health problem forcing him to leave slightly early for lunch, so didn’t finish. For the next four days, I reminded students in class that they could come in after school or during lunch to finish their tests, if they needed time. Homer didn’t bother.

So despite the fact that Homer had much stronger conceptual understanding of quadratics than Max, and roughly equal fluency in both lines and quadratics, he only got a C+ to Max’s C because Homer doesn’t really care about his grade so long as he’s passing.

Arrgghhh.

I called in both boys for a brief chat.

For Max, I reiterated my concern that he’s not doing as well as he could be. He constantly stares off into space, not paying attention to class discussions. Then he finishes work, often very early, often not using the method discussed in class. It’s fine; he’s not required to use my method, but the fact that he has another method means he has an outside tutor, that he’s tuning me out because “he knows this already”. He rips through practice sheets if he’s familiar with the method, otherwise he zones out, trying to fake it when I stop by. I told him he’s absolutely got the ability to get an A in class, but at this point, he’s at a B and dropping.

Max asked for extra credit. He knew the answer, because he asks me almost weekly. I told him that if he wanted to spend more time improving his grade, he should pay attention in class and ask questions, particularly on tests.

We’ve had this conversation before. He hasn’t changed his behavior. I suspect he’s just going to take his B and hope he gets a different teacher next year who’ll make the tutor worth the trouble. At least he’s not trying to force a failing grade to get to summer school for an easy A.

I was extremely direct with Homer,  expressing (snarling) my disappointment that he wouldn’t make the effort to be excellent, when he was so clearly capable of more. What was he doing that was so important he couldn’t take 20 minutes or so away to finish a test, given the gift of extra time? Homer stood looking a bit abashed. Next test, he came in during lunch to complete his work. And got an A.

Max got a B- on the same test, with no change in behavior.

I haven’t included any of the top students’ work because it’s rather boring; revelations only come with error patterns. But here, in a later test, is an actual “weak student”, who I shall dub Wesley.

Wesley had been forced into Algebra 2, against his wishes, since it took him five attempts to pass algebra I and geometry. He was furious and determined to fail. I told him all he had to do was work and I’d pass him. Didn’t help. I insisted he work. He’d often demand to get a referral instead. Finally, his mother emailed about his grade and I passed on our conversations. I don’t know how, but she convinced him to at least pick up a pencil. And, to Wesley’s astonishment, he actually did start to understand the material. Not all of it, not always.

This systems of equations question (on which many students did poorly) was also previous material. But look at Wesley! He creates a table! Just like I told him to do! It’s almost as if he listened to me!

He originally got the first equation as 20x + 2y = 210 (using table values); when I stopped by and saw his table, I reminded him to use it to find the slope–or, he could remember the tacos and burritos problem, which spurred his memory. You can’t really see the rest of the questions, but he did not get all the selections correct. He circled two correctly, but missed two, including one asking about the slope, which he could have found using his table. He also graphed a parabola almost correctly, above (you can see he’s marked the vertex point but then ignored it for the y-intercept).

He got a 69, a stupendous grade and effort, and actually grinned with amazement when I handed it back.

Clearly, I’m much better at motivating underachieving boys than I am “math zombies”. Unsurprising, since motivating the former is my peculiar expertise going back to my earliest days in test prep, and I’ve only recently had to contend with the latter. However, I’ve successfully reached out and intervened with similar students using this approach, so it’s not a complete failure. I will continue to work on my approach.

None of the boys have anything approaching a coherent, unified understanding of the math involved. In order to give them all credit for what they know and can do, while still challenging my strongest students, I have to test the subject from every angle. Assessing all students, scoring the range of abilities accurately, is difficult work.

As you can see, the challenges I face have little to do with Asperger’s kids who can’t explain what they think or frustrated parents dealing with number lines or boxes of 10. Nor is it anything solved by lectures or complex instruction. My task is complicated. But hell, it’s fun.