Assessments with Multiple Answers

Multiple Answer Math tests are my new new thing, and I’m very pleased with how it’s going so far. I thought I’d talk about some of the problems in depth, see if anyone has suggestions.

Most of these questions come from an A2/Trig test I wrote this weekend, focusing on systems of equations, but my tests are always cumulative.

One of the things I really like about this format: I can combine free response and selected responses very easily. So here they had to graph the plane, then answer questions which may or may not have to do with the graph. So I could both test their ability to graph a plane see if they understand how distance works in three dimensions, check out their attention to detail, and see if they remember what a trace is. Query: is “slope of a trace line” acceptable? I’ve never taught 3-dimensional systems before, and the book only said “trace”. But when I was teaching it, I kept forgetting and say “trace line”. I wanted them to demonstrate they could visualize the plane in three dimensions and see the slopes of the lines forming the plane, and I couldn’t find any sample questions. Probably an oddball question.

“a” and “e” contain typos. I originally had a different line, until I realized it’d be too hard to graph on the coordinates I provided. So I changed everything, or tried to. Missed two things. First, I intended “a” to be correct, but forgot to change the constant. That’s okay, it will allow for attention to detail. But “e” is just a kluge question, since I changed the points but forgot to change the distance. Before, it was a test of evaluation; now it’s a more obvious wrong answer.

This question makes me very happy. Transformations, function operations, evaluation, and then a transfer of knowledge test! We’d never done any problems like “e” before. No one squawked, and I even saw some kids solving it graphically.

(I stole this graph from online, but can’t find it any more. If it’s yours, let me know and I’ll provide a link.)

I tiptoed conceptually into linear programming, but we did a lot with feasibility regions and of course, systems of inequalities. I describe my approach for Algebra II, but I step it up a bunch of paces for A2/Trig. I expect them to be able to graph lines and inequalities. They get review during the modeling section, but that’s all.

Another one I just think is elegant because it approaches the absolute value from so many different angles: algebraically, graphically, and then a function conceptual question for good measure.

I use this on both Algebra II and Algebra II/Trig. We math teachers try to beat into the kids’ heads the idea that a function can be defined or expressed in four ways: verbal, algebraic, graphic, and tabular. If this were a multiple choice question, students would just test one value and see what happens. But it’s multiple answer, and plugging in numbers takes a long time. Plotting the points and sketching the lines, on the other hand, works very nicely and very quickly—if you know how to graph those lines.

Every so often you can really mess with the kids’ minds, like this:

None of the “obvious” answers are right. The kids really have to trust their abilities.

This is almost pure concept. I introduced the algebra of rational expressions; we’ll do the graphs later. Well over half the kids correctly selected e, but a lot missed b. Ack.

Here’s a couple that work for either pre-calc or algebra II. The quadratic runs the gamut from conceptual to technical. The circle question is more purely technical, but that’s because there’s a lot to test.

I’m having a much easier time grading these now, once I realized I was actually creating True/False tests.

Still to be resolved, however: I have to distinguish between “left the problem blank because I didn’t know” and “not true”. Right now, I evaluate the test to determine what the student is doing, but in the future I think I’m going to have a field they can mark “T” or “F”. If it’s blank, it’s wrong.

So, for example, take a look at this question again:

Answers A, D, and E are true. The others are false. I give this question 14 points, 2 for each letter.

Almost all my students correctly select A as true, because they’ve built the equation themselves as an exercise and understand the parameters. They likewise know that B is false. Some of them read “maximum” as “initial” and wrongly select C, but many otherwise weak students with good attention to detail get it correct. So even my weak students are likely to get 6 points on these three letters.

Then we get to the tougher ones (they aren’t always in order of difficulty). Students have to understand what elements of the parabola equate to max height, time to max height, and zero height. Obviously, I cover these extensively, but kids have a harder time with this. I don’t just teach them a method. I expect them to know that max height is the parabola’s vertex, so that the x value is time to max height, and the y value is the height.

I had at least 12 students who correctly factored the problem, thus correctly NOT selecting E, but also NOT selecting D. Strong technically, weak on the concept of a “zero”. I gave them partial credit (a point) and yelled at them on the paper: things like “Noooooooooo!” and “Arggggghhhh.” and arrows and question marks and “Yo! What do you think (2t-3) means, exactly!?!”

The vertex questions E and G give students the most trouble, but that seems to be less about concept and more about a reluctance to work with fractions. My algebra II students actually do better than my precalc students because we spend a whole unit on this, as opposed to a few days in precalc.

So an average weak student will get 8-10 points out of 14. Very few students get all 14 points, maybe 8 out of 60. Most get 10-12. If they show their work and I can see they were on the right track with just an algebra error, I give partial credit. Other times, I can clearly see their math was terrible, even if they got the right answer. In those cases, I mark the question correct and then dock them 2 points for bad math.

While I don’t normally review tests, I always go through these and give the correct answers and discuss grading decisions.

I strongly recommend giving these a try. They’re lots of fun to make and again, typos are a lot easier to hide.

Content Knowledge and Reading Comprehension: Bold Talk and Backpedaling

Empty buckets seldom burst into flames. –Robert Pondiscio, Literacy is Knowledge.

People who push curriculum as a solution are generally pushing content knowledge, and they’re pushing content knowledge as a means of improving reading comprehension. Most of these people are in some way associated with Core Knowledge, the primary organization pushing this approach. They aren’t pushing it for money. This is a cause.

Pondiscio’s piece goes to the same well as E. D. Hirsch, who founded the Core Knowledge Foundation to promote the cause of content knowledge in curriculum, Lisa Hansel, the CK Foundation’s current Pondiscio, and Daniel Willingham, who sits on the board of Core Knowledge.

Pondiscio even borrows the same baseball analogy that Hirsh has used for a decade or so, to illustrate the degree to which content knowledge affects reading comprehension. Many Americans are unfazed by “A-Rod hit into a 6-4-3 double play to end the game”, but might be confused by “I’ll see how the wicket is behaving and then decide who are the bowlers I’ll use in the last few overs.”

We can understand content if we have the background knowledge, Hirsh et. al. assure us, but will “struggle to make sense” of reading if we’re unfamiliar because, as Pondiscio asserts, “Prior knowledge is indispensable”.

Let’s take a look at what some people do when they read without requisite content knowledge. (you can see other examples from my early childhood here).

Let’s pick another sporting event—say, the Kentucky Derby, since I don’t pay much attention to it. I googled, saw a headline at Forbes: “Final Kentucky Derby Futures Wagering Pool Opens Today”.

I don’t watch horseracing, I don’t bet, I know about futures because they were a plot point in “Trading Places”, but until that google I had no idea that people could bet on who won the Derby now, in advance. And now I do.

I was not confused. I didn’t struggle, despite my lack of prior knowledge. I constructed knowledge.

But Pondiscio says that any text on horse-racing is a collapsing tower of wooden blocks, “with each block a vocabulary word or a piece of background knowledge”, to anyone unfamiliar with horseracing. I have too few blocks of knowledge.

Robert Pondiscio would no doubt point out that sure, I could figure out what that Kentucky Derby headline meant, because I knew what the Kentucky Derby was. True. I’ve known what the Kentucky Derby was ever since I was 9 or so. I didn’t get the information from my parents, or my privileged life (I grew up decidedly without privilege). I read through all the Highlight articles at the doctor’s office and picked up a Sports Illustrated out of desperation (the internet is a glorious place; I just found the article) and then did exactly what Pondiscio suggests is impossible—read, understood, and learned when before I knew nothing.

I first knew “derby” as a hat, probably from an Enid Blyton story. But I had recently learned from “The Love Bug” that a derby was also a race. What did racing have to do with hats? But now I learned that horse races could be derbies. Since horses were way older than cars, the car races must have gotten the “derby” idea from horses. Maybe jockies got hats when they won horse races. (I learned many years later, but before today, that I was wrong.) I not only built on my existing knowledge base, I learned that the Kentucky Derby was a yearly horse race almost a century old and the results this year were upsetting. No one expected this horse to win, which probably was why people were upset, because just like the bad guy had a bet with the Chinese guy in “The Love Bug”, people made bets on who won. The article also gave me the impression that horses from Venezuela don’t always win, and that lots of horse races had names.

Pondiscio gives another example of a passage requiring background knowledge: the Dutch in New Amsterdam. Oddly enough, I distinctly remember reading just that sort of passage many years ago back in the fifth or sixth grade, about New Amsterdam first being owned by the Dutch, then control going to the English. I knew about Holland from Hans Brinker, which I’d found in someone’s bookshelf, somewhere, when I was six or seven. So New York was first founded by the Dutch–maybe that’s why they called the dad Mynheer in Legend of Sleepy Hollow just like they did in Hans Brinker, because according to the cartoon I’d seen on Wonderful World of Disney, Sleepy Hollow took place in New York .And then the English took it over, so hey, York must be a place in England. So when done, I knew not only that the Dutch had once been in the New World, but that other countries traded colonies, and that while we all spoke English now, New York had once been Dutch.

I didn’t carefully build content knowledge. I just got used to making sense of chaos, grabbing onto whatever familiar roadmarks I saw, learning by a combination of inference and knowledge acquisition, through haphazard self-direction grabbing what limited information I could get from potboiler fiction, magazines, and limited libraries, after gobbling up all the information I could find in schoolbooks and “age-appropriate” reading material. And I learned everything without prior knowledge other than what I’d acquired through previous reading, TV and movies as came my way. I certainly didn’t ask my parents; by age six I acknowledged their expertise in a limited number of topics: cooking, sports, music, and airplanes. In most important topics, I considered them far less reliable than books, but did deem their opinions on current events useful. Yes. I was obnoxious.

My experiences are not unique. Not today, and certainly not in the past. For much of history, people couldn’t rely on information-rich environments and supportive parents to acquire information, so they turned to books. Using vocabulary and decoding. Adding to their existing knowledge base. Determinedly making sense of alien information, or filing it away under “to be confirmed later”.

But of course, say the content knowledge people pushing curriculum. And here comes the backpedal.

E. D. Hirsch on acquiring knowledge:

Almost all the word meanings that we know are acquired indirectly by intuitively guessing new meanings as we get the overall gist of what we’re hearing or reading.

That describes almost exactly what I did for much of my childhood. But this is the same Hirsch who says “Reading ability is very topic dependent. How well students perform on a reading test is highly dependent on their knowledge of the topics of the test passages.” Nonsense. I scored at the 99th percentile of every reading test available, and I often didn’t know anything about the topic of the test passage until I read it—and then I’d usually gleaned quite a bit.

Pondiscio slips in a backpedal in the same piece that he’s pushing content.

Reading more helps, yes, but not because we are “practicing” reading or improving our comprehension skills; rather, reading more is simply the most reliable means to acquire new knowledge and vocabulary.

This is the same Pondiscio who said a couple years ago:

What is needed is high-quality preschool that drenches low-income learners in the language-rich, knowledge-rich environment that their more fortunate peers live in every hour of every day from the moment they come home from the delivery room.

Well, which is it? Do they think we learn by reading, or that we only learn by reading if we were fortunate enough to have parents who provided a knowledge-rich environment?

Take a look at the Core Knowledge promotional literature, and it’s all bold talk: not that more content knowledge aids comprehension, but that content knowledge is essential to comprehension.

I’ve likewise tweeted about this with Dan Willingham:

Me: Of course, taken to its logical conclusion, this would mean that reading doesn’t enable knowledge acquisition.

Willingham: if you have *most* of the requisite knowledge you can and will fill in the rest. reading gets harder and harder. . . ..as your knowledge drops, and the likelihood that you’ll quit goes up.

Me: The higher the cog ability, the higher ability to infer, fill in blanks.

Willingham: sooo. . . hi i.q. might be better at inference. everyone infers, everyone is better w/ knowledge than w/out it. yes?

So Willingham acknowledges that IQ matters, but that as knowledge and IQ level drops, engagement is harder to maintain because inference is harder to achieve. No argument there, but contrast that with his bold talk here in this video, Teaching content is teaching reading, with the blanket statements “Comprehension requires prior knowledge”, and attempts to prove that “If you can read, you can learn anything” are truisms that ignore content knowledge. No equivocation, no caveats about IQ and inference.

So the pattern: Big claims, pooh-poohing of reading as a skill that in and of itself transfers knowledge. If challenged, they backpedal, admitting that reading enables content acquisition and pointing to statements of their own acknowledging the role reading plays in acquiring knowledge.

And then they go back to declaring content knowledge essential—not useful, not a means of aiding engagement, not important for the lower half of the ability spectrum. No. Essential. Can’t teach reading without it. All kids “deserve” the same content-rich curriculum that “children of privilege” get not from schools, but from their parents and that knowledge-drenched environment.

And of course, they aren’t wrong about the value of content knowledge. I acknowledge and agree with the surface logic of their argument: kids will probably read more readily, with more comprehension, if they have more background knowledge about the text. But as Daniel Willingham concedes, engagement is essential as well—arguably more so than content knowledge. And if you notice, the “reader’s workshop” that Pondiscio argues is “insufficient” for reading success focuses heavily on engagement:

A lesson might be “good readers stay involved in a story by predicting” or “good readers make a picture in their mind while they read.” ..Then the children are sent off to practice the skill independently or in small groups, choosing from various “high-interest” books at their individual, “just-right” reading level. [Schools often have posters saying] “Good readers visualize the story in their minds.” “Good readers ask questions.” “Good readers predict what will happen next.”

But Pondiscio doesn’t credit these attempts to create engagement, or even mention engagement’s link to reading comprehension. Yet surely, these teachers are simply trying to teach kids the value of engagement. I’m not convinced Pondiscio should be declaring content knowledge the more important.

Because while Core Knowledge and the content folks have lots of enthusiasm, they don’t really have lots of research on their product, as Core Knowledge representatives (q6) acknowledge. And what research I’ve found never offers any data on how black or Hispanic kids do.

Dan Willingham sure seemed to be citing research lately, in an article asking if we are underestimating our youngest learners, citing a recent study says that we can teach young children knowledge-rich topics like natural selection. He asks “whether we do students a disservice if we are too quick to dismiss content as ‘developmentally inappropriate,'” because look at what amazing things kids can learn with a good curriculum and confidence in their abilities!

Of course, a brief perusal of the study reveals that the student populations were over 70% white, with blacks and Hispanics less than 10% total. Raise your hand if you’re stunned that Willingham doesn’t mention this tiny little factoid. I wasn’t.

Notice in that study that a good number of kids didn’t learn what they were taught in the first place, and then a number of them forgot it quickly. Which raises a question I ask frequently on this blog: what if kids don’t remember what they’re taught? What if the information doesn’t make it to semantic memory (bottom third of essay). What evidence do the curriculum folks have that the kids will remember “content” if they are taught it in a particular sequence? (Note: this essay was too long to bring up Grant Wiggins’s takedown of E. D. Hirsch, but I strongly recommend it and hope to return to it again.)

Like reformers, curriculum folk are free to push the bold talk, because few people want to raise the obvious point: if content knowledge is essential, instead of helpful, to reading comprehension, then no one could ever have learned anything.

But contra Pondiscio, empty buckets do burst into flames. People do learn without “essential” content knowledge. Even people from less than privileged backgrounds.

Here’s the hard part, the part too many flinch from: Smart people can learn this way. All anyone has ever needed to acquire knowledge is the desire and the intellect. For much of history educated people had to be smart and interested.

In recent years, we’ve done a great job at extending the reach of education into the less smart and less interested. But the Great Unspoken Truth of all education policy and reform, be it progressive, critical pedagogy, “reform” or curricular, is that we don’t know how to educate the not-smart and not-interested.

Boobies and Bronies

So Grayson Bruce can take his My Little Pony backpack to school. All hail tolerance.

Matt Walsh and Sean Williams (neither of whom I’ve ever read before) have covered most of one aspect of this insanity—namely, there’s something wrong with a 9 year old boy taking 4-year-old girl paraphernalia to school, and parents have a responsibility to help their kids recognize reality. (I’m not entirely convinced that there isn’t something wrong with naming a 9 year old Grayson Bruce, but let’s leave that aside.)

But since reasonable people can disagree, surely public schools or their districts should be able to decide what approach they want to take. They might, for example, realize that nine-year-old kids are developmentally prone to demand peer pressure, so set behavior guidelines that keep the circle pretty small. So when a boy with a little girl’s backpack faces derision and taunting, the school might see both the taunting and the backpack as problems. But no, the mother takes the story public, and a wide range of idiots go out of their way to denounce the school for not letting a 9 year old boy have a my little pony backpack.

Or, at a time when kids are still figuring out the difference between boys and girls, a district might prefer to gently restrict a gender-confused kid to the bathroom of his or her biological birth. But no, state cobbles together a law that allows kids to choose what gender they are and use the gendered bathroom of their choice.

Or suppose a district has clear policies for handling online bullying on Twitter or Facebook between studens. Meanwhile, two sophomores from School A go to an unsupervised party in an entirely different district. While at that party, they make snarky remarks about a student who attends School B. Other kids from School A join in. Pretty soon, they start to tweet these remarks, and put a few of them on facebook. Other School B kids mention this to the student, who becomes upset. The parents come in with a lawyer to talk to School B’s principal, demanding that this stop because schools are responsible for cyber-bullying. School B’s principal is now held responsible for behavior that didn’t occur on campus, wasn’t committed by his students at a party that wasn’t even in his district.

Or, districts or schools might decide to eliminate the scenario of 15 year old boys snickering about their boobie bracelets, shrieking “I HEART BOOBIES!” as the girls with the actual boobies giggle and ask if they heart ALL boobies? Like these ones here? So the district says no bracelets in school. Meanwhile, a couple of middle school girls—yeah, girls. Because girls wearing the bracelets are the problem—decide to sue. The federal court sides with the ACLU and says kids wearing the bracelets “want to remove the stigma of breasts”—and then the Supreme Court, with all those conservatives, sides with the ACLU. So now, teachers have to tolerate freshman boys leering and saying they ‘heart boobies’ and do so in the knowledge that the state and federal courts valued their ability to do that more than the ability of districts to maintain reasonable decorum.

It is sometimes hard not to conclude that the public is the greatest enemy of public education. But at this point, I’ve still got courts in the first slot, the feds in the second, and state legislatures in third.

Why I Blog

I’m pretty sure Dan Meyer wasn’t asking the likes of me, but as I read through the comments, I was reminded again that many teachers blog because they want a community.

I do not want a community, either virtual or meat space. In 20 years, I have had an actual casual acquaintance with just one neighbor, although I can’t remember his name. I may never have known it. He was a chain-smoking, beer guzzling truck driver, we both had teenage sons. A few years ago, the fire alarm at my last apartment complex went off in three buildings at 2:00 am, and so awful a sound was it that we all ran from our apartments fearing not fire but the very real possibility that we’d stab our eardrums with corkscrews in hopes of ending the torture. So we’re all downstairs milling about, dozens if not hundreds of Chinese, Indians, and the occasional Filipino, all of them complete strangers to me and each other, and everyone looking to me for answers. I thought at first they assumed the only white person in sight was custodial staff or maybe off-duty management (hey, I have an ego), but apparently they had designated me the person in charge of conversing with the fire fighters. When the big red trucks showed up, I represented with authority, telling the firemen (they were all men) that, while the community would not be ungrateful for any conflagrations found and extinguished, the Really Important Thing was to stop that horrible noise, which was this sound at this frequency. The firefighters obligingly made the dreadful noise go away, and I got all the credit—perhaps the Chinese thought money was involved, I don’t know. Anyway, the firefighters told me it was a false alarm, I thanked them, and started the trek back upstairs when a Indian gentleman approached me and asked for a status update. Good lord, I’d forgotten my leadership role. I turned, found a small mound of lawn to stand on, and they gathered round while I told them to go back to bed. They smiled, waved, and I never saw any of them again.

I’m not anti-social; in fact, I’m quite friendly. I can’t easily explain why I don’t seek out communities, but perhaps this OCEAN profile (that’s Big Five, right?) explains the paradox (I took the test for the first time last month when a commenter linked it in). Notice that I’m slightly above average on extraversion (in fact, it probably overstates my sociability) but abnormally low on agreeableness—that is, I’m quite comfortable with rudeness. In Myers-Briggs, I’m an INTP, but I’m very much a gregarious introvert.

All of this is to say that I occasionally run into trouble with math teacher bloggers. They’re mad because I disrespected Dan, or because I simply think they’re wrong or because I hold that cognitive ability plays a role in educational outcomes or….you know, I can’t really keep track of all the reasons.

Some of the problems arise because I’m blogging for an entirely different reason than most teachers are, particularly most math teachers. I’m not seeking bonds, looking for collaborative opportunities, looking to share, to network, to get validation. In fact, I’m not really a blogger at all. I’m a writer. I write to convince, to argue, to persuade–even, I hope, to entertain. But I’m not having a conversation.

In a much bigger galaxy, far far away, think about The View. The math teacher bloggers see themselves as Barbara, Joy, Whoopi, and so on (please don’t make me look up the names, the only time I watch the show is when my dad’s in town). Maybe Ellen. They want to exchange opinions and ideas in a safe place, with people they trust, the audience and the stars all part of a big, productive conversation in which everyone can express opinions in a restricted range in a polite voice. If so, they will be respected and heard, disagreement will be polite and constructive. To put it mildly, that’s not how I roll. In that galaxy, I’m akin to a Krauthammer, Brooks, McArdle, albeit several thousand rungs lower on the ladder. It’s kind of fun to imagine how these writers would response to Barbara Walters asking them, sternly, if they thought their columns were contributing to a productive conversation.

So every so often some teacher blogger tells me in frustration that I’m not being productive, that I’d get a better response if I were more constructive, engaged more, and I’m like dude, what are you talking about? I have so much bigger an audience than I ever dreamed of when I began two years ago. But while I’m happy to welcome them to my audience, they aren’t really who I’m writing for. And they can’t conceive of what I’m talking about, because they are writing for each other, to reaffirm their connections, their community, their sounding boards. (Keeerist, it sounds so very girly.)

I don’t want to make it sound like Them and Me. The vast majority of these folks don’t know I exist until I do something to offend. Many of them have far bigger teacher audiences than I, and are far better known. And while they don’t like me, I like many of them as teachers, read their blogs frequently and comment occasionally. But even in my comments, it’s clear I’m coming from a different planet.

Paul Bruno, the only other US teacher I know of who writes broadly about policy, doesn’t seem to experience this dichotomy between his writing and his place in the teacher “community” . But then, he doesn’t write much about teaching, and of course he has, er, a different Big Five score. Probably not as comfortable being rude.

Anyway. I thought I would write this post for the teacher audience, so the next time I’m lectured for being unproductive, I can just link this in.

I also blog for the reasons expressed here.