Boaler’s Bias (or BS)

I began this piece a week ago intending to opine on the Boaler letter. However, I realized I have to confess a strong bias: I read Boaler in ed school and nearly vomited all over my reader. And that will take a whole post.

Experiencing School Mathematics: Traditional and Reform Approaches to Teaching and Their Impact on Student Learning

Boaler, a Brit who has held math education academic positions in England as well as at Stanford, performed a three-year study of two English schools, matched up in demographics and test scores. Phoenix Park believed in progressive, student-centered instruction, whereas Amber Hill taught a traditionalist method—more than traditionalist, they taught math by rote and drill, which is by no means required for teacher-centered instruction.

Boaler was ostensibly investigating the two instruction methods, but the fix was clearly in. Despite Boaler’s constant assurances that the Amber Hill teachers were dedicated and caring, the school presents as an Orwellian fantasy:

One of the first things I noticed when I began my research was the apparent respectability of the school. Walking into the reception area on my arrival, I was struck by the tranquility of the arena. The reception was separated from the rest of the school by a set of heavy double doors. The floors were carpeted in a somber gray; a number of easy chairs had been placed by the secretary’s window and a small tray of flowers sat above them. …Amber Hill was unusually orderly and controlled. Students generally did as they were told, their behavior governed by numerous enforced rules and a general school ethos that induced obedience and conformity. All students were required to wear a school uniform, which the vast majority of students wore exactly as the regulations required. The annual school report that teachers sent home to parents required the teachers to give the students a grade on their “co-operation” and their “wearing of school uniform.” The head clearly wanted to present the school as academic and respectable, and he was successful in this aim at least in terms of the general facade. Visitors walking around the corridors would see unusually quiet and calm classrooms, with students sitting in rows or small groups usually watching the board. When students were unhappy in lessons, they tended to withdraw instead of being disruptive. The corridors were mainly quiet, and at break times the students walked in an orderly fashion between lessons. The students’ lives at Amber Hill were, in many ways, structured, disciplined, and controlled

(page 13)

Phoenix Park, on the other hand:

…had an attractive campus feel. The atmosphere was unusually calm—described in a newspaper article on the school as peaceful. Students walked slowly around the school, and there was a noticeable absence of students running, screaming, or shouting. This was not because of school rules; it seemed to be a product of the school’s overall ambiance. I mentioned this to one of the mathematics teachers one day and she agreed, saying that she did not think she had ever heard anybody shout—teacher or student. She added that this was particularly evident at break times in the hall: “The students are all so orderly, but no-one ever tells them to be.”…. Students were taught all subjects in mixed-ability groups. Phoenix Park students did not wear school uniforms. Most students wore fashionable but inexpensive clothes such as jeans, with trainers or boots, and shirts or t-shirts worn loosely outside. A central part of the school’s approach involved the development of independence among students. The students were encouraged to act responsibly—not because of school rules, but because they could see a reason to act in this way.

(emphasis mine) (page 18)

And yet, while the Amber Hill students were well-behaved little automatons, the Phoenix Park kids–the ones who simply behave well by choice and idealism, not some lower-class aspiration to respectability–ran amok:

In the 100 or so lessons I observed at Phoenix Park, I would typically see approximately one third of students wandering around the room chatting about non-work issues and generally not attending to the project they had been given. In some lessons, and for some parts of lessons, the numbers off task would be greater than this. Some students remained off task for long periods of time, sometimes all of the lessons; other students drifted on and off task at various points in the lessons. In a small quantitative assessment of time on task, I stood at the back of lessons and counted the number of students who appeared to be working 10 minutes into the lesson, halfway through the lesson, and 10 minutes before the end of the lesson. Over 11 lessons, with approximately 28 students in each , 69%, 64%, and 58% of students were on task, respectively [the corresponding numbers at Amber Hill were in the 90%s].
….
More important than either of these factors, however, is that the freedom the students experienced seemed to relate directly to the relaxed and non-disciplinarian nature of the three teachers and the school as a whole. Most of the time, the teachers did not seem to notice when students stopped working unless they became very disruptive. All three teachers seemed concerned to help and support students and, consequently, spent almost all of their time helping students who wanted help, leaving the others to their own devices.

(page 64, 65)

But far from criticizing the school for abysmal classroom management, Boaler blames the students.

However, this freedom was also the reason the third group of students hated the approach. Approximately one fifth of the cohort thought that mathematics was too open, and they did not want to be left to make their own decisions about their work. They complained that they were often left on their own not knowing what to do, and they wanted more help and structure from their teachers. The students felt that the school’s approach placed too great a demand on them—they did not want to use their own ideas or structure their own work, and they said that they would have preferred to work from books. What for some students meant freedom and opportunity, for others meant insecurity and hard work. There were approximately five students in each class who disliked and resisted the open nature of their work. These students were mainly boys and were often disruptive— not only in mathematics, but across the school. (page 68)
…

In every mathematics lesson I observed at Phoenix Park, between three and six students would do little work and spend much of their time disrupting others. I now try to describe the motivation of these 20 or so students, who represented a small but interesting group. The students who did little work in class were mainly boys, and they related their lack of motivation to the openness of the mathematical approach and, more specifically, the fact that they were often left to work out what they had to do on their own. …..Many of the Phoenix Park students talked about the difficulty they experienced when they firststarted at the school working on open projects that required them to think for themselves. But most of the students gradually adapted to this demand, whereas the disruptive students continued to resist it.

In Years 9 and 10, I interviewed six of the most disruptive and badly behaved students in the year group: five boys and one girl. They explained their misbehavior during lessons in terms of the lack of structure or direction they were given and, related to this, the need for more teacher help. These students had been given the same starting points as every-body else, but for some reason seemed unwilling to think of ways to work on the activities without the teacher telling them what to do. This was a necessary requirement with the Phoenix Park approach because it was impossible for all of the students to be supported by the teacher when they needed to make decisions. The students who did not work in lessons were no less able than other students; they did not come from the same middle school and they were socioeconomically diverse. In questionnaires, the students did not respond differently from other students, even on questions designed to assess learning style preferences. The only aspect that seemed to unite the students was their behavior and the fact that most of them were boys. The reasons that some students acted in this way and others did not were obviously complex and due to a number of interrelated factors. Martin Collins [one of the Phoenix Park teachers] believed that more of the boys experienced difficulty with the approach because they were less mature and less willing to take responsibility for their own learning than the girls. The idea that the boys were badly behaved because of immaturity was also partly validated by the improvement in the boys’ behavior as they got older .

(page 73) (emphasis mine)

Meanwhile, the Amber Hill girls were miserable:

All of the Amber Hill girls interviewed in Years 9 and 10 expressed a strong preference for their coursework lessons and the individualized booklet approach, which they followed in Years 6 and 7, as against their textbook work. The girls gave clear reasons why these two approaches were more appropriate ways of learning mathematics for them; all of these reasons were linked to their desire to understand mathematics. In conversations and interviews, students expressed a concern for their lack of understanding of the mathematics they encountered in class. This was particularly acute for the girls not because they understood less than the boys, but because they appeared to be less willing to relinquish their desire for understanding…..Just as frequently, I observed girls looking lost and confused, struggling to understand their work or giving up all together. On the whole, the boys were content if they attained correct answers. The girls would also attain correct answers, but they wanted more. The different responses of the girls and boys to group work related to the opportunity it gave them to think about topics in depth and increase their understanding through discussion. This was not perceived as a great advantage to the boys probably because their aim was not to understand, but to get through work quickly. These different responses were also evident in response to the students’ preferences for working at their own pace. In chapter 6, I showed that an overwhelming desire for both girls and boys at Amber Hill was to work at their own pace. This desire united the sexes, but the reasons boys and girls gave for their preferences were generally different. The boys said they enjoyed individualized work that could be completed at their own pace because it allowed them to tear ahead and complete as many books as possible….The girls again explained their preference for working at their own pace in terms of an increased access to understanding. The girls at Amber Hill consistently demonstrated that they believed in the importance of an open, reflective style of learning, and that they did not value a competitive approach or one in which there was one teacher-determined answer. Unfortunately for them ,the approach they thought would enhance their understanding was not attainable in their mathematics classrooms except for 3 weeks of each year .

(page 139)

(all emphasis mine)

So in each school, there were students who really hated the teaching method used. But Boaler blames the complex-instruction haters at Phoenix Park (of course, it’s just a coincidence they are mostly male), for their immaturity and disruption, because they didn’t like the open-ended discovery method she so vehemently approves of. Meanwhile, she not only sympathizes with the Amber Hill girls, poor dears, who didn’t like the procedure-oriented teaching method at their school, but continually slams the Amber Hill boys who do enjoy it because those competitive, goal-driven little twerps aren’t interested in learning math but just doing more problems than their pals.

It was at this point I threw my reader across the room.

Moreover, reading between the lines of Boaler’s screed shows clearly that both schools are doing what I would consider an utterly crap job of teaching math. Boaler also mentions Phoenix Park is the low achiever in its affluent school district, and both schools have dismal test scores (which, let me be clear, could be true even if both schools were doing an outstanding job in math instruction).

Indeed, Boaler’s entire thesis—that the “reform” approach leads to better test scores—is poorly supported by her own data. Boaler received special permission to evaluate the students’ individual GCSE scores. She coded problems as either “procedural” or “conceptual”.

Amber Hill, of the dull, grey school and the dreary uniforms, actually outscored Phoenix Park, the progressive’s paradise, on procedural questions. While Phoenix Park outscores Amber Hill on conceptual problems, it wasn’t by all that much.

Like any dedicated ideologue, Boaler misses the monster lede apparent in these representations: Phoenix Park’s score range is nearly double that of Amber Hill’s, suggesting that discovery-based math helps high ability kids, while procedural math helps low ability students. Low ability students lost out at Phoenix Park, because they couldn’t cope with the open-ended, unstructured approach. Boaler didn’t give a damn about those kids, because they were boys. Meanwhile, high ability kids do better with an open-ended approach, gaining a better understanding of math concepts.

This finding has been well-documented in subsequent research—at least, the research done by academics who aren’t hacks bent on turning math education into a group project. I wrote about this earlier.

Here, too, is a takedown of some of the specifics in her research. You can read the whole thing, but here are the primary points in direct quotes:

• “Also these scores are very similar. A notable difference is that rather a lot of students at Amber Hill fail, whereas more students at Phoenix Park get the very low grades E,F,G. Boaler sees this as a positive thing about Phoenix Park. A possible explanation (which Boaler does not give) has to do with the fact that the GCSE is actually not one exam, but three exams….. it is perfectly conceivable that at Amber Hill many students aimed higher than they could achieve and failed. Note that it is essential for further education to receive at least a C, so that participating in the basic exam is virtually useless. The figures show that nonetheless at Phoenix Park at least 43.5 percent of the students (the Fs and Gs) participated in this exam and by doing this gave up their chance at higher education without even trying.”
• “This indicates that, compared to the nation, the students at Phoenix Park did worse on the GCSE than they did on the NFER. So Phoenix Park seems not to have done its students a lot of good. The same is of course true for Amber Hill, which performed very similarly to Phoenix Park. I also took a look on the internet at typical average scores of schools on the GCSE. It seems that Phoenix Park and Amber Hill are just about the schools with the worst GCSE scores in the UK. I cannot help but think that Amber Hill was specifically chosen for this fact.”
• “Boaler doesn’t say anything about the GCSE scores of Amber Hill at the moment that she decided to include this school in her study, but there is not reason to believe that it was markedly different from the above mentioned scores for Amber Hill. If that is the case, then Boaler seems to have been stacking the deck in favor of Phoenix Park and its discovery learning approach to mathematics teaching.”
• “Boaler also doesn’t mention that the grades for the GCSE at both schools are lower than one would expect given the NFER scores. She seems determined to interpret everything in favor of Phoenix Park. ”

If you’ve read anything about the Boaler/Milgram/Bishop debate, some of these Boaler critiques may sound a tad familiar. But don’t get them confused. This is a different study. Which means Boaler has pulled this nonsense twice.

It was reading horror shows like Boaler that made me loathe progressive educators. It took me a while to acknowledge that they weren’t all dishonest hacks bent on distorting reality. Not all progressives are determined to create an ideological force field that repels all sane discussion of the genuine advantages and disadvantages of different educational approaches, and an honest acknowledgement that student cognitive ability—which appears unevenly distributed by both race *and* gender, at least as we measure it—is a factor in determining the best approach for a given student population. And ultimately, I find myself slightly more sympathetic to progressives than reformers because at least progressives (and here I include Boaler) actually know about teaching, even if they often do it with blinders on.

So getting all this out of my system means I’m not writing—yet—about Boaler/Milgram/Bishop. But then, I imagine my opinion’s pretty clear, isn’t it?

Ironically, I know people who know Boaler, and assure me she’s quite nice. But then, she’s British. It’s probably the accent.