Rocketship Academy wants to go national. Rocket Academy is a hybrid charter school chain that focuses solely on getting low income Hispanic elementary school students to proficiency. (Note: Larry Cuban has some excellent observations from his visit to a Rocketship Academy.)
First things first: I’ve checked the numbers every way I can think of, and Rocketship’s numbers are solid. They don’t have huge attrition problems that I can see. They are, in fact, getting 60% or higher proficiency in most test categories, and the bulk of their students are Hispanic, many of them not proficient in English. Of course, that brings up an interesting question–if they are proficient on the ELA tests, why aren’t they considered proficient in English? But I digress.
The larger point is this: getting high test scores on California’s elementary school math tests ain’t all that much to get worked up about. Here’s some data from the 2011 California Standards test in math:
I used two standards, because the NCLB obsession with “Proficient and higher” is, to me, moronic. I prefer Basic or higher. The blue line is the percentage of all California in grades 2-9 scoring Basic or higher in General Math, the red is the percentage of same scoring Proficient or higher.
So it gets a bit tricky here, because after 6th grade, the entry to algebra varies. In order to simplify it slightly, I’m ignoring the seventh grade algebra track (call it “accelerated advanced” path), which is about 40,000 students this year, fewer in previous years.
I combined 8th and 9th grade students in General Math and attached that result to the red and blue lines.
Then I separated two groups–the ones who took algebra in 8th grade, and the ones who took algebra later than that. The first group are those who entered algebra in 8th grade, passed it, and continued on the “average advanced” course path, culminating with Calculus senior year. The second group are those who took algebra for the first, second, or third time in high school and then continued on. For each group, I calculated percentages for Basic + and Proficient +.
Notes:
- Through grade 6, the scores represent all students. In grade 7 and 8/9, general math scores reflect only those students who haven’t moved onto algebra. That’s probably why the proficiency levels drop to 50% and lower for the last two groups.In other words, the green and purple lines represent the advanced track students–most, but not all of the the strong algebra and higher math students. The turquoise and orange lines represent the weaker students taking algebra and higher.
- Roughly 80% of all students test at Basic or higher from second through sixth grade.
- Over 70% of the strong studentws test at Basic or higher from algebra through “summative math” (taken for all subjects after Algebra 2).
- The percentage of students testing proficient from second through sixth grade starts at 65%, rises slightly, and then drops steadily.
- In no course do more than 50% of the strong students in algebra and higher achieve a score of Proficient or higher.
- In no course do more than 50% of the weaker students in algebra or higher achieve a score of Basic or higher.
So the chart reveals that all California second through sixth graders, high and low ability, averaged higher scores on their tested subject than the strongest high school students did.
I used 2011 scores, and I may have made a minor error here or there, but the fall off has been in the scores for several years now, and it’s easy enough to check.
What could cause this? Why are California’s elementary school students doing so phenomenally well, and then fall apart when they get to high school? Let’s go through the usual culprits.
California’s high school math teachers suck.–Well, in that case, there’s not much point in demanding higher standards for math teachers, because California’s high school math teachers have had to pass a rigorous content knowledge test for over 20 years. California’s elementary school teachers have to pass a much easier test–which is much harder than anything they had to pass before 2001. In other words, try again.
The teachers aren’t covering the fundamentals! So when the students get to algebra, they aren’t prepared.–But hang on. Elementary school kids, the ones being taught the fundamentals, are getting good test scores. What evidence do you have that they aren’t being taught properly?
Well, they’re only getting good test scores because the tests are too easy!—dingdingding! This is a distinct possibility. Perhaps the elementary tests aren’t challenging enough. Having looked at the tests, I’m a big believer in this one. I think California’s elementary math tests, through seventh grade, are far less challenging to the tested elementary school population than are the general math and specific subject tests are to the older kids. (On the other hand, the NAEP scores show this same dropoff.)
However, while that might explain the disparity between the slower track math student achievement and elementary school, it doesn’t adequately address why the students in the “average advanced” track aren’t achieving more than 50% proficiency, does it?
Trigonometry is harder than memorizing math facts–We should take to heart the Wise Words of Barbie. Math achievement will fall off as the courses get more challenging. Students who excelled at their times tables and easily grasped fractions might still struggle with complex numbers or combinatorics.
So if you ask me—and no one does. Hell, no one has even really noticed the fall-off—it’s a combination of test design and subject difficulty.
Whatever the reason, the test score falloff has enormous implications for those who are banking on Rocketship Academy, KIPP, and all those other “proven” charters that focus exclusively on elementary school children.
Elementary school test scores are false gods. We have no evidence that kids who had to work longer school days simply to achieve proficiency in fifth grade reading and math will be, er, “shovel ready” for algebra and Hamlet. KIPP’s College Completion Report made no mention of its college students SAT scores, or indeed made any mention of demonstrated ability (e.g., AP tests), and color me a cynic, but I’m thinking they’d have mentioned both if the numbers were anything other than dismal.
So let’s assume that those Rocketship scores are solid (and I do). So what? How will they do in high school? Where’s the follow through? Everyone is banking on the belief that we can “catch them early”. Get kids competent and engaged while they are young, and it all falls into place.
Fine. Just let me know when the test scores back up that lovely vision.
Added in January 2014: Well, hey now. Growing Pains for Rocketship’s Blended-Learning Juggernaut.
Alas, it seems that Rocketship’s scores are declining, their model doesn’t scale, they are making decisions based on cost rather than learning outcomes and, my FAVORITE part:
Lynn Liao, Rocketship’s chief programs officer, said the organization has also received troubling feedback on how students educated under the original blended learning model fare in middle school.
“Anecdotal reports were coming in that our students were strongly proficient, knew the basics, and they were good rule-followers,” Ms. Liao said. “But getting more independence and discretion over time, they struggled with that a lot more.”
That graven image gets you every time, doesn’t it?
July 30th, 2012 at 6:40 pm
In the sentence
“In other words, all California second through sixth graders, high and low ability, demonstrated higher achievement on their tested subject the strongest students (those in the average advanced track) did, even with the average or lower ability students sorted out.”
I think there should be a “than” before “the strongest students”.
Thanks for the post.
July 31st, 2012 at 2:47 am
Thanks for the catch!
August 1st, 2012 at 7:02 pm
I was actually very good at maths in primary school. Then I hit secondary school and BAM! They put me in the borderline special needs class as soon as they could. I always felt like it was somehow my “fault”, because goddammit I was good at it before.
I don’t have any deep comments or questions. I just like this post because it tells me that my experience was well within the normal spectrum.
August 19th, 2012 at 12:07 am
[…] Instead, I’m going to revisit a chart I updated from the last time I posted it: […]
September 9th, 2012 at 8:37 pm
[…] written before, improving elementary school or middle school scores is a false god when it comes to improving actual high school outcomes. Children who need tons of […]
September 19th, 2012 at 5:33 am
[…] Here’s the really funny thing: I’ve described what I do, and what many high school teachers do—develop our own curriculum to cover the standards. But it’s clear from even a cursory overview of the education debate, that a million teachers planning their own curriculum is not what eduformers foresee as the future of education in this country. It’s also clear, however, that most education reforms never make it to the classroom and don’t have a clue how teaching actually happens, particularly in high school. (Eduformers believe that once elementary school is fixed, all will be well. They’re wrong.) […]
January 2nd, 2013 at 2:37 am
[…] The False God of Elementary Test Scores–another idea I return to frequently. Many believe that raising elementary test scores and achievement will lead to stronger high school achievement. No evidence of that, folks. […]
January 31st, 2013 at 11:25 am
[…] Reformers faithfully hold out this hope: bad teachers are creating lazy students who could do the work but just don’t want to. Oh, yeah, and if we catch them in elementary school, they’ll be fine in high school. […]
April 5th, 2013 at 10:07 pm
[…] will disappear! Look at KIPP’s results! Look at Rocketship Academy!” and I warn him to beware the false god of elementary school test scores. If the achievement gap is a function of IQ distribution, then effective education methods will not […]
May 5th, 2013 at 1:43 am
[…] course the classes aren’t rigorous enough. They can’t be. I refer you again to the false god of elementary school test scores and the Wise Words of […]
April 23rd, 2014 at 1:20 am
[…] get high placement than blacks. But generally, her data shows something I’ve documented before, that achievement falls off each year because school gets […]
February 10th, 2016 at 6:29 pm
[…] will work better for elementary schools than middle or high schools, perhaps because the gaps are smaller […]
May 20th, 2016 at 7:35 pm
[…] alas, they weren’t. I believe that the country is nearing the end of its faith in the false god of elementary school test scores, the belief that the achievement gap in high school is caused simply by not sufficiently […]
May 23rd, 2016 at 1:01 pm
The problem is the Yale or jail assumption. Let the kids go into votech or trade school tracks as they choose. Better yet would be the German system of gymnasium versus apprenticeship. But you really don’t need to get that complicated.
My 80s high school had 3 tracks (and you could select whatever you wanted, was free choice, but it was available!) And you weren’t restricted to a track. You could take one honors and the rest basic (say math or bio or what interested you). You could take a shop class even if you were super advanced. STILL wish I had taken mechanical drawing (drafting)…was really funny how I told my guidance counselor I would never need it…and then ended up in an HVAC/plumbing designer job a few years later!
A. Votech (home ed, auto shop (some general shop too): courses like cookbook chemistry. Gave the kids a path to a degree. Yeah, you have some drop, but you gotta figure this is a better choice than getting killed in hard classes, for those who struggle. And it avoids slowing down or disrupting the other classes. And the purely applied classes actually have some cool aspects (well shop does, does home ec even exist any more).
B. Regular: standard high school classes (Algebra 2, not with trig; regular chemistry, regular English, “western cultures”, etc.)
C. Honors: accelerated chem, “world civ”, etc.
C.5. AP classes: true approximations of college courses, taken almost entirely by advanced/strong seniors.
August 24th, 2016 at 4:52 pm
[…] that charters actually address, while bragging about improving test scores, which they don’t (in any meaningful […]
October 16th, 2016 at 11:21 pm
[…] I’ve written before, in math as it is in English, elementary school “proficiency” is much easier to acquire […]
November 7th, 2016 at 6:16 pm
[…] it, on the 8th grade standardized tests at any rate. Education Realist might call this the “false god of elementary test scores,” citing the persistence of gaps on the SAT and other high-school level tests, and while I do […]
December 10th, 2016 at 7:39 pm
[…] “No, but of course higher test scores in elementary scores would lead to better high school performance.” […]
December 13th, 2016 at 9:52 pm
[…] tests anyways, so the costs of data collection have gone way down. Education Realist has often wondered aloud why Elementary and Middle School test scores are so often the buttress of evidence for charter […]
October 5th, 2020 at 6:59 am
[…] more blacks and Hispanics do well in younger grades. I first wrote about this in my seminal article The false god of elementary math scores–well, seminal in the Ed Realist oeuvre, at […]
January 3rd, 2022 at 8:09 pm
[…] The false god of elementary school test scores–July 30, 2012 Another one I almost didn’t include because it definitely has the rabbit hold problem about Rocket Ship at the beginning. However, like Pseudofacts, it’s an early example of my actually looking at readily available information and pointing out the obvious. Plus, great title. ignore me, lazy way to space […]