Braindumping the PSAT: A Few Questions for David Coleman

The day after the first PSAT sitting, two parents (at least, I think they were parents), posted this this exchange
on College Confidential (click to enlarge):

psatbdavailcopy

The day of the PSAT carried this slightly more obscure exchange:

psat2015earlycopy

Suzyq7’s comment seems to come out of nowhere, because almost certainly the College Confidential moderators purged a post or two. It appears that FutureMMAChamp or some other poster explained that some testers knew exactly what they’d gotten wrong because they had an early copy of the test, which led to Suzyq7’s outburst. That post got purged, so it’s hard to make sense of the conversation, but the gravamen of the charge comes through.

Notice the lack of “what on earth are you talking about?” responses. These posters aren’t being challenged for their grasp on reality.

So on October 4th, someone posted actual PSAT content. No one knew for sure it was PSAT content, I assume, which is why the content remained on the site until October 15th. At that point, it appears, the moderators became aware of the posts and purged them. Or maybe the posts are still online, although lord knows I’m a determined searcher and I can find no record of them. The moderators also deleted or modified posts referencing the content whenever possible.

From October 3rd until some point after October 15th, actual PSAT test questions were readily available on a forum that sees about 2 million unique viewers a month. Then that same forum, with the help of Google, Reddit, Twitter, Tumbler, and other social media sites, provided October 28th testers with a roadmap of all the questions on the test.

You really have to chuckle, don’t you?

All those reporters writing indulgently about the PSAT testers violating their promise to refrain from discussing test questions. Most or all of the tests passage texts have been revealed: Frederick Douglass 4th of July speech, the Jason Goldman’s article on researchers establishing differences between dogs and wolves (images included), and Julia Alvarez’s In the Name of Salome (the one about Herminia and her papa). Brian Switek’s piece on Nasutoceratops, the large nosed horn faced dinosaur or a similar piece was used in the writing section.

Peter Greene of Curmudgucation went so far as to include tweets with images from the test, and thinks it’s a big joke because the PSAT is “a test which everyone takes essentially on the same day” (it’s not. More on that in a minute). But then, Peter tells his students that the “P” stands for “Practice”, so hey. (It’s “Preliminary”, but then, the SAT doesn’t stand for anything any more, so maybe Peter thought he could just invent a P.)

I can only assume that the journalists, idealistic humanity majors sorts, found the tumblr posts creative. They might not have been as thrilled by the more, er, explicit discussions that were taking place out in the open Internet.

For example, the google doc in which participants discuss as many of the questions as the participants can remember. You can check out the original, but my pdf takes less time to load and omits the first page of obscenities. (The first link the author made was apparently deleted by Google as a violation of the terms of service.)

The google doc participants discuss the math and reading questions, with the occasional English query as well. They go into considerable detail; the enterprising student can become considerably aware of the pitfalls even without authoritative answers. This isn’t a particularly impressive brain dump file compared to the SAT recreations I’ve written about. But of course, the reddit thread where I found the google doc contained links to the several of the reading passages directly, and another thread openly discussed the specifics of a math problem. An employee or founder of Wave Tutoring was cheerfully in the same thread as the google doc link, giving advice and offering his services.

And there’s still good ol’ College Confidential, which has been the venue for organized SAT braindumping for years. The moderation appears a bit more vigilant this year, although the goal seems to be hiding evidence that cheating occurs rather than, you know, actually ending cheating.

So this page is blank because the moderator purged it, but the cache shows detailed question recollection. (Image capture in case cache disappears.) When a poster expresses surprise that there isn’t more specific chatter, he or she is told that just minutes earlier three pages of comments had been wiped out, presumably because students had been actively and specifically discussing the test.

I wonder if the reporters would have written cheerful stories about the google docs, the reading questions and topics, and the carefully worked math problems. Well. Not really. What I really wonder about is why the reporters can’t be bothered to write about the google docs, the reading topics, and the carefully worked math problems.

In the meantime, you can see why it’s all worth a chuckle. All this effort: the google files, the Tumbler memes, the careful hints, the chuckles, the sly media approval, and all this time the entire test was available online—and, undoubtedly, in hard copy for the right price.

Another College Confidential thread on SAT cheating via the forum inquires of the moderators why they allow blatant discussion of SAT questions without banning and posting a list of the offenders. A moderator responded that they have created a friendly place and that “public shaming” is not productive.

The discussion continues on to debate whether the site should be shut down so that the 10/28 PSAT isn’t compromised by all the “specific questions” being discussed.

What, you didn’t know that the PSAT isn’t over for the year? October 28, this Wednesday. Many high schools districts across the country take the PSAT on the “alternate date”: North Colonie Central Schools, Greenville High School, West High School, Dos Pueblos High School, Ridgefield High School, and the entire Seattle Public School District.

Is it the same test? Probably. I’m pretty sure the College Board used the same test for the “alternate” test date (note the wording) in the past. This year, with a new format, an entirely different test is probably impossible.

Wouldn’t it be cool if someone asked the College Board about it? Maybe a reporter, even.

Hey, David Coleman! Has your company discussed the publication of actual test material at College Confidential? The ACT constantly monitors the College Confidential boards for mention of their tests, but your company doesn’t. That’s why people routinely post passages and answers to SAT tests for the Chinese and Koreans too poor to pay for the actual tests from organized Chinese crime rings, while the ACT has almost no international market. But you don’t care about market advantage, right? You’re “non-profit”. In the future, are you planning on using previously issued tests for the international market, so the Chinese and Koreans can buy copies of the tests and pretend they are capable of 800 SAT reading scores when in fact they can’t even read English?

And speaking of the the Chinese SAT cheating ring, are your employees selling the tests? Maybe they’ve decided to develop a sideline in PSAT tests for Americans? Or perhaps the source is just a corrupt principal who sold a few copies to a test prep company and well, kids talk. But given the huge dollars schools pay for your product, have you considered delivering and proctoring the tests with College Board employees?

Do you have a different test planned for 10/28? If so, how will you ensure that the two different test dates are equally reliable?

If not, do you think it’s fair that all 10/28 PSAT testers, as well as 10/14 testers who had an actual copy of the test are better prepared to compete for the National Merit Scholarship Program, winning recognition and scholarships?

And please, Dave, don’t try and fob off questions with “Only a few schools take the test on the alternate date” or “The College Board spends millions on test security but we can’t be responsible for corrupt high school principals” or “We rely on our students’ honor and integrity and while there are sadly a few bad apples, the majority of our testers act responsibly”.

This is your product! You sell it to schools in exchange for a metric ton of money and student information–which you then turn around and sell to colleges, along with….oh, yeah, the TEST SCORES from a test that was available online for two weeks before the first sitting.

You’re busy breaking your arm patting yourself on the back for paying Khan Academy to provide low cost test prep to disadvantaged blacks and Hispanics—because you’re basically ignorant of the fact that blacks and Hispanics are more likely to get test prep than whites (all races are pikers compared to East Asians). Given your products’ abysmal integrity, why shouldn’t blacks and Hispanics abandon test prep and get in on the advance knowledge action? Right now, it’s probably (but not certainly) restricted to Asians, but that will change if you continue to shrug off the blatant test corruption that happens every month, every year–corruption that the ACT does not have in anything approaching the same level.

And while I’m on the topic, hey, Tim McGuire, president of the National Merit Scholarship Corporation! Have you ensured that PSAT is issued fairly and consistently, giving all testers an even shot at the scholarships you offer? Or are you so worried about constantly losing colleges that you ignore the fact that the PSAT is becoming as corrupt as its parent? Have you considered perhaps using an ACT product for scholarships?

Tish tosh, you say. No one really cares about the PSAT. It’s just a “practice” preliminary test. The scores don’t matter. They aren’t used for college admissions. So what’s the difference?

Yeah, you’re right. I shouldn’t point out that the kids most likely to use the advance copies are the kids who have a shot at National Merit scholarships. I shouldn’t remind everyone that the braindumping for the SAT, another College Board product, is exponentially worse than the violations I’ve discussed here. I shouldn’t worry that we’re becoming as corrupt as China. I shouldn’t worry that taxpayers pay millions to the College Board, a non-profit company, to deliver a testing product whose validity and reliability can’t be assured. I shouldn’t care that reporters don’t care enough to worry the College Board enough to bother scaring College Confidential, that reporters, like the colleges dropping the National Merit Program, only care about the average performance by SES and race.

What I really worry about, frankly, is all the organized braindumpers thinking jesus, that Ed. What a dolt. Only losers use College Confidential. You can download advance copies of all College Board products at darknet/yangchan for a small fee.

Advertisements

About educationrealist


19 responses to “Braindumping the PSAT: A Few Questions for David Coleman

  • viijay

    Which high school sophomore or Junior goes looking for PSAT questions in college confidential forum? Do they even know if such a thing exists? Is there any use for PSAT?

    Even getting good scores in PSAT does not mean you will get national merit scholarship, the selectors look ffor things like leadershoip, etc. (Both my kids got good PSAT scores but were not selected for NMS)

    • Powerlurker

      Over 90% of National Merit Semifinalists (a classification entirely dependent on PSAT scores) end up becoming Finalists. The biggest reason for not making it is SAT scores that don’t tally with PSAT scores, the second biggest is poor grades.

      Getting a scholarship after becoming a finalist involves a whole set of other criteria largely unrelated to test performance (such as meeting the qualification criteria set by a scholarship sponsor).

      • viijay

        Basically correct. The scholarship sponsor asks for leadership roles, which is somewhat harder to define.

        The only point I had is that PSAt scores are not terribly important enough for idiots to post test questions in college confidential before the exam.

  • viijay

    With your permission, I would like to change track, and want to ask your opinion on the TNVPK results, and its import on the inability of children to retain gains made in 4th grade and 8th grade test scores. There has been attempts to attribute the lack of gain in 12th grade test scores to cognitive ability only, but I think there is something more. No amount of learning more earlier will make you a better learner later in life. The TNVPK and comparisons of 4, 8, and 12 grade scores in time seems to state that learning too much earlier in life is not helpful.

    Finally, all of these have an impact on overloading of AP tests in high schools. I see increasing evidence that kids who overdose on AP do not go on to grad school-> research->higher income.

    • Powerlurker

      Why would a kid who has better options want to go to grad school (assuming that you use “grad school” to refer to a masters or Ph.D granting program as opposed to a professional program like medicine, business, or law)? A Ph.D overspecializes you and delays your career for 5-7 years before putting you in a position where you will make good, but unspectacular money. The kid who gets 5s on all his APs and goes to a good school is more likely to go to med school or do the finance to MBA route.

      • Vijay

        The point was that a large increase in students taking the AP exams and number of exams taken (between 7 and 10 times) from, say, 200 to 2014, has not been correlated to a large increase in finance MBAs or MDs or Ph.Ds. There has no particularly large increase even in STEM degrees. I was just trying to correlate that to increases in 4th grade and 8th grade test scores not correlating to 12th grade score increases.

        Sounds stupid, after rewriting the whole thing. However, it does appear that higher scores in lower grades, and more advanced courses does not seem to lead to anything.

    • Roger Sweeny

      viijay, what TNVPK results are you referring to? The story that came out recently compared kids who won the lottery to get into Tennessee Voluntary Prekindergarten with kids who lost. The kids who won did better in kindergarten and grade 1 but slightly worse in grades 2 and 3.

      http://www.brookings.edu/research/papers/2015/10/08-expectations-of-sustained-effects-from-scaled-up-prek-challenges-tennessee-study-farran-lipsey#.Vij_POcE4fQ.twitter

      I can’t resist passing on two comments from a post on this at marginal revolution:

      “Lots of program are designed this way:

      1. See what the smart kids do.

      2. Make everyone else do that.

      If you are blank-slater, this is really appealing, because we could make everyone smart. However, what it really does is push everyone through a grind that most people — except for the insufferable nerds, like me — utterly despise.

      It’s really cruel to keep on trying to force people who will never be engineers to do calculus, but for some reason we keep at it, for their own good.”

      and

      “School is awful. Kids hate school. Starting kids in school earlier brings that out sooner. I hope this doesn’t surprise anyone who actually remembers what being in school was like.”

      Or to put things in bloodless economese, for many young people, school knowledge gained is not a monotonically increasing function of time spent in school.

      • Vijay

        In “Philip Dick,Preschool and Schrödinger’s Cat”. EDRealist referred to Perry study as an example where preschool did not support the claimed results. There is now a more detailed TNVPK study, while not sufficiently long term, provides more support of lack of stickiness of early interventions. I am trying, without proof, to extrapolate that to the lack of stickiness of fourth grade and eighth grade results to high school, which probably is the ultimate goal of the US system.

        If the knowledge gained is not linear, and if we can increase eight grade test scores, then the situation screams for the Hauptschule/Berufsschule; Realschule; Gymnasium/Abi diploma. Some argue that we have something like this already in the US with close to 40% going to a Hauptschule bereft of Beruufsschule!

        Some of the school boredom/girl gets beaten up in HS may b an outcome of everyone having to go to a classroom in which they have no interest.

      • Roger Sweeny

        vijay, if you are “trying, without proof, to extrapolate,” it’s a good idea to say that at the beginning. Otherwise, people will get the wrong idea.

        Some of the school boredom/girl gets beaten up in HS may b an outcome of everyone having to go to a classroom in which they have no interest.

        I think everyone in education kind of knows this. But the people in authority (who did well in school) believe, with a faith that will not yield to doubt, that if only things were taught right, students would realize how important and interesting their schoolwork is. They would not be bored. They would learn.

        The lack of empathy in people who think they are wildly empathetic (or at least sympathetic) is stupefying.

  • marie

    This is making me furious. My son took the PSAT on the first day and discussed a complicated math problem with me afterward. He worked through it and decided that he got the right answer but said he didn’t think very many other kids would get it right. I asked him if he was tempted to show the question and answer to a friend of his who was taking the test at the later date (in the same city!). He was mortified at the mere suggestion and said he’d signed something on the test promising not to do that.

    I was proud of him. Maybe he’s a chump.

    • marie

      okay I need to update my own post. When my son and his friend discussed the PSAT after both had taken it, turns out they didn’t get the same questions at all. Phew! They kept bringing up questions they faced and there was only head shaking.

      Seems these tests really want kids to be steeped in environmentalism and thinking about evolution all the time. I wonder what that means for the future. Probably not what they think.

  • purpletigerbot

    I wasn’t sure where to comment, but I’d figure this article would be suitable:

    ETS releases GRE scores by country(http://testprep.about.com/od/Revised_GRE_Exam/fl/2014-GRE-Scores-by-Country-of-Citizenship.htm) and of course China is a ridiculous outlier (the scores on the new range from 130-170 with 150ish being the average and 170 being the 99th percentile on both sections).

    Country # Verbal(percentile) Quant(percentile)
    China 42,098 147.5V(33) 164.2Q(88)
    Hong Kong 696 149.5V(41) 160.1Q(78)
    Korea (ROK) 5,260 149.3V(41) 160.0Q(78)
    Singapore 916 157.7V(74) 160.3Q(78)
    Taiwan 3,072 146.1V(29) 161.2Q(80)
    Japan 1,228 145.3V(25) 157.3Q(68)

    How can Chinese students score half a deviation above other more wealthy East Asian countries in quant when they all score basically the same on every other quant diagnostic measure? And how can they even parse difficult word problems required for high scores with such low English ability? Just grinding and plug and play? Or question dumping and memorization? Is there massive cheating and ETS just doesn’t care?

    • educationrealist

      THanks! I hadn’t seen this.

    • purpletigerbot

      More fun stuff in an interview with Robert A. Schaeffer, “public-education director of the National Center for Fair and Open Testing”:

      Q. In an age of high-tech gadgetry and instant communication, it’s hard to keep information about tests from flying all over the world. What’s it mean when students can go right home and describe the test they just took on College Confidential, as many students do?

      A. They’re not even waiting until they get home. They go on a bathroom break, they post on Facebook and Twitter, describing the items that are on a test. People will often put up links to a Google Doc where kids compile entire tests right after they’ve been administered. “Oh, I remember Question 3, it was this,” and then someone will say, “No, it was more like this.” They’ll rebuild an entire test, or nearly an entire test, with individual contributions. The notion of secrecy in the 21st century, where you can communicate an image or words around the globe in a blink of the eye, it just doesn’t exist.

      The full article and interview can be found here:
      Theories Abound to Explain College Board’s Abrupt Ban on Certain SAT Takers
      http://chronicle.com/article/Theories-Abound-to-Explain/235557

      • educationrealist

        I’ve been writing about the braindumping, too. See my PSAT piece.

      • purpletigerbot

        > I’ve been writing about the braindumping, too. See my PSAT piece.
        Oh, I’ve read the article 😉 . It was just nice to see it acknowledged in the Chronicle of Higher Ed and I figured I’d share.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: