Testing

Here is another installment in my series reviewing the NY State Regents exams in mathematics.

This is question 8 from the Integrated Algebra exam.

“Four students are playing a math game at home.  One of the math game questions asked them to write an algebraic equation.“

The context of this question is utterly absurd  The question might as well have been

“Four students are taking a math test.  One of the questions asked them to write an algebraic equation.  Which student answered the test question correctly?”

Why not just ask “Which of the following is an example of an algebraic equation?”.  Maybe there are people who believe that framing questions as games, or humanizing them, will engage test-takers more, but it’s hard to believe that contrivances like this do anything but further separate students from the concepts they purport to assess.  This is another example of the negative impact these poorly-designed tests can have on students, which I discuss at length in my talk “g=4, and Other Lies the Test Told Me“.

Math teachers are familiar with the notion of pseudo-context, but I’m not sure what I would call this.  Meta-pseudo-context?  Pseudo-meta-context?  Pseudo-pseudo-context?  Ridiculous, at the very least.

Related Posts

• Regents Recaps
• g = 4, and Other Lies the Test Told Me
• Regents Recap — January 2016: No It Wasn’t

Here is another installment in my series reviewing the NY State Regents exams in mathematics.

June, 2014 saw the administration of the first official Common Core Regents exam in New York state,  Algebra I (Common Core).  Roughly speaking, this exam replaces the Integrated Algebra Regents exam, which is the first of the three high school level math Regents exams in New York.

One of the biggest differences in the two exams is how raw scores were converted to scaled scores.  In the graph below, Integrated Algebra is represent in blue, Algebra I (Common Core) in orange.  Raw scores are on the horizontal axis, and scaled scores are on the vertical axis.

The raw passing score (a scaled score of 65) is roughly the same for both exams:  30/87 for IA, and 31/86 for CC.  But notice the divergence in the plots after a raw score of 30.  This is because the raw “Mastery Score” (a scaled score of 85) is quite different for the two exams:  65/87 on IA, vs 75/86 on CC.

It’s curious that the exams could be evaluated in such a way that passing requires the same raw score on both, but mastery requires a much higher score on one than the other.  If the exams were of equal difficulty, this would mean the same percentage of students would pass the exam, while dramatically fewer students would achieve mastery on the Common Core exam.

This is especially curious, since the tests don’t really seem that different to me apart from some substantial changes in content emphasis.  It’s hard not to see this as merely a deliberate decision to lower the mastery rate.

Furthermore, the consensus is that raw scores on the Common Core exam are substantially lower than on the Integrated Algebra exam.  Based on the exams and on conversations I’ve had, I wouldn’t be surprised if scores on the Common Core exam were lower than scores on the Integrated Algebra exam by 10 raw points on average.  This would lead to an even larger drop in mastery rates, as well as a drop in passing rates.

There’s an interesting opportunity here, though.  I’m certain that a large number of students across New York state took both the Algebra I (Common Core) exam and the Integrated Algebra exam.  This offers a unique opportunity to directly compare the tests and the conversion charts the state decided upon.

I think the Department of Education should release that data so we can see for ourselves just how different student performance was on these two exams, and judge for ourselves the consequences of these very different conversion charts.

Related Posts

• Regents Recaps
• Regents Recap — June 2016: Scale Maintenance
• Regents Recap — June 2015: Cubics, Conversions, and Common Core

Here is another installment in my series reviewing the NY State Regents exams in mathematics.

June, 2014 saw the administration of the first official Common Core Regents exam in New York state,  Algebra I (Common Core).  Roughly speaking, this exam replaces the Integrated Algebra Regents exam, which is the first of the three high school level math Regents exams in New York.

The rhetoric surrounding the Common Core initiative often includes phrases like “deeper understanding”, and the standards themselves speak directly to students communicating about mathematics.  These are noble goals.

So, when it comes to the Common Core exams, it’s not surprising that we see directives like “Explain how you arrived at your answer” and “Explain your answer based on the graph drawn” more often.

But including such phrases on exams won’t accomplish much if the way the student answers are assessed doesn’t change.  Here’s number 28 from the Algebra I (Common Core) exam, together with its scoring rubric.

Notice that the scoring rubric gives no indication as to what constitutes a “correct explanation”.  When scoring these exams, groups of readers are typically given a few samples of student work and discuss what a “correct explanation” looks like.  But people grading these exams often have drastically different ideas about what constitutes justification and explanation.  Given the importance Common Core seems to attach to explanation, I’m surprised that the scoring rubric takes no official position here.  In fact, this rubric is essentially identical to those used for the pre-Common Core Integrated Algebra exam.

There’s a real danger in simply tacking on generic “Explain … / Describe …” directives to exam items.  Consider number 34 from the Algebra I (Common Core) exam.

It’s not really clear to me what a valid response to the directive “Describe how your equation models the situation” would look like.  Nor an invalid response, for that matter.  So what do students make of such of a directive?  I suspect that, for many, it just becomes another part of the meaningless background noise of standardized testing, another place where they simply have to guess what the test-makers want to hear.  And according to the rubric, the test-graders will have to guess, too.

Yes, students should be communicating about mathematics, their processes, and their ideas.  But just adding “Explain how you got your answer” to a test question isn’t going to do much to help achieve that goal.

Related Posts

• Regents Recaps
• Regents Recap — January 2015: It’s True Because It’s True
• Regents Recap — August 2015: Modeling Data
• Regents Recap — June 2016: What Do They Want to Hear?

Here is another installment in my series reviewing the NY State Regents exams in mathematics.

June, 2014 saw the administration of the first official Common Core Regents exam in New York state,  Algebra I (Common Core).  Roughly speaking, this exam replaces the Integrated Algebra Regents exam, which is the first of the three high school level math Regents exams in New York.

The Algebra I (Common Core) exam was structured differently than the Integrated Algebra exam.  Algebra I (Common Core) contained 24 multiple choice questions, while Integrated Algebra contained 30 multiple choice questions.  This change, in and of itself, likely translates to lower average scores on the Algebra I (Common Core) exam:  merely guessing on those 6 extra multiple choice questions would yield an extra 3 points on average, and multiple choice questions are generally easier to score points on than free response.

The free response sections are structured slightly differently, as well, but not substantially so.  In fact, apart from some significant content differences, not much distinguishes the new Common Core exam from the old Integrated Algebra exam.

Many of the questions on the Algebra I (Common Core) exam are similar in style and content to questions on other math Regents exams.  For example, here is number 1 from the Common Core exam compared with number 5 from the Integrated Algebra exam.

And here is number 3 from the Common Core exam compared with number 14 from the Integrated Algebra exam.

A few of the questions on the Algebra I (Common Core) exam do show improvement over similar items on other exams.  Consider number 8 from the Common Core exam, compared with number 8 from the Integrated Algebra exam.

The two questions address the same issue, but the top is more mathematically precise.  (I really dislike the “Which step could be used?”-type problems).

Overall, in terms of the way questions and posed and structured, there is not much of a difference between the new and old exams.  The Algebra I (Common Core) exam is pretty much standard test fare.  When the current testing culture is criticized, a common response is that we just need to make better tests.  This is easy enough to say, but surprisingly hard to do.  And it doesn’t look like this first Common Core Regents exam is substantially different, or better, than its predecessor.

Here is another installment in my series reviewing the NY State Regents exams in mathematics.

June, 2014 saw the administration of the first official Common Core Regents exam in New York state,  Algebra I (Common Core).  Roughly speaking, this exam replaces the Integrated Algebra Regents exam, which is the first of the three high school level math Regents exams in New York.

So how does the new Algebra I (Common Core) exam compare with the old Integrated Algebra exam?  In terms of mathematical content, one difference jumps out immediately:  functions.

On the 2014 Algebra I (Common Core) exam, the word “function” appears 17 times.  On the 2014 Integrated Algebra exam, the word “function” appears once.

An informal reading indicates that functions play a part in 14 of the Algebra I (Common Core) exam questions, which are worth a total of 36 points.   This is around 42% of the exam (36/86 points).  In addition to several direct, function-related questions, like about domains and modeling, we see a number of familiar questions recast in the language of functions.

For example, number 12 from the Algebra I (Common Core) exam asks about “zeros of the function”.  On an Integrated Algebra exam, the same type of question would likely have been asked in the context of “solving an equation”.

I’m not going to dispute the importance of this concept:  functions are central objects in mathematics, and in high school math in particular.  But this is clearly a substantial change in focus for this exam, and I doubt many could have predicted the extent of this change.  Functions should be covered in any algebra course, but this dramatic shift in assessment undoubtedly penalized many students.  A student with a weak grasp on this one concept would face a huge obstacle on the Algebra I (Common Core) exam.

In addition to this major shift in emphasis, several topics which were previously covered in the highest-level exam (Algebra 2 / Trig)  are now on the Algebra I (Common Core) exam:  recursively defined functions; correlation coefficient; and the quadratic formula.  This does not come as a surprise, but it’s worth noting that these topics have been moved ahead by 2-3 years in the typical student sequence.  Also of note was that students were asked to factor a fourth-degree polynomial (of quadratic type) on the Algebra I (Common Core) exam, which as far as I know has only previously appeared on the Algebra 2 / Trig exam.

Will the next iteration of Algebra I (Common Core) emphasize functions so heavily?  What will the first iteration of Geometry (Common Core) exams look like?  Our initial experience with Common Core Regents exams in New York suggest it will be hard to predict.