Testing

As part of my review of the 2012 August New York State Math Regents exams, I came upon this question, which rivals some of the worst I have seen on these tests.  This is #11 from the Geometry exam.

This question purports to be about knowing when we can conclude that two intersecting planes are perpendicular.  Sadly, the writers, editors, and publishers responsible for this question clearly do not understand the mathematics of this situation.

Each of the answer choices is a statement about two lines in the given planes being perpendicular.  The problem suggests that three of these statements provide sufficient information to conclude that the given planes are perpendicular.  The student’s task is thus to identify which one of the four statements does not provide sufficient information to draw that conclusion.

There is a serious and substantial flaw in the reasoning that underlies this question:

Knowing that two lines are perpendicular could never be sufficient information to conclude that two containing planes are perpendicular.

In fact, given any two intersecting planes, you can always find two perpendicular lines contained therein, regardless of the nature of their intersection.  A simple demonstration of this fact can be seen here.  Thus, knowing that the planes contain a pair of perpendicular lines tells you nothing at all about how the planes intersect.

It’s not that this particular question has no correct answer; it’s that the suggestion that this question could have an answer demonstrates a total lack of understanding of the relevant mathematics.

How much does this matter?  It’s a two-point question, it’s flawed, so we throw it out.  No harm done, right?

Well, imagine a student taking this exam, whose grade for the entire year, or perhaps even their graduation, depends on the outcome of this test.  Imagine the student encountering this problem, a problem that not only has no correct answer, but whose very statement is at odds with what is mathematically true.  It’s not out of the realm of possibility that in struggling to understanding this erroneously-conceived question, a student might get rattled and lose confidence.  Test anxiety is a well-known phenomenon.  The effect of this problem may well extend past the two points.

Teachers are also affected.  A teacher’s job may depend upon how students perform on these exams, but there isn’t any discussion about their validity.  A completely erroneous question makes it through the writing, editing, and publishing process, and has an unknown affect on overall performance.  After thousands of students have taken the exam, a quiet “correction” is issued, and the problem is erased from all public versions of the test.

What’s most troubling about this, to me, is that this is not an isolated incident.  Year after year, problems like this appear on these exams.  And when confronted with criticism, politicians, executives, and administrators dismiss these errors as “typos”, or “disagreements in notation.”

These aren’t typos.  These aren’t disagreements about notation.  These are mathematically flawed questions that appear on exams whose express purpose is to assess the mathematical knowledge of students and, indirectly, the ability of teachers to teach that knowledge.  If the writers of these exams regularly demonstrate a lack of mathematical understanding, how can we use these exams to decide who deserves to pass, who deserves to graduate, and who deserves to keep their jobs?

Related Posts

• Regents Recaps
• The Worst Regents Question of All Time?
• More Trouble With Functions
• Regents Recap — June 2016: Algebra is Hard

 

This is the fourth entry in a series examining the 2011 NY State Math Regents exams. The basic premise of the series is this: If the tests that students take are ill-conceived, poorly constructed, and erroneous, how can they be used to evaluate teacher and student performance?

In this series, I’ve looked at mathematically erroneous questions, ill-conceived questions, and under-represented topics. In this entry, I’ll look at a question that, when considered in its entirety, is the worst Regents question I have ever seen.

Meet number 32 from the 2011 Algebra II / Trigonmetry Regents exam:

If , find .

This is a fairly common kind of question in algebra: Given a function, find its inverse. The fact that this function doesn’t have an inverse is just the beginning of the story.

In order for a function to be invertible it must, by definition, be one-to-one. This means that each output must come from a single, unique input. The horizontal line test is a simple way to check if a function is one-to-one. In fact, this test exists primarily to determine if functions are invertible or not.

The above function fails the horizontal line test and thus is not invertible. Therefore, the correct answer to this question is “This function has no inverse”. And now the trouble begins.

Let’s take a look at the official scoring guide for this two-point question.

[2]   , and appropriate work is shown.

This is a common wrong answer to this question. If a student mindlessly followed the algorithm for finding the inverse (swap x and y, solve for y) without thinking about what it means for a function to have an inverse, this is the answer they would get. According to the official scoring guide, this wrong answer is the only way to receive full credit.

It gets worse. Here’s another line from the scoring guide.

[1]  Appropriate work is shown, but one conceptual error is made, such as not writing with the radical.

In summary, you get full credit for the wrong answer, but if you forget the worst part of that wrong answer (the sign), you only receive half credit! So someone actually scrutinized this problem and determined how this wrong answer could be less correct. The irony is that this conceptual error might actually produce a more sensible answer. The further we go, the less the authors seem to know about functions.

And it gets even worse. Naturally, teachers were immediately complaining about this question. A long thread emerged at JD2718’s blog. Math teachers from all over New York state called in to the Regents board, which initially refused to make any changes. A good narrative of the process can be found at JD2718’s blog, here.

The next day, the state gave in and issued a scoring correction: Full credit was to be awarded for the correct answer, the original incorrect answer, and two other incorrect answers. By accepting four different answers, including three that were incorrect, you might think the Regents board would have no choice but to own up to their mistake. Quite the opposite.

Here’s the opening text of the official Scoring Clarification from the Office of Assessment Policy:

Because of variations in the use of notation throughout New York State, a revised rubric for Question 32 has been provided.

There are no variations in the use of this notation, unless they wish to count incorrect usage as a variation. I understand that it would be embarrassing to admit the depth of this error, which speaks to a lack of oversight in this process, but this meaningless explanation looks even worse. This is a transparent attempt to sidestep responsibility, or, accountability, in this matter.

It’s not just that an erroneous question appeared on a state exam. First, someone wrote this question without understanding its mathematical consequences. Next, someone who didn’t know how to solve the problem created a scoring rubric for it, and in doing so demonstrated even further mathematical misunderstanding. Then, all of this material made it through quality-control and into the hands of tens of thousands of students in the form of a high-stakes exam. And in the end, facing a chorus of legitimate criticism and complaint, those in charge of the process offer up the lamest of excuses in an attempt to save face and eschew responsibility.

It might not seem like such a big deal. But what if your graduation depended on it? Or your job? Or your school’s very existence? Then it’s a big deal. At least, it should be.

Related Posts

• Regents Recaps
• Another Embarrassingly Bad Math Exam Question
• More Trouble With Functions
• Regents Recap — June 2016: Algebra is Hard