Regents Recap — January 2013: Question Design

Here is another installment in my series reviewing the NY State Regents exams in mathematics.

One consequence of scrutinizing standardized tests is a heightened sense of the role question design plays in constructing assessments.

Consider number 14 from the Integrated Algebra exam.

Regents 2013 January IA 14

In order to correctly answer this question, the student has to do two things:  they need to locate the vertex of a parabola; and they need to correctly name a quadrant.

Suppose a student gets this question wrong.  Is it because they couldn’t find the vertex of a parabola, or because they couldn’t correctly name the quadrant?  We don’t know.

Similarly, consider number 21  from the Geometry exam.

Regents 2013 January G 21

This is a textbook geometry problem, and there’s nothing inherently wrong with it.  But if a student gets it wrong, we don’t know if they got it wrong because they didn’t understand the geometry of the situation, or because they couldn’t execute the necessary algebra.

Using student data to inform instruction is a big deal nowadays, and collecting student data is one of the justifications for the increasing emphasis on standardized exams.  But is the data we’re collecting meaningful?

If a student gets the wrong answer, all we know is that they got the wrong answer.  We don’t know why; we don’t know what misconceptions need to be corrected.  In order to find out, we need to look at student work and intervene based on what we see.

And what if a student gets the right answer?  Well, there is a non-zero chance they got it by guessing.  In fact, on average, one out of four students who has no idea what the answer is will correctly guess the right answer.  So a right answer doesn’t reliably mean that the student knows how to solve this problem, anyway.

So what then, exactly, is the purpose of these multiple choice questions?

Regents Recap — January 2013: Where Does This Topic Belong?

Here is another installment in my series reviewing the NY State Regents exams in mathematics.

There seems to be some confusion among the Regents exam writers about when students are supposed to learn about lines and parabolas.  Consider number 39 from the January 2013 Integrated Algebra exam:

regents january 2013 ia 39

Compare the above problem with number 39 from the June 2012 Geometry exam:

regents june 2012 g 38These questions are essentially equivalent.  They both require solving a system of equations involving a linear function and a quadratic function by graphing.  Yet, they appear in the terminal exams of two different courses, that are supposed to assess two different years of learning.

When, exactly, is the student expected to learn how to do this?  If the answer is “In the Geometry course”,  the Algebra teacher can hardly be held accountable if the student doesn’t know how to solve this problem.  And if the answer is “In the Integrated Algebra course”, what does it mean if the student gets the problem wrong on the Geometry exam?  Is that the fault of the Geometry teacher or the Algebra teacher?  The duplication of this topic raises questions about the validity of using these tests to evaluate teachers.

And if that isn’t confusing enough, check out this problem from the 2011 Algebra 2 / Trig exam.

regents june 2011 at 39Here, we see the same essential question, except now the student is required to solve this system algebraically.  These three exams–Integrated Algebra, Geometry, Algebra 2/Trig–span at least three years of high school mathematics.  In the integrated Algebra course, a student is expected to solve this problem by graphing.  Then, 2 to 3 years later, a student is expected to be able to solve the same kind of problem algebraically.

What does that say about these tests as measures of student growth?

Regents Recap — January 2013: What Are We Testing?

Here is another installment in my series reviewing the NY State Regents exams in mathematics.

One significant and negative consequence that standardized exams have on mathematics instruction is an over-emphasis on secondary, tertiary, or in some cases, irrelevant knowledge.  Here are some examples from the January 2013 Regents exams.

First, these two problems, number 10 from the Algebra 2 / Trig exam, and number 4 from the Geometry exam, emphasize notation and nomenclature over actual mathematical content knowledge

Regents 2013 -- Tertiary knowledge 1

Rather than ask the student to solve a problem, the questions here ask the student to correctly name a tool that might be used in solving the problem.  It’s good to know the names of things, but that’s considerably less important than knowing how to use those things to solve problems.

The discriminant is a popular topic on the Algebra 2 / Trig exam:  here’s number 23 from January 2013:

Regents 2013 January AT 23

It’s good for students to understand the discriminant, but the discriminant per se is not really that important.  What’s important is determining the nature of the roots of quadratic functions.

If you give the student an actual quadratic function, there are at least three different ways they could determine the nature of the roots.  But if you give them only the discriminant, they must remember exactly what the discriminant is and exactly what the rule says.  This forces students and teachers to think narrowly about mathematical problem solving.

In number 3 on the Algebra 2 / Trig exam, we see a common practice of testing superficial knowledge instead of real mathematical knowledge.

Regents 2013 January AT 3

Ostensibly, this is a question about statistics and regression.  But a student here doesn’t have to know anything about what a regression line is, or what a correlation coefficient means; all the student has to know is “sign of the correlation coefficient is the sign of the coefficient of x”.  These kinds of questions don’t promote real mathematical learning; in fact, they reinforce a test-prep mentality in mathematics.

And lastly, it never ceases to amaze me how often we test students on their ability to convert angle measures to the archaic system of minutes and seconds.  Here’s number 35 from the Algebra 2 / Trig exam.

Regents 2013 January AT 35

A student could correctly convert radians to degrees, express in appropriate decimal form, and only get one out of two points for this problem.  Is minute-second notation really worth testing, or knowing?

Regents Recap — January 2013: Unstated Assumptions

Here is another installment in my series reviewing the NY State Regents exams in mathematics.

Consider the following problem from the January 2013 Algebra 2 / Trigonometry exam:

regents june 2011 at 28

What’s interesting about this problem is what it doesn’t tell you.

The student here is expected to assume that the terms in the sequence keep going up by 9.  That is, the student is expected to assume that the sequence is arithmetic.  Once the student makes that assumption, they can use the appropriate formula to sum the first 20 terms.

Encouraging students to assume that arbitrary sequences are arithmetic is bad practice.  It can develop in students a sense that sequences are usually arithmetic, which will make it harder for them to understand non-arithmetic sequences later on.

More generally, we don’t want students getting into the habit of making unconscious assumptions about problems.  By forcing them to make assumptions that fit the problem to the test, questions like this train students not to ask questions like “Are we sure the next term is 50?  What else might it be?”.  This was one of the main points of my talk “g = 4, and Other Lies the Test Told Me“.

It’s not outrageous that think that this sequence might be arithmetic.  But the mathematical world is a rich and complex place.  Are we really sure that this sequence is arithmetic?  After all, maybe what we’re really looking at is the sequence of numbers whose digits add to five!

Am I thinking too deeply about this sequence?  Maybe.  But as a teacher, thinking deeply about mathematics is precisely what I want my students to do.

When Does Teaching Cease to Be a Challenge?

An interesting conversation about teacher retention emerged recently, beginning with Shawn Cornally thinking outloud about how to keep good teachers in the system, and later moving over to Dan Meyer’s blog where he discussed the twin pressures on novice teachers.

In describing how good teachers often ultimately feel a pull out of the classroom, Dan Meyer says

The job becomes untenable at about the same time that it becomes unchallenging.

The point he’s making is that teaching becomes easier, and as it does, it ceases to be a challenge for good teachers, who are then more likely to leave the classroom in search of other challenges.

There are valid, relevant issues raised here, but the suggestion that teaching ceases to be a challenge at some point sounds crazy to me.

Teaching is always a challenge.  Experience may make certain practices more efficient, but in some ways, that efficiency only makes the deeper challenges easier to see.

Consider the endless challenges offered by the three major components of teaching: knowledge of students, knowledge of pedagogy, and knowledge of content.

A good teacher must know their students.  Every new student, and new class, presents unique challenges to a teacher, who has to forge positive relationships and create productive environments.  This may get easier with experience, but it’s always a challenge, and can always be improved upon.

Since teaching is about understanding how learning happens, the fact that we don’t fully understand how learning happens creates another set of evolving challenges.  There are always new ideas to consider, new practices to try, new approaches to instructional design, and of course, new technologies to integrate.  Trying to figure out how learning happens is a daily challenge for a teacher, and it may never fully be understood.

And when it comes to content knowledge, no teacher could feel more challenged than a math teacher.  Some of the smartest people in the world spend their lives in a perpetual state of learning mathematics.  There is always more mathematics to study, new connections to find, new perspectives on old problems, and old problems to make new again.  Understanding mathematics is a never-ending challenge.

At some point, a good teacher may decide that these challenges are no longer meaningful enough to justify the great effort and investment that teaching requires.  It’s understandable, and in that case, leaving the classroom may be a courageous and noble decision:  walking away from something you do well in order to follow a deeper passion is admirable, and it also sets a good example for students.  But this isn’t because teaching ceases to be challenging; it’s because the individual no longer feels motivated by those challenges.

The task of teaching is infinitely deep and infinitely varied.  I often feel that, in becoming a better teacher, I simply become more aware of what I need to do differently.  Like mathematics itself, teaching becomes more complex the more you know about it.  What could be more challenging than that?

Follow

Get every new post delivered to your Inbox

Join other followers: