When it comes to educational testing, the stakes are higher than ever.  For a student, tests might determine which public schools you can attend, if and when you graduate, and which colleges are available to you.   For schools and districts, aggregate test scores and the “progress” they show might determine what kind of state and federal aid is available.

As a means of evaluating teachers, student test scores are playing an increasing role.  Indeed, state laws have been re-written to mandate the use of standardized test data as a substantial factor in rating teacher performance.

There is controversy regarding the value of standardized tests, even as measures of student achievement (in most cases, their purported purpose).  A very public debate has emerged as politicians attempt to make education more “data-driven” and hold teachers and schools more “accountable”.   But one fundamental question is rarely raised in this conversation:  are these tests any good?

If the tests we use to evaluate students, schools, and now teachers, are ill-conceived, sloppy, and erroneous, how legitimate a measure of teaching and learning could they possibly be?  The issue of test quality and relevance seems like an important one, but it gets very little attention.

In this series, I address the question “Are These Tests Any Good?” by looking at a collection of questions from the 2011 New York State Math Regents Exams.  My cursory analysis reveals many significant issues with how these tests are created:  mathematical errors; poorly-worded questions; the de-emphasis of knowledge; and misalignment with course curricula.

If we can’t create legitimate, relevant, appropriate tests, should we really be using them to evaluate teachers?  Or students?

Are These Tests Any Good?

Part I:  Mathematically Erroneous Questions

Part II:  Ill-Conceived Questions

Part III:  Underrepresented Topics

Part IV:  The Worst Math Regents Question of All Time

Part V:  9th Grade Questions on 10th and 11th Grade Exams

Here are some other resources on this topic.

This blog by JD2718 offers a similar critique of Regents exams from 2009.

Here’s some fun I had with one of this year’s Regents questions involving the famous 13-14-15 triangle.

Related Posts

• Regents Recaps

This is the fifth entry in a series examining the 2011 NY State Math Regents exams.  The basic premise of the series is this:  if the tests that students take are ill-conceived, poorly constructed, and erroneous, how can they be used to evaluate teacher and student performance?

In this series, I’ve looked at mathematically erroneous questions, ill-conceived questions, under-represented topics, and what is perhaps the worst question in Regents history.  In this entry, I’ll use questions from two exams to discuss duplication, lowered-expectations, and poor test construction.

Number 37 from the 2011 Geometry Regents exam is a 4-point question which asks students to solve the following system of equations graphically:

Number 39 from the 2011 Algebra 2 / Trigonometry Regents exam is a 6-point question which asks students to solve the following system of equations algebraically:

These two systems of equations are roughly equivalent in terms of difficulty.  Why is a question suitable for the Geometry exam appearing on a the Alg 2/Trig exam, and as the highest-valued question (6 points) to boot?  In New York state, the Alg 2/Trig course follows Geometry in the standard sequence, so it is strange to see the same kind of problem on two state exams that are designed to be taken at least a year apart.

It’s true that the Alg 2/Trig test question asks for an algebraic solution, as opposed to a geometric solution, but that is essentially the only difference between the two.  This being the case, this speaks to a serious problem in how these tests are conceived and designed.

Looking at these two tests, one might conclude that learning to solve this kind of system algebraically is an important part of the Alg 2/Trig course:  why else would the official exit exam require the use of this technique in solving a problem that could have been solved last year?

Solving systems algebraically is definitely is a fundamental skill; so fundamental, in fact, that it is part of the Integrated Algebra curriculum (see the Integrated Algebra Pacing guide on the official schools.nyc.gov website).  Integrated Algebra is the course students take before they take Geometry!  Since many students take IA in 9th grade and take Alg 2/Trig in 11th or 12th grade, this means that a 6-point question on the Alg 2/Trig exam is testing the student’s ability to solve a problem they should have been able to solve two math courses ago.

Students should be able to solve this kind of problem at all mathematical levels, but why is material from two courses ago playing such a prominent role on an advanced exit exam?  What Alg 2/Trig course material is being shortchanged in order to re-test more elementary skills?  And to the point, how can this be considered a legitimate assessment of what a student learned in an Alg 2/Trig course?

Furthermore, in each case the scoring guide allows for half credit if the problem is solved using a method different than the one specified.  This is a reasonable policy, but what then is the purpose of a question specifically designed to test knowledge of a technique?  On the Alg 2/Trig test, a student can earn half credit for solving the system graphically; that means a student can get 3 of the 6 points by simply doing exactly what they did on the same problem on last year’s Geometry exam.

This example highlights how some questions on these exams aren’t directly connected to the content of their respective courses.  If a test isn’t legitimately designed around the curricula and content of the course, how can teachers and students effectively prepare?  How could such tests be valid assessments of what a student learns in that class?  Or how effectively a teacher teaches?  These are all questions that aren’t asked enough in the debate about standardized tests, student performance, and teacher accountability.

Related Posts

• Regents Recaps
• Regents Recap — June 2016: How Much Should This Be Worth?