By: Barry Shechter
Connect with Barry on LinkedIn
After we compiled a sufficient number of exams... We looked at the choices to determine which were high distractors, meaning they were incorrect answers...
...students that didn't quite understand the material would select them
I worked in the School of Education at Queens College for multiple Educational Psychologists and psychometric analysts. One of the tools I created under the direction of Lawrence Castiglione, Ed.D. was for measuring test distractors on exams and for producing exams using a weighted formula.
Students took exams and filled in the OCR cards which were then scanned through a card reader. The program marked any outliers, for example F where the choices were A through E, and incompletely darkened circles. Those cards were hand entered into the program. The program then graded each of the OCR cards and computed the grade for the student and the professor.
After we compiled a sufficient number of exams in the same subject area we had a database of questions and their choices. We ran SPSS against the data to see which question choices were most frequently chosen by students going down to the least chosen. We looked at the choices to determine which were high distractors, meaning they were incorrect answers although students that didn't quite understand the material would select them and which were low or null distractors.
As we progressed we developed a better question and choices set and made the exams more productive in terms of diagnostics.
Using data analysis we can make better exams which will aid students in diagnosing where in the subject matter they need to ask more questions for better understanding.
The idea is to get students to ask more questions in class for better comprehension and retention.
I used this program for my Computer Science classes at Queens College as I had 80 or more questions per exam and needed a way to randomize the exams and give the students a better tool to diagnose their strengths and those points they needed to strengthen.
The concept was clear. Improve the tests to improve the feedback to the students in order for them to maximize their understanding and retention of the material.