Towards Better Test Taking Software
Almost thirty years ago, when I was teaching at Tulane Business School, I had a project to develop software to let students take exams, and professors grade them, on computer—the name of the product was going to be "Electric Blue Book." Unfortunately, it never got to completion--something that has happened with a number of my projects. But many years later, other people did create such software. The version I'm familiar with, because my law school uses it, is Computest.
What is depressing about the software is that all of the ingenuity has gone, not into making it easier to take and grade exams, but into making it harder to cheat. From the standpoint of the user, the software is clearly inferior to what I designed but never got programmed. For example ...
In my version, the student would see a page listing questions and showing, for each question, whether he had done it and whether, if so, he wanted to go back to it if there was time. In my version, the professor would be able to have the software feed him all the answers to question one in random order, then all the answers to question two, and so on. Doing it that way makes it easier to remember what you are giving how much credit for than if you graded all of exam one, then all of exam two, then ... .
It also eliminates an important bias in grading—the tendency to form an opinion of a student based on his answer to one question and then let that opinion distort the grade you give him on another. As my wife pointed out when I was discussing this recently with her, that is a problem discussed by Daniel Kahneman in his (very good) Thinking Fast and Slow. He actually did an experiment, and found that grading question by question instead of exam by exam resulted in the performance of students coming out less consistent—made it less likely that the student who got a good grade on question one also got a good grade on question two. He concluded that he had a strong bias towards imposing consistency on his grading, even when it wasn't there.
Unfortunately, none of this is present in Computest. It would be nice if someone involved in producing the program happened to read this post and did something about it, but not very likely.
But not all of it has to be done by them. It should be possible to write a program that would take the output of Computest, the files representing what each student wrote on the exam, and reorganize it so that the professor doing the grading could grade by question instead of by exam. I cannot think of any comparable way of fixing things at the other end, of making it easier to take the exam, but perhaps one of my readers can.
What started me thinking about this was the recent experience of getting exams that had been taken using Computest on a flash drive from my school, grading them on my computer and returning them by giving the flash drive with graded (and commented) exams on it to the students to pass among themselves, with each copying his exam. It would have been easier if I could have emailed the exams to them—for one thing, that would have given them a chance to look at them before class and see if they had questions. But Computest requires the student's ID number, not his grade, and under my school's blind grading system that means that I don't know the name and so cannot email the exam back. I can imagine a number of ways of fixing that problem, but do not know if any will prove workable.