Game Theory

A side comment by Matt about quizzes triggered a thought (so many of these interactions are induced rather than spontaneous)

I have all my old lecture notes and materials so the only real thing I have to do is make up new quizzes. Students are good at nothing if not gaming the system and they’d notice repeated quizzes pretty quickly.

When I TA’d I did labs, but the same idea applied. It was assumed that the students had access to old lab reports and exams (especially if they were in a fraternity or sorority) so the one thing we could make different was a question or two tacked on to the end of the calculations. And that did trip up a couple of students, who had obviously just copied from some old report to which they had access. Professors had various strategies about re-using questions, but I think the use of computers has made it far easier to keep a large database and mix-and-match questions that simple memorization of old exams prohibitive for introductory classes.

When I was teaching in the navy it wasn’t an issue. Quizzes didn’t count toward your grade, so there was no real incentive to cheat, other than trying to get out of some extra problems to be worked because the instructor might assign them to people who failed several quizzes. There was no master file of exam questions because they were treated as restricted material — the students did not keep them, and they were strictly accounted for. But to cut down on the possibility of some “oral tradition” information flow between the different classes in session, questions were not re-used until the class that had taken that exam had graduated.

We had one incident that occurred just before I had transferred into one division — an exam went missing. The most likely explanation is someone miscounted, but what was recorded was that there were 126 exams (and they were numbered) and after the exam was administered #126 was nowhere to be found. So the exam was assumed to have been compromised for future tests, and all of the questions on the exam had to be removed from the exam bank. As it turned out, I inherited the job of writing that particular exam, so it fell to me to repopulate the stockpile — two brand spankin’ new questions per exam for the next year, so I got a lot of practice coming up with new material. Which isn’t that hard, because an old question with new numbers and solving for a different variable is a “new” question. The use of old questions wasn’t laziness, though — we didn’t grade on a curve, and the goal was to test each class the same, so you kept statistics, and made tests that had a predicted result of between a 3.1 and 3.2 on a 4.0 scale. A venerable question was well-trusted, and a new or changed question could throw the result off. If a class got an unexpectedly high or low score (usually low), an audit was initiated to try and ensure that there was nothing hinky going on. This was especially odious for the early exams, before the class had a chance establish itself as being above- or below-average. If a class had underperformed on earlier exams, tanking a later exam didn’t raise eyebrows. But at least once the conclusion was that it was the Russian judge a new question or two were harder than had been predicted, and had shaved a few tenths off the score.

But even within that strict paradigm, an exam-writer could game the system a little. No matter how much you’d drill it into the students’ heads to skip a tough question and go back to it later, there were those who didn’t. They’d invariably leave an easy question or two blank because they took too much time on another question that they still got mostly wrong. So putting tougher questions toward the front would tend to lower scores a little bit.

0 thoughts on “Game Theory

  1. You taught in the Navy? Were you at the Academy? I was an adjunct at the Academy in the spring of 2001. It’s what convinced me to go into academia and teaching (I was a self-employed engineer at the time).