Nils Rauhut and Tziporah Kasachkoff “Everything you always wanted to know about multiple choice question in philosophy—but were afraid to ask.”

Nils: When I came to philosophy, I thought multiple choice questions were a violation of the spirit of philosophy. But really, multiple choice questions do things that no other assessment cannot do.

Tziporah: I always thought multiple choice questions were appropriate for philosophy. After all, we take the GRE. But I did have problems with essay exams, because the research shows that all sorts of arbitrary things bias essay grading, including penmanship, the name that appears on the top of the test, and framing effects. Also, essay exams are impractical.

Audience guy: I’m interested in multiple choice questions as learning events. Research has shown that mcq with feedback improves performance on essays down the line.

Tips:   

·         Good multiple choice questions take time to write.

·         It really helps to have someone else take your multiple choice tests. If you can’t give your test to a colleague, give it to yourself 10 days later.

·         If a large % of the class gets it wrong, there is something wrong with the question or the way you teach the material.

·         If students identify an ambiguity, be humble with t. Just because you are the author of a question, doesn’t mean you are the final judge on what the correct answer is.

·         Don’t ever put in wildly implausible answers.

The literature refers to the part of the question that the answers are about is called the stem. The false answers are called the distracters..

A bad multiple choice question.

What is an argument in philosophy?

a.                   A factual disagreement between people

b.                  Giving reasons for belief

c.                   A shouting match

d.                  Any verbal attempt to persuade

Problem, according to Tziporah: (c) is too implausible to be a good distracter.

First question: What do I want this multiple choice test to do?

Tests to check if students have done the reading.

Nils uses basic reading comp questions (7-8 minutes) at the start of classes, and have them grade their peers tests, to embarrass the students who haven’t done the readings. You need at least 10 questions for this to work.

Questions to ask yourself: Is it really the case that someone hasn’t done the reading will do badly and someone who has done the reading will do well.

He hands out a sample quiz. Could a student who did the reading actually ace this quiz? Paul: This quiz should be open book.

Tests to check higher level of comprehension.

Tziporah: gives an example of a forking sequence of questions, where you have to justify your answer to the first question in the second question. Students don’t get questions for the right answer unless they also answer the reason question correctly. Students also get no credit for wrong answers for the first question, no matter how good the answer to the second question is.

Often these aren’t used for [summative] assessment at all.

Tests practice key logical skills.

• Identify the conclusion
• Identify similarities to other arguments.
• Identify an objection.