Data was collected from two terms of a 10-week course at a large public university, with quizzes being administered five times throughout the semester, and 372 students total completing all items. The quizzes contained three multiple-choice questions, and were administered 3-4 days after the material was taught. The quiz was returned after 3-4 days. Two types of questions, previously tested and related, then appeared on a 50-item cumulative final exam. To prevent bias, topics were randomly selected from a list to be on post-lecture quizzes, and questions were created by writers blind to the purpose of the questions. Furthermore, each question-pair (on quiz and on final) was assigned to one condition of identical repeat, conceptual-repeat, baseline control, or filler. Identical-repeat pairs were one question that would be on both the quiz and final. Conceptual-repeat pairs had the same concept, but different questions on the quiz and final. Control questions were on the final, but not any quizzes, serving as a baseline control in that they could have been on quizzes, but were not. Filler questions were randomly selected questions on quizzes to bring up the number of questions to three and did not appear on the final content or question-wise. From the five quizzes across the term, the same 15 questions were on the 50-question final for every student.

Methods for Multiple-choice testing as a desirable difficulty in the classroom

Students showed a 10% increase ability to answer previously tested questions on the final than on the quiz. Their ability to do so was also greater than their ability to answer baseline questions (M=92%, SD=13% to M=72%, SD=21%). Students also showed a greater ability to answer questions that had concepts that were previously quizzed, at a rate greater than their ability to answer baseline questions (M=90%, SD=14%). In all cases, final exam performance for same and related items was above baseline items and performance on initial quizzes. If students had correctly answered a question on the initial quiz, they were more likely to correctly answer a related question on the final.

Results from Multiple-choice testing as a desirable difficulty in the classroom

The results showed that the retrieval practice provided by testing/quizzing functioned as a desirable difficulty for students that was not only limited to specific information, but also conceptually similar information. Benefits of this desirable difficulty are thought to be from retrieval practice during quizzes, the quizzes being a source of information for students to know what concepts they need to study, the quizzes encouraging students to engage in spaced study, and additional learning at the time of feedback. Benefits could also have been seen during quizzes, when information about concepts that were incorrect alternatives had to be processed when deciding which choice was correct. This could then lead to strengthening associations between alternatives and information about those alternatives that could have been on the final exam.

Discussion for Multiple-choice testing as a desirable difficulty in the classroom

University of Texas at Austin

Results of previous studies done in differing classroom environments and different quiz formats showed that there should be benefits of testing if questions on the final exam were related in content to questions on quizzes throughout the semester. The two main goals of the study were to determine the effects of quizzing content that would be on a final exam compared to no previous quizzing, and to determine the effects of quizzes on the retention of information that are conceptually related to quizzed content but had not been on quizzes. In short, to see if testing is a desirable difficulty for learning specifically tested material and/or material related to quizzed material.

Multiple-choice testing as a desirable difficulty in the classroom

Bjork, E. L., Little, J. L., &amp; Storm, B. C. (2014). Multiple-choice testing as a desirable difficulty in the classroom. Journal of Applied Research in Memory and Cognition, 3(3), 165-170. doi:10.1016/j.jarmac.2014.03.002
https://doi.org/10.1016/j.jarmac.2014.03.002

Learn Before

Learn After