Monday, June 3, 2013

Exam wrappers

This description of "exam wrappers" is taken from an online newsletter hosted by Stanford University and edited by Rick Reis: Tomorrow's Professor. For more information, visit http://mailman.stanford.edu/mailman/listinfo/tomorrows-professor

What are Exam Wrappers?

Exam wrappers are short activities that direct students to review their performance (and the instructor's feedback) on an exam with an eye toward adapting their future learning. Exam wrappers ask students three kinds of questions: How did they prepare for the exam? What kind of errors did they make on the exam? What could they do differently next time? Each of the question types is discussed next.

1. How students prepare for the exam. Asking students to reflect on how they prepared for the exam forces them to confront the choices, explicit or implicit, they made about their studying. This prompts students to consider issues such as whether they studied enough or sufficiently in advance. Similarly, asking students which of various study strategies they employed (e.g., reviewing notes, solving practice problems, rereading the textbook) highlights that there are many options they could have taken. It also presents various study strategies that students might not have even considered and thus suggests some new possibilities for how they might prepare differently next time.

2. What kind of errors students made. Once they have received a grade, students do not always think carefully about their performance on an exam. If they did well, they might mark it as a success without much further thought; if they did poorly, there's strong temptation to leave the painful event behind. Thus, the second set of questions posed in exam wrappers is designed to encourage students to analyze their performance in greater depth, giving students something constructive to do with feedback and graded exam offers. One way to do this is to identify the critical components or stages of the tasks on the exam and have students estimate their degree of difficulty (e.g., how many points they lost) with each component. For example, did they read the question carefully, did they have trouble "setting up" the problem, did they fail to understand the concepts involved, or did they make mistakes on the required arithmetic or algebra? Focusing students' reflection at this level informs their analysis of their own performance. Moreover, the labels for the different possibilities provide a concrete language for students to use when assessing their own performance. Note that this part of the exam wrapper is a natural place for instructors to tailor the questions to their own needs. Instructors may also want to adapt these labels to include specific misconceptions or difficulties that have been revealed in their course by past students. Or, they may want to include more general issues that impact students' exam performance. For instance, an instructor I worked with recently was concerned about test anxiety adversely affecting her students' exam performance, so she incorporated this issue into her exam wrapper.

3. How students should study for the next exam. Students can use their responses to the first and second types of exam-wrapper questions to think about how they should approach the next exam. A key goal of the third type of question exam-wrapper question is to help students see the association between their study choices and their exam performance so they can better predict what study strategies will be effective in the future. One way to do this task is to ask students to look back at their responses to the first two parts of the wrapper and then to list specific ways they might prepare differently for the next exam to improve their performance. Another option is to prompt students to attribute their various difficulties (from part two of the wrapper) to specific study strategies (from part one) they did or did not employ. Rather than merely telling students to "study harder" or "do more practice problems before the exam", this third type of exam-wrapper question helps students discover effective study strategies on their own. In effect, exam wrappers are asking students to give their future selves advice.

Why Exam Wrappers Work

The earlier section on design considerations described five constraints that a metacognitive intervention should satisfy to be practical and effective. Understanding the ways in which exam wrappers satisfy these constraints helps explain why they work.

1. Impinge minimally on class time. Exam wrappers require only a few minutes and are completed at a time when students arguably are somewhat distracted anyway. Moreover, instructors can eliminate even this minimal impact on class time by giving exam wrappers as part of homework.

2. Be as easily completed by students within the time they are willing to invest. With typically only one page of questions, none of which requires much writing, exam wrappers take relatively little student time.

3. Be easily adaptable. As we have seen, exam wrappers include three main question types, and these question types can be applied to almost any course, as long as it has exams. In addition, this general approach can be applied to any type of graded assignment.

4. Be repeatable yet flexible. The core questions being in an exam wrapper do not diminish in value when asked repeatedly. At the same time, it is easy to adjust the details of exam wrappers so as to keep the exercise fresh. Instructors can easily vary the specific content of exam wrapper questions, add new questions, and tailor the questions to their particular instructional situation.

5. Exercise the skills instructors want their students to learn. The reflections required to complete an exam wrapper leads students to assess their own strengths and weaknesses, evaluate their performance, identify which strategies work for them, and generate appropriate adjustments. These are key metacognitive skills that many instructors want to promote.

How to Use Exam Wrappers

Here I describe a basic recipe for how to use exam wrappers, along with variations and options instructors may find useful.

Step1: Students prepare for and take the exam using their typical study strategies. No special intervention is need for this first exam.

Step 2: The instructor gives students the exam wrapper instrument when the graded exams are returned and asks students to complete the exam wrapper as soon as possible upon seeing their exam performance. Ideally, this is done right then in class and need only take 10 minutes of class time. But there are other possibilities. Students can complete the exam wrapper as homework, submitting their responses by a specified deadline. Or students can complete the exam wrappers online as a nongraded assignment (e.g., within a course management system or online instructional environment or with an online survey system).

Step 3: The instructor collects the exam wrappers. Although exam wrappers are not graded activities, it is important to collect them because (a) the exam wrapper will need to be returned to students at a later point (see Step 4) and this prevents them from getting lost, and (b) the instructional team may want to review students' responses to gain insight into their students' learning that they otherwise might not be able to obtain. In particular, the instructor or teaching assistants can skim students' responses to see whether there are patterns in how students analyzed their strengths and weaknesses or in how students described their approach to studying for the exam. For example, instructors may be surprised to learn the amount of time students spent studying (either how much or how little), when students chose to start their studying (e.g., 2 a.m. the night before), or how students spent their study time (e.g., memorizing formulae rather than solving practice problems). The instructional team can also consider the additional instructional support students said they would like to receive and possibly provide something along these lines. A wide variety of adjustments may be suggested based on what the instructional team learns from the exam wrappers about students? strengths and weaknesses, and study strategies.

Step 4: At the time when students should begin studying for the next exam, the instructor returns the completed exam wrappers (from the previous exam) to students. The idea here is that students review their own recommendations for how to study more effectively, given their own past experiences, strengths, and weaknesses. Depending on the class format and the time available, there are many variations on this step that instructors can use.


Step 5 (optional, but desirable): Repeat steps 2 through 4 for subsequent exams.

When an instructor provides exam wrappers regularly across multiple exams, students get repeated practice in applying the skills of self-regulated learning. This helps students build a habit of mind to monitor their own learning, reflect on their study strategies, and make appropriate adjustments. For reasons mentioned earlier, it can be useful to include nontrivial variation in the structure of subsequent wrappers while still prompting the desired metacognitive processes. For example, an exam wrapper used for an exam later in the semester can be streamlined compared to the first exam wrapper. Having the wrapper still gets students to engage in reflection and analysis, but fading the scaffolding encourages students to take on more of the responsibility for this process.

These five steps are easy to implement, take relatively little time, and are very flexible. The metacognitive practice from using wrappers in a course offers substantial benefits. And when multiple instructors do this across different courses, students can learn metacognitive skills in multiple contexts, thereby increasing their likelihood of transferring the skills to new learning situations in the future. This is exactly what we did at Carnegie Mellon University, implementing wrappers in several introductory math and science courses, as described in the next section.