Collaborative Oral Take-Home Exams

Giving Collaborative oral take-home examinations allows the instructor to assess how well students handle the kind of non-routine problems we would all like our students to be able to solve.

Background and Purpose

I began assigning Collaborative Oral Take-Home Exams (hereafter called COTHEs) because of two incidents which happened during my first year of teaching at Franklin & Marshall College. My classes have always had a large in-class collaborative component, but until these two incidents occurred, I gave very traditional exams and midterms.

The first incident occurred as I was expounding to my students in an integral calculus course upon the virtues of working collaboratively on homework and writing projects. One of the better students asked: if working together is so desirable, why didn't I let students work together on exams? What shook me up so much was the fact that I had no answer. I could think of no good reason other than tradition, and I had already parted from tradition in many other aspects of my teaching.

The second incident followed closely on the heels of the first. I visited the fifth International Conference on Technology in Collegiate Mathematics, where I heard Mercedes McGowen and Sharon Ross [3] share a talk on alternative assessment techniques. McGowen presented her own experiences with using COTHEs, and this—combined with my student's unanswered question still ringing in my ears—convinced me to change my ways. Although the idea behind COTHEs originated with McGowen, I will discuss in this article my own experiences—good and bad—with this form of assessment.

Method

I usually assign four problems on each COTHE, each of which is much more difficult than anything I could ask during a 50-minute period. I hand out the exam itself a week before it is due. Because of this, I answer no questions about "what will be on the midterm," and I hold no review sessions— they do the review while they have the exam.

These are the instructions I provide to my students:

Directions This is an open-note, open-book, open-group exam. Not only are you allowed to use other reference books and graphing calculators, but in fact you are encouraged to do so, as long as you properly cite your resources. However, please do not talk to anybody outside your group, with the possible exception of me; any such discussions will, for the purposes of this midterm, be considered plagiarism and will be grounds for failure for the midterm or the course.

You should work on this midterm in groups no larger than 3 people, unless you ask for permission beforehand. Everybody in the group will receive the same grade, so it is up to the group to make sure that all members understand the material.

You and your group will sign up for a half-hour time slot together. On the day of your exam, please bring your written work to hand in — the group may submit one joint copy or several individual ones. You may bring solutions, books, and calculators to use as reference. You should be prepared to show your graphs to me, and you may of course use your written work to aid you with the oral questions. However, you will not be graded directly on your written work; rather you will be graded on your understanding of the material as presented to me.

I begin each appointment by explaining the ground-rules for the interview. I will choose students one at a time to answer the questions. While that student is answering, all other group members must be silent. Once that student has finished, the other students will be permitted to add, amend, or concur with the answer. I accept no answer until the whole group has agreed. Then I move to a new student and a new problem. The questions I ask tend to go backwards: what did you find for an answer to this problem? How did you go about finding this answer? Did you try any other methods? What made this problem hard? Does it look like other problems that you have seen?

Although each exam appointment is 1/2 hour, I leave 15 minutes in between appointments for spill-over time, which I invariably end up using. The interviews themselves usually take a whole day for a class of 30 students — when I teach 2 calculus classes, I have interviews generously spread over a 3-day period, with lots of time left free. While this certainly seems like a lot of time, I spend no time in class on this exam: no review sessions, no questions about what will be covered, no in-class test, and no post-exam follow up. Better yet, I spend no time grading papers!

Since I assign the grades as the students take their exams, I have had to learn how to present the grades face-to-face. I've never had a problem telling students that they earned an `A.' To students who earn `B's, I explain that they are doing "solid" work, but that there is still room for improvement — and I explain just where (perhaps they need to be able to relate visual understandings back to the algebraic definitions). I tell students who do `C' work that they're really not doing quite as well as I would like, for reasons X, Y, and Z. Notice that the emphasis here is not that I think that they are stupid or lazy, but rather that I have high expectations for them, and that I hope that this situation of weak work is temporary. For students who earn `D's or worse, I explain that I am very worried about them (again, for reasons as broad as that their algebra skills are very weak and they seem to be confused about fundamental concepts in the course) and that they will need substantial effort to do better in the course.

Findings

The idea of an oral exam in mathematics strikes terror in the heart of the student, but knowing the questions beforehand and having their whole group present at the interview goes a long way to mitigating that fear. Despite the newness and scariness of taking oral exams, students wind up being fond of this form of exam. Indeed, by the end of the course what they are most worried about is taking the final exam as individuals—a turnabout I hadn't expected!

Use of Findings

From the instructor's perspective, the exam interviews become a mint of insight into the minds of the student. I gain a real appreciation of what was hard and what was easy about each problem. I get a chance to challenge misconceptions directly, and also to encourage flashes of insight. Moreover, because every student not only has to come to my office, but also talk directly to me, it becomes much easier to carry conversations about mathematics or study habits into out-of-exam time. The benefits of these brief obligatory encounters spill over into the rest of the semester.

Several of my students told me that for the first time they learned from their exam mistakes; that when they get traditional exams back covered with comments, they can not bear to read any of it, so painful is their remembrance of the midterm. Whereas in a COTHE, they get the feedback even as they present their results; and since they have debated the results with their teammates, they care about the answer. All of them appreciate the time they get to do the problems —a frequent comment I get is that they'd never be able to do these problems on a timed exam. (Of course, I would never assign such difficult problems on a timed exam, although that does not seem to occur to them.)

I now design my exams to be both forward and backward looking—in this way I can use the interviews as a way of preparing students for material to come, and myself for their reaction to it. (For example, Problem D at the end of this paper requires students to do something that we encourage them to do in the regular course of the class: to read a textbook and learn new mathematics. The COTHE allows me to test this skill and to build on this material.)

What my students tell me during their exam tells me a lot about their "trouble spots." One year several groups of students were concerned about whether a function could be concave up at a point of discontinuity (see Problem A at the end of this paper). Although this would not have been my own choice of an important topic to cover, I then spent a half day in class working with the students on this issue and its relative importance to the subject of Calculus, and the students seemed to appreciate the class.

Often I discover—and can immediately correct— difficulties with interpreting the displays on graphing calculators. Sometimes I discover, to my delight, that the students really understand a concept (such as horizontal asymptotes) and that I can safely use that concept to introduce new topics (such as limits of sequences).

I enjoy using "stupid mistakes" as an opportunities to see what students know about the problem. If a students makes an arithmetical or algebraic error at the beginning of a problem on a written test, it can change the nature of the rest of the problem. On an oral exam though, I might say, "At this step, you decided that the absolute value of -x is x. Can you tell me why?" At that point, the students bang themselves on the head and then huddle together to work the problem through again. This opportunity to see the students in action is illuminating (plus it provides them the opportunity to get full credit on the problem). We discuss the correct solution before the students leave.

Success Factors

The most hard-won advice I have learned about assigning COTHEs has to do with the grades:

Do not assign grades by points. Providing so many points for getting so far, or taking off so many points for missing thus-and-such a fact, tends to be at odds with what the students really know. Also, because the students do so well on this kind of exam, the grades bunch up incredibly. I once was in the awkward position of explaining that `83-85 was an A' and `80-82 was a B.' Neither the students nor I appreciated this.

Do decide on larger criteria beforehand, and do share these criteria with the students. Pay attention to the larger, more important areas that make up mathematical reasoning: mechanics (how sound are their algebra skills; can they use the graphing calculator in an intelligent way?); concepts (do they understand the principles that underlie the notation? Can they make links between one concept and another?); problem solving (can they read mathematics, attempt various solutions, etc.?). I assign grades based on the strength of these areas, and try to provide at least cursory feedback on each.

There are occasional concerns from the students about getting group grades, and occasional concerns I have about losing track of how individuals are doing (particularly I worry about not catching the weak students early enough), and for that reason I'm moving in the direction of a two-part exam (similar to the "pyramid" exam described by Cohen and Henle in [1]). The first part of my two-part exam would be a COTHE, and the second would be a series of follow-up questions taken individually in class.

References

[1] Cohen, D. and Henle, J. "The Pyramid Exam," UME Trends, July, 1995, pp. 2 and 15.

[2] Levine, A. and Rosenstein, G. Discovering Calculus, McGraw-Hill, 1994.

[3] McGowen, M., and Ross, S., Contributed Talk, Fifth International Conference on Technology in Collegiate Mathematics, 1993.

[4] Ostebee, A., and Zorn, P. Calculus from Graphical, Numerical, and Symbolic Points of View, Volume 1, Saunders College Publishing, 1997.

Selected COTHE Questions

Problem A.

(a) Create a formula for a function that has all the following characteristics:

• exactly 1 root;
• at least 1 horizontal asymptote; and
• at least 3 vertical asymptotes.

(b) Using any method you like, draw a nice graph of the function. Your graph should clearly illustrate all the characteristics listed in part (a).

(c) You should be prepared to talk about direction and concavity of your function.

Problem B. Let defined on the domain [-2, 3] — that is, -2 x 3. Please notice that this is , h´(x), not h(x) !

(a) Sketch a graph of h(x) on the interval. (Note: there is more than one such sketch.)

(b) Locate and classify the critical points of h. Which values of x between -2 and 3 maximize or minimize h?

(c) For which value(s) of x does h have an inflection point?

Problem C. Let (where L is the average of the number of letters in your group's last names). The tangent line to the function g(x) at x = 2 is y = 7x - 4F, where F is the number of foreign languages being taken by your group. For which of the following functions can we determine the tangent line from the above information? If it is possible to determine it, do so. If there is not enough information to determine it, explain why not.

(a) A(x) = f(x) + g(x) at x = 2.

(b) B(x) = f(x) · g(x) at x = 2.

(c) C(x) = f(x)/g(x) at x = 2.

(d) D(x) = f(g(x)) at x = 2.

(e) E(x) = g(f(x)) at x = 2.

Problem D. Attached is an excerpt from Calculus from Graphical, Numerical, and Symbolic Points of View [4]. Read this excerpt and then use Newton's Method to find an approximate root for the function f(x) = x³ - Lx + 1 where L is the average of the number of letters in your group's last names. You should start with x₀ = 0, and should make a table which includes

x₀, f(x₀), f´(x₀), x₁, f(x₁), f´(x₁),…, x₅, f(x₅), f´(x₅),

You might also refer to Project 3.5 in your own Discovering Calculus [2].