 |
||
 |
 |
 |
 |
At a small, private men's liberal arts college in the Midwest, a comprehensive examination, known as comps, has been a tradition for seventy years. It has evolved into the assessment technique which the department uses to assess student learning. Comps are taken by seniors over a two-day period just prior to the start of the spring semester. The exam consists of two parts: a written component in the mathematics major and an oral component over the liberal arts.
Background and Purpose
Wabash College is a small (850 student), non-religious private college for men, founded in 1832 in a small town in Indiana. The department of mathematics and computer science, with 7 full-time faculty members, offers a major and a minor in mathematics, and a minor in computer science. We graduate between 2 and 18 mathematics majors per year, many of them double majors with the other major in a field which uses mathematics heavily. In order to graduate, every student must pass a comprehensive examination (often called "comps"). This requirement has been part of the college curriculum since 1928. This examination consists of two parts: (1) a written component in the major, two days, three hours per day, and (2) an oral component over the liberal arts. The purposes of the written part of the examination include (1) having students review their courses in their major so that they will become aware of connections which may have been missed in individual courses and (2) satisfying the faculty that the graduating seniors have an acceptable level of competence in their major. The content of the written component is left to each department to put together as it pleases. The purpose of the oral component is to cause students to reflect on their whole liberal arts experience: how they have examined, and perhaps changed, their values and beliefs during their time here; what were the positive and negative aspects of their Wabash education; how they may begin to fit into the world beyond Wabash; what directions they still need to grow in and how they can continue to grow after they leave. The oral is a 50-minute exam with each student examined by three faculty members, one from the major, one from the minor, and one at-large. Thus, the major is also examined in this component. It provides for the department very different information from that provided by the written part, as will be discussed below. Because we have been giving these examinations for many years, there has been a lot of opportunity to reflect on and change the curriculum in response to weaknesses we find in our seniors on comps.
Method
The written part of the comprehensive examination is given two days before the start of the spring semester. The mathematics exam is always given in the afternoons, while most other disciplines give theirs in the mornings. Thus, mathematics majors who are double majors (many are) can take two sets of exams during the two days. In November, the department discusses exactly what format that year's comps will take. There is always an essay or two (recently over post-calculus core courses, e.g., abstract algebra and real variables), as well as problems over the calculus sequence, on the first day, and advanced topics on the second day. However, the number of essays, how the topics are selected, exactly where the dividing line between the calculus sequence and advanced topics will be, and whether students are allowed aids such as calculators or textbooks, changes from year to year based on changes we have been making in the courses students take. For example, recently we decided that we were interested in how well students understand calculus concepts, not in how well they remembered formulas they had learned three years earlier. So we allowed them to bring texts or a sheet of formulas to the first day's exam. We recently changed from requiring all students to take both real variables and abstract algebra for the major to requiring only one of these courses. Therefore this year's exam only had one essay question, a choice between a topic in algebra and a topic in analysis. However, beginning with next year's class, all students will have taken a full semester of linear algebra as sophomores, and so we will then be able to require everyone to write an essay on a topic from linear algebra and one from either real variables or abstract algebra.
The essay topics are rather broad; from algebra: algebraic substructures or homomorphisms of algebraic structures; from real variables: differentiability in R and Rn, or normed vector spaces. The instructions say: write an essay which includes definitions of appropriate terms, relevant examples and counter-examples, statements of several important theorems, and the proof in detail (including proofs of lemmas used) of at least one of these theorems. Students are given a short list of potential topics in advance, and are told that the topics they will have to write on will be chosen from among these. They are encouraged to prepare these essays in advance, and even speak with members of the department about what they're planning to include. However, they write the essay without notes on the day of the exam. We're looking for their ability to choose appropriate examples and theorems, to explain them, and to put this all together into a coherent whole.
The non-essay portions of the exam are typical course test problems. Whoever has taught a given course within the last two years writes three problems over that course, and gives them to the exam committee (which consists of two people). The committee then takes all the submitted questions and chooses which to use. They try to have a good balance of easy and difficult problems. Since over the course of their four years the students have had a wide choice of elective courses, the exam committee must make sure that the second day's examination on advanced topics has a sufficient balance of problems to enable each student to choose problems from courses he has studied. The problems are not supposed to require memorization of too much information or otherwise be too obscure; rather, they should cover the main points of the courses. A few sample questions:
From calculus: Find a cubic polynomial g(x) = ax3 + bx2 + cx + d that has a local maximun at (0,2) and a local minimum at (5,0).
From number theory: Find all solutions in positive integers for 123x + 360y = 99.
Students are given identification numbers (the correspondence between names and ID numbers is sealed until the written exams have been graded and grades decided on), and all questions are double-graded, with the department sharing equally the job of grading. When there is a substantial disagreement between the two graders of a given question, they look over all the papers which answer that question and regrade that question together. The college requires that the oral component comprise 1/4 of the grade, but that, to pass the comprehensive examination, students must pass both the oral and written parts separately.
Each student has a different oral examination committee. All examiners are to examine the student over the liberal arts generally, but usually each one has about 15 minutes during which (s)he has principal responsibility for the questions (but the others are encouraged to pursue the student for more details if they feel the response is inadequate). Generally, the questions mathematicians ask of students who are mathematics majors are questions which would not have been asked in the written portion. They involve philosophical questions ("is mathematics a science or an art"), or test the student's ability to speak about mathematics to non-mathematicians ("please explain to Professor Day, who teaches classics, the idea of the derivative"). While we have several other methods of assessing the major, the oral exam is the only assessment tool for the minor (other than examinations in individual courses), and students with mathematics minors are often asked questions about the relation between mathematics and their chosen major (usually a physical or social science).
Findings
When I first came to Wabash, the comprehensive exams were often a source of considerable discomfort to the department. When asked to explain what the derivative was, mathematics majors would often start (and end!) by explaining that the derivative of xn was nxn-1. Students' grades, even those of good students, on the written comprehensives were often poor. Many majors have advisors outside of mathematics, and in the past, some accumulated a number of D's in mathematics courses. They then came to the comprehensive examination unable to pass. It's too late, in the second semester of the senior year, to tell a student he'd better get serious or find another major. We have made numerous changes in the program due to these results, and have also started failing students when they do sufficiently poorly (below 60% overall). When this happens, the department must write a new examination and the student must take the comps again in April in order to be able to graduate with his class. If he fails again, then he has to wait until the following year's exams. This has made most students take the exams fairly seriously. (About 10 years ago, 4 students failed on the first try, none of whom had done any serious studying. All passed on the second attempt. In the last 5 years, we've only had to fail one student.) At the moment, when students have studied for the exams, the results largely are consistent with the department's view of the student: good students (usually 1 or 2 each year) — heading to graduate school or promising careers — write exams which merit distinctions (90% or better, approximately), mediocre students (roughly 1/3) barely pass, and middle level students — the remainder — either do a good pass or a "high pass" (80% to 90%). Mathematics majors are now graduating with a better understanding of modern mathematics than in the past. The performance of minors and other students on the oral portion, however, is still causing some changes in the lower level curriculum.
Use of Findings
Over the years, having obtained unsatisfactory results in comprehensive exams, we have made many changes in departmental offerings and requirements. We started requiring abstract algebra and real variables when it became clear (from oral exam answers like the one mentioned above) that students were not getting a sufficiently rigorous theoretical background in mathematics. We started trying calculus reform when we saw how little students remembered from their calculus experience, and the minimal understanding demonstrated on the oral exam by minors and others who had not gone further. These changes in turn led to other changes, such as the introduction of a required, theorem-based linear algebra course in the sophomore year. On the other hand, because we want to allow a variety of electives, having added the linear algebra course, we decided to allow students to choose between real variables and abstract algebra, while encouraging them to take both. We have also moved to more active learning in the classroom, to help students become better able to handle the material. Students are learning more: we are graduating fewer students that we are embarrassed to acknowledge as mathematics majors than we had in the past.
To deal with the problem of students having advisors outside mathematics and surprising us by taking and failing comps, we adapted St. Olaf College's contract major. Each student signs a contract with the department describing what direction he plans his major to take (towards actuarial work, applied mathematics, pure mathematics, etc.). The contract is signed before registration for the second semester of the sophomore year, when students have their first choice of mathematics courses (between multivariable calculus and differential equations, though both are recommended). Each contract has some flexibility within it, and by petitioning the department, students can switch contracts if they find that their interests change. But by requiring them to sign a contract as sophomores, students begin working with the department much earlier in their student careers, when it is still possible to prod the student to the required level of effort.
Success Factors
Our capstone course, which involves readings in history of mathematics and preparation of a senior expository paper, is taken in the semester just before the comprehensive examination. The combination of a course spent reflecting on the history of mathematics together with the comprehensive examination for which the students review the courses they have taken helps students place the individual courses in perspective. Toward the end of the capstone course students are given a copy of the previous year's examinaton and there is some opportunity to discuss the upcoming comps.
The faculty meets weekly over lunch to discuss departmental business. Comps help focus our discussions, both before writing the test in the fall and after finishing grading them in the spring. In the fall we think about the progress our seniors have made, and what their focuses are. In the spring, we think about what changes we need to make in the program due to weaknesses which show up on the students' examinations.
The timing of the written comps has changed over the years. A long time ago, they were in April. However, if a student failed the examination he would have to wait until the following year to retake it and graduate. So they were moved to late January, during classes. Then, because seniors not only missed two days of classes, but often a whole week (a few days before to study, and a day or so after to celebrate), the written part of the comps was moved to the first two days of the spring semester. This way, grades are turned in by spring break, and any students who fail can retake them once in April and thus have a chance to graduate with their class.
We have had to fail students every several years on the exams; otherwise, some students do not take them seriously. However, since we switched to the contract major, students who have failed the written part have admitted that they didn't put much effort into studying for the exam. Nonetheless, both because of the additional trouble it causes the department, and because sometimes we're not convinced that a given student is capable of performing better on a second attempt, there is a contingent of the department which opposes failing anyone. It is always painful to fail our students, both because it's not clear whether we failed them or they failed us, and because of the additional time and suffering it causes us all. However, having established a history of doing this, our students in the last few years have taken the examinations fairly seriously and the results seem to reflect well what they have learned.
Alumni remember their comprehensive exams,
especially the oral portion, vividly, particularly the composition of
the committee and the questions they struggled with. They
view the process as an important rite of passage, and one of
the hallowed traditions of the college.
|
||
|
|
|
|