 |
||
 |
 |
 |
 |
This assessment program stresses breadth — pre- and post-testing, journals, comparative test questions, student interviews and questionnaires, and more.
Background and Purpose
Dickinson College is a four-year, residential, liberal arts institution serving approximately 1,800 students. During the past decade, the introductory science and mathematics courses at the college have been redesigned to emphasize questioning and exploration rather than passive learning and memorization. The Workshop Calculus project1, now in its sixth year of development, is part of this college-wide effort.
Workshop Calculus is a two-semester sequence that integrates a review of pre-calculus concepts with the study of fundamental ideas encountered in Calculus I: functions, limits, derivatives, integrals and an introduction to integration techniques. The course provides students who have had three to four years of high school mathematics, but who are not prepared to enter Calculus I, with an alternate entry point into the study of higher-level mathematics. It seeks to help these students, who might otherwise fall through the cracks, develop the confidence, understanding and skills necessary to use calculus in the natural and social sciences and to continue their study of mathematics. After completing both semesters of Workshop Calculus, workshop students join their peers who have completed a one-semester Calculus I course, in Calculus II.
All entering Dickinson students who plan to take calculus are required to take the MAA Calculus Readiness Exam. Students who score below 50% on this exam are placed in Workshop Calculus, while the others enter Calculus I. The two strands serve the same clientele: students who plan to major in mathematics, physics, economics or other calculus-based disciplines. While both courses meet 5 hours a week, Calculus I has 3 hours of lecture and 2 hours of laboratory sessions, while there is no distinction between classroom and laboratory activities in Workshop Calculus, which is primarily hands-on.
A forerunner to Workshop Calculus consisting of a review of pre-calculus followed by a slower-paced version of calculus was developed in response to the fact that 30-40% percent of the students enrolled in Calculus I were having difficulty with the material, even though they appeared to have had adequate high school preparation. Students' reactions to the course, which was lecture-based with an emphasis on problem solving, were not positive. They did not enjoy the course, their pre-calculus skills did not improve and only a few ventured on to Calculus II. In addition, colleagues in client departments, especially economics and physics, continued to grumble about the difficulty students had using calculus in their courses. Workshop Calculus was the department's response.
With the Workshop approach, students learn by doing and by reflecting on what they have done. Lectures are replaced by interactive teaching, where instructors try not to discuss an idea until after students have had an opportunity to think about it. In a typical Workshop class, the instructor introduces students to new ideas in a brief, intuitive way, without giving any formal definitions or specific examples. Students then work collaboratively on tasks in their Workshop Calculus Student Activity Guide [1]. The tasks in this manual are learner-centered, including computer tasks, written exercises and classroom activities designed to help students think like mathematicians — to make observations and connections, ask questions, explore, guess, learn from their errors, share ideas, and read, write and talk mathematics. As they work on assigned tasks, the instructor mingles with them, guiding their discussions, posing additional questions, and responding to their queries. If a student is having difficulty, the instructor asks the student to explain what he or she is trying to do and then responds using the student's approach, trying not to fall into let-me-show-you-how-to-do-this mode. The instructor lets — even encourages — students to struggle, giving only enough guidance to help them overcome their immediate difficulty. After completing the assigned activities, students participate in class discussions, where they reflect on their own experiences. At this point, the instructor can summarize what has been happening, present other theoretical material or give a mini-lecture.
Method
Assessment activities are a fundamental part of the Workshop Calculus project. With the help of external collaborators2, we have analyzed student attitudes and learning gains, observed gender differences, collected retention data and examined performance in subsequent classes. This information was used to help make clearer the tasks expected of students laid out in our Study Guide. More significantly, it has provided the program with documented credibility, which has helped the Workshop program gain the support of colleagues, administrators, outside funding agencies, and even the students themselves. The following describes some of the tools we have used. For a more in-depth description of these assessment tools and a summary of some of the results, see [2].
a. Collecting Baseline Data. During the year prior to introducing Workshop Calculus, baseline information was collected concerning students' understanding of basic calculus concepts. Workshop Calculus students were asked similar questions after the new course was implemented. For example, students in the course prior to the Workshop version were asked:
In response to the first question, 25% of the students stated that the derivative represented the slope of the tangent line, and half of these students used this fact to give a reasonable explanation to the second question. The remaining students answered the first question by giving an example and either left the second question blank or wrote a statement that had little relationship to the question; they could manipulate symbols, but they didn't understand the concepts.
This data showed the need to emphasize conceptual understanding of fundamental concepts and to have the students write about these ideas. Consequently, Workshop students learn what an idea is — for example, what a derivative is and when it is useful — before they learn how to do it — in this case, the rules of differentiation — and they routinely write about their observations.
b. Administering Pre- and Post-tests. Workshop Calculus students answer questions prior to undertaking particular activities and then are re-asked the questions later. For example, on the first day of class, students are asked to write a short paragraph describing what a function is, without giving an example. Although they all claim to have studied functions in high school, many write gibberish, some leave the question blank, and only a few of students each year describe a function as a process. After completing the activities in the first unit of their activity guide (where they do tasks designed to help them understand what a functions is, without being given a formal definition of "function"), nearly 80% give correct, insightful descriptions of the concept of function.
c. Analyzing Journal Entries. At the end of each of the ten units in the Workshop Calculus activity guide, students are asked to write a journal entry, addressing the following questions:
The students' responses provide the instructor with valuable insight into their level of understanding; their candid replies also provide important feedback about what works and doesn't work and about changes that might need to be made.
d. Asking Comparative Test Questions. Student performance in Workshop Calculus has been compared to students at other institutions. For instance, on the final exam for Workshop Calculus, in the spring of 1994, students were asked four questions from an examination developed by Howard Penn at the US Naval Academy to assess the effectiveness of using the Harvard materials at the USNA versus using a traditional lecture-based approach [3]. (See the Appendix for these questions.) We are pleased to report that the Workshop Calculus students did about as well as the USNA students who were using the Harvard materials, even though the Academy is certainly more selective than Dickinson College.
e. Conducting Critical Interviews. A representative group of Workshop students have been interviewed in structured ways, to help determine what they are thinking as they work on a given problem and to determine their level of understanding of a particular concept. For this, a questionnaire is administered to students, and we categorize responses and approaches used, even taping one-on-one "critical interviews" with a representative group of students, transcribing the interview sessions, and analyzing the results. After using the first version of the Workshop Calculus materials in the spring of 1992, this approach was used, for instance, to analyze students' understanding of "definite integral." After analyzing students' responses on the questionnaire and during the interview sessions, we realized that some students could only think about a definite integral in terms of finding the area under a curve and had difficulty generalizing.3 Based on this observation, the tasks pertaining to the development of definite integral in the Student Activity Guide were revised.
f. Scrutinizing the End-of-Semester Attitudinal Questionnaire. At the end of each semester, students are asked to rate the effectiveness of various activities, such as completing tasks in their activity guide, participating in discussions with peers, using a computer algebra system. They are also asked to rate their gains in understanding of fundamental ideas — such as their understanding of a function, or the relationship between derivatives and antiderivatives — to compare how they felt about mathematics before the course with how they feel after completing the course, and to describe the most and least useful activities in helping them learn mathematics.
Student responses are analyzed for gender differences. For instance, both male and female students who took Workshop Calculus in 1993-1994, claimed, on the average, that they felt better about using computers after completing the course than before. Men showed a greater increase in confidence, however, and even after the course, women were not as comfortable using computers as the men were initially.
g. Gathering Follow-up Data. Information about student attitudes is also collected from students who took either the Workshop Calculus sequence or the regular Calculus I course, one or two years after they complete the course (irrespective of whether they have gone ahead with mathematics or not).4 Their responses are used to determine the impact of the course on their attitudes towards mathematics and their feelings about the usefulness and applicability of calculus in any follow-up courses. For instance, students are asked whether they strongly agree, somewhat agree, somewhat disagree, or strongly disagree to 46 statements, including:
In addition, we gather data about Workshop Calculus students who continue their study of mathematics and/or take a course outside the Mathematics Department that has calculus as a pre-requisite, asking questions such as:
Findings and Use of Findings
With the exception of the critical interviews, the assessment tools used for Workshop Calculus are easy to administer and the data can be easily analyzed. Moreover, these tools can be used in a variety of courses at a variety of institutions. In general, our assessment process requires developing a clear statement of intended outcomes, designing and utilizing supportive assessment tools, and analyzing the data and summarizing the results in ways that can be easily and readily understood (for instance, bar charts or graphs are helpful).
Success Factors
Something should be said here for the advantages of using a variety of assessment measures. We feel that our program benefits from the breadth of our methods for collecting information about performance, teaching, learning, and the learning environment. In particular our focus is on understanding how students are thinking, learning, and processing the courses. Often assessment measures raise more questions than they answer, and when students are asked questions, they may be reading and responding on different wavelengths from those we are broadcasting on. We have found that an advantage to using a "prismatic" assessment lens is that we obtain a variety of ways of exploring issues with students, and therefore we become closer to understanding students and more inspired to make the changes that will be beneficial to all of us. Good assessment offers students feedback on what they themselves report. We feel that our methods are appreciated by students, and that the department is perceived as caring as reforms are instituted. Good assessment also promotes ongoing discussion. And our measures have certainly helped to stimulate ongoing faculty dialogue, while unifying the department on the need for further assessment. It has been six years since the Workshop Calculus project began and we are still learning!
References
[1] Hastings, N.B. and Laws, P. Workshop Calculus: Guided Exploration with Review, Springer-Verlag, New York, vol. 1, 1996; vol. 2, 1998.
[2] Hastings, N.B. "The Workshop Mathematics Program: Abandoning Lectures," in D'Avanzo, C. and McNeal, A., Student-Active Science: Models of Innovation in College Science Teaching, Saunders Publishing Co., Philadelphia, 1997.
[3] Penn, H., "Comparisons of Test Scores in Calculus I at the Naval Academy," in Focus on Calculus, A Newsletter for the Calculus Consortium Based at Harvard University, 6, Spring 1994, John Wiley & Sons, Inc., p. 6.
Appendix
Q1 showed five graphs and asked which graph had f´(x) < 0 and f´´(x) < 0.
Q2 showed the graph of a function h and asked at which of five labeled points on the graph of h is h´(x) the greatest.
Q3 showed the graph of a function which was essentially comprised of two line segments—from P(-1,0) to Q(0,-1) and from Q(0,-1) to R(3,2)—and asked to approximate the integral of the function from -1 to 3.
Q4 showed the graph of the derivative of a function, where f´ was strictly positive, and asked at which of five labeled points on the graph of f´ is f(x) the greatest.
——————
1 The Workshop Calculus project has received support from the Knight Foundation, the US Department of Education Fund for Improvement of Postsecondary Education (FIPSE) and the National Science Foundation (NSF).
2 Ed Dubinsky, from Georgia State University, and Jack Bookman, from Duke University, helped assess student learning gains and attitudes in Workshop Calculus. They were funded by FIPSE.
3 Ed Dubinsky helped design the questionnaire and analyze the results.
4 Items on the follow-up questionnaire were developed by Jack Bookman, Project Calc, Duke University.
|
||
|
|
|
|