 |
||
 |
 |
 |
 |
An entirely different approach to assessing the mathematics major has been developed at a state-supported, coeducational, liberal arts college in the Midsouth. Graduating seniors participate in focus group sessions which are held two days prior to graduation. These are informal sessions with a serious intent: to assess student learning in the major.
Background and Purpose
Mary Washington College is a state-supported, coeducational, predominantly undergraduate residential college of the liberal arts and sciences. It is located in Fredericksburg, Virginia, a historic city, which is about halfway between Washington, D. C. and Richmond, Virginia. The College is rated as "highly selective" in its admission status, and enrolls approximately 3000 undergraduates. There is also a small graduate program. There are ten full-time faculty members in the Department of Mathematics. On the average, 20 students graduate each year with degrees in mathematics.
Mary Washington College began to develop a program of outcomes assessment in 1989. The College wisely decided that each department, or major program within a department, should be responsible for developing its own plan to assess how well it was preparing its majors. By 1991, both faculty and administrators had learned more about outcomes assessment, and decided that assessment should be conducted according to a four-year cycle. Each major program has one faculty member, the "Outcomes Assessment Coordinator," who is responsible for the assessment.
Method
During the first semester of Assessment Year 1, faculty examine and revise, if necessary, their list of "Outcomes Expected." This list of goals and objectives details the essential knowledge, skills, and abilities that students who complete their major program should possess. Following this, faculty decide how they will determine the extent to which these outcomes are achieved by their major students. Faculty determine what methodologies and instruments will yield relevant data, and decide upon a timetable for these evaluation procedures. Over a four-year assessment cycle, each major program must collect data using at least one of each of the following: a direct measure (e.g. tests, capstone courses, portfolios), an indirect measure (e.g. focus groups, exit interviews), and a survey of program alumni. During the second semester of Assessment Year 1, and during Assessment Years 2 and 3, outcome assessment coordinators collect and analyze the data gathered through the various forms of assessment. Generally, it is expected that data will be collected during one semester and then analyzed and interpreted during the next semester.
During Assessment Year 4, assessment coordinators, along with other faculty, compile, analyze and interpret all of the findings of the past three years regarding their major program. Although changes in the major program which are based on assessment results can be proposed at any time during the assessment cycle, it is often helpful to wait until Year 4 when changes can be based on the cumulative results of various forms of assessment.
The 1996-97 academic year was Year 1 in the Mary Washington College assessment cycle. Departments have adjusted their schedules so that all are now on the same four-year cycle. This was the start of the mathematics departments' third cycle (although one cycle was three years rather than four). During the fall 1996 semester the Department of Mathematics extended and refined the outcomes expected of its majors. They are listed below.
Outcomes Expected
Expression of Mathematical Ideas
Critical Thinking
Discovery
Applications
Appreciation
During the past eight years the mathematics department has employed the following measures to assess its major program: mathematics tests, both alumni and faculty surveys, and focus groups with graduating seniors. During 1989, 1990 and 1995 we administered in-house assessment tests which asked the students to read, write and interpret mathematics. On each occasion, students enrolled in Calculus II (beginning-level "serious" mathematics students), and students enrolled in Real Analysis (senior level mathematics majors) took the test. Faculty compared and analyzed the answers given by the two groups of students.
In 1993, we conducted an alumni survey of recent mathematics graduates. The surveys included multiple-choice questions in which the alumni rated different aspects of their mathematics education as well as free response questions where students wrote suggestions and advice for improving the program. In 1994 we circulated a survey asking mathematics faculty for their comments on various aspects of the program. The outcomes assessment coordinator held individual comments in confidence and reported a summary to the department.
In 1992 and 1996, we conducted focus group meetings with graduating seniors. These focus groups were designed to provide information on how our majors perceived different aspects of the mathematics program and to elicit their opinions on ways the program might be improved. While all of the data have been useful, the focus groups of graduating seniors have been the most enlightening. On both occasions, we conducted the focus groups on the two days preceding graduation, after final grades had been submitted. Seniors knew that they could give their honest impressions of the program at this time without fear of negative repercussions. We invited all of our graduating seniors to attend one of two one-hour sessions. Nearly all of the seniors attended. We served drinks and snacks and there was a relaxed, congenial atmosphere; it was one last time for majors to get together. Two faculty members conducted the sessions. They took turns where one asked the questions and the other recorded. Neither faculty had taught these seniors in their upper-level courses so the students were free to make comments about the instruction in these classes. The faculty informed the students that the purpose of the session was to record the students' perceptions of their education in the mathematics program and asked that they be open and honest in their responses. Faculty distributed the list of questions to the students and then one of the faculty read aloud each question in turn and encouraged discussion. Examples of questions we asked included:
The faculty encouraged students to extend, agree or disagree with each other's remarks by asking questions such as "Do you all feel the same way?" or "Would anyone like to elaborate on that?" Everyone was encouraged to express their opinion. The students were very forthright, open and honest in their responses.
Findings
It was interesting to note that the data we obtained through the various types of assessment were remarkably consistent. The results of our assessments were very positive. However, we did find a few areas where our program could be strengthened. We were very gratified to hear our majors say that one of the strongest aspects of the program was that it had taught them to think and that they were confident in their abilities to solve problems. These, of course, are major goals of the program. The students were thankful for the small classes and personal attention. Seniors in the focus groups and alumni who were surveyed were very pleased with their preparation in mathematics in general. However, both groups stated that they would like to see more emphasis placed on the applications of mathematics.
Another recommendation of the focus groups was to further integrate technology into the program and encourage students to become familiar with computers. These students also recommended that we improve the department's career advising, and suggested that we integrate some group projects in our courses, since prospective employers seemed to value this kind of experience.
In addition to learning about our major program, we have also learned about the process of assessment. The cyclic approach to outcomes assessment seems to work very well. Within certain guidelines, major programs can choose what forms of assessment best meet their needs. Using a variety of assessment techniques helps departments see the broad picture and determine which findings are consistent across measures. Since assessment activities are spread over several years, they are not overwhelming.
The focus groups, in particular, seem to yield a large quantity of valuable information for the time and effort spent. Conducting the focus groups is quite enjoyable and rewarding for the faculty — assuming that you have a good program. The students enjoyed this chance to share their opinions with the faculty and in so doing to help maintain a strong mathematics program at their soon-to-be alma mater. Moreover, they were gratified that the faculty respected them enough to care about their opinions.
Use of Findings
The results from the focus group, along with the results from our other assessment activities, helped the mathematics department determine how well we were meeting our goals and plan actions to maintain a strong major program. Partly in response to the recommendation to place more emphasis on mathematical applications, we hired two new faculty whose areas are in applied mathematics and we strengthened the program offerings in these areas. These faculty have allowed us to offer more sections of our "applied" courses and they have also developed and taught new upper-level courses in Chaos, and Linear Models. They also have developed a new sophomore-level course entitled "Mathematical Modeling," an interdisciplinary course in basic mathematical modeling investigating various scientific models with an environmental theme. These faculty have both worked with students on independent studies in various applied fields, and have sponsored several students in internships. In addition to hiring these new faculty, we urged all faculty members to integrate more applications of mathematics into their courses wherever possible. These modifications should help students meet the outcomes expected in the application area.
We have further integrated technology into our program by using graphing calculators in all of the Precalculus and Calculus courses and by using computers in the Statistics, Differential Equations and Linear Algebra courses, as well as in some Special Topics courses. Also, we now strongly encourage students to take computer science courses to complement their program in mathematics.
In response to improving career advising, we have hosted professionals from the area to speak to students and we now have a "Careers Information Link" on the mathematics department's home page where students can find general information on careers for mathematics majors, internship possibilities, and specific information about organizations which are currently hiring.
Also in response to focus group recommendations, we have improved placement in freshmen courses, fought to keep class sizes as small as possible to ensure quality instruction, and have integrated group work and projects into existing coursework.
The results from the senior focus groups were so informative that the mathematics department is currently planning to conduct alumni focus groups. We will invite recent alumni in this area to meet with us to seek their perceptions of how well our mathematics program prepared them. We trust that we will get the same in-depth, thoughtful responses that we did from our senior majors. We then plan to use this information to help us write an alumni survey for those mathematics graduates who could not attend the focus groups.
Success Factors
We have learned much from the results of our various forms of outcomes assessment. We understand, however, that we must be cautious in interpreting the results of assessment. For example, the responses in both the focus groups and on alumni surveys were their perceptions of the program and we viewed them as such. A few of these perceptions were factually incorrect, such as on issues related to course scheduling. Student and faculty opinions did not always agree, such as on the value of certain courses and major requirements. However, this does not negate the importance of students' perceptions and opinions.
We believe that our focus groups have been
successful due, in part, to precautions that we took in planning
and
conducting the focus groups. We scheduled the
group sessions for a convenient time after grades had been
turned in. Students signed up during their classes for one of
the sessions. The day before their session, the
department secretary called with a reminder. We selected faculty
with whom the students could be open and honest and
chose questions which were open-ended and invited
discussion. While conducting the sessions, faculty waited patiently
for responses and accepted the students' opinions without
being defensive toward negative comments. Faculty
encouraged the students to respond to and extend each others'
responses. This was the students' turn to talk and the faculty's turn
to listen.
|
||
|
|
|
|