 |
||
 |
 |
 |
 |
In an evolving assessment program at a private, medium-sized, comprehensive university in the Midwest a variety of assessment techniques are being developed to assess student learning. How two of them — exit interviews and the Educational Testing Service's Major Field Test in Mathematics — are part of the fabric of the mathematics program's assessment cycle is described in this article.
Background and Purpose
Xavier University is a Catholic, Jesuit institution having approximately 2800 full-time undergraduates and about 1,100 part-time undergraduates. The Department of Mathematics and Computer Science is housed in the College of Arts and Sciences and has fourteen full-time members and six part-time members. All but three teach mathematics exclusively. There are about 45 mathematics majors with an average of ten graduates each year. The mathematics major at Xavier must complete 42 semester hours (thirteen courses) of mathematics, which include three semesters of calculus, differential equations, two semesters of linear algebra, abstract algebra, real analysis, and four upper division electives.
In the fall of 1994, each department received a request from the office of the academic vice president to devise plans to assess programs for its majors by March of 1995. Each plan was to cover the following topics:
Although we have a bachelor's program for computer science in the Department, this paper will only address assessing the program for mathematics majors at Xavier University.
According to the mission statement of the University, the primary mission of the University is to educate, and the essential activity at the University is the interaction of students and faculty in an educational experience characterized by critical thinking and articulate expression with special attention given to ethical issues and values. Moreover, Jesuit education is committed to providing students with a supportive learning environment, addressing personal needs, and developing career goals along with the academic curriculum.
Method
Our assessment plan records students' growth and maturity in mathematics from the time they enter the program until they graduate. The plan was submitted in the spring of 1995 and put into place during the school year 1995-96. We chose the following methods for assessing the major:
During the first few weeks of the fall semester, freshman mathematics majors receive letters stating the expectations for majors for the coming years. In particular, they are informed that they must maintain a portfolio, make an oral presentation during their senior year, and meet our standards for satisfactory performance on the MFT (Major Field Test) in Mathematics.
Portfolio: Portfolios are maintained by faculty advisors. Each semester the student must submit final examinations and samples of their best work from each mathematics class. A copy of any research or expository paper in an upper level class should also be included. Hopefully, these works will provide evidence of how well the students are learning to synthesize and integrate knowledge from different courses in the program. Moreover, the samples should indicate that a student can use various approaches to solve problems. Samples of work will be collected for the portfolio by advisors during formal advising periods.
A number of additional documents are standard. These include an entry profile, an entry questionnaire, counseling forms, and advisor notes. The entry profile contains information on a student prior to enrolling at the University, such as the mathematics courses taken in high school, grade point average and rank in graduating class, SAT and/or ACT scores, and advanced placement (AP) credits. It also contains the placement scores from the examinations in mathematics and foreign language which are administered by the University. The entrance questionnaire, a short questionnaire given in the fall term, reveals the student's perception of mathematics and his expectations of the program and faculty. (See [1] in this volume for further information on student portfolios).
MFT: The Department has given some form of a comprehensive exam for many years. The Advanced Test in Mathematics of the GRE served as the last measure until the 1994-1995 academic year. It was replaced by the Major Field Test (MFT) in Mathematics, which is distributed and scored by the Educational Testing Service. The profiles of the students taking the MFT are a far better match with the profiles of our students than with those taking the GRE. Moreover, not only are individual and mean scores reported, but also subscores which help point out the strengths and weaknesses of the examinees in broad areas within mathematics.
The Department has set as successful performance a score at the 60% mark or higher. However, this mark in not rigidly enforced. The Department undertakes an extensive study of the individual case whenever this goal is not met. Then a recommendation is made whether to pass the student or ask the student to retake the exam. A student may request a departmental comprehensive examination after two unsuccessful performances on the MFT. If a student's score on this exam is also unsatisfactory, the student may be advised to seek a bachelor's degree in liberal arts. A degree in liberal arts would necessitate no additional requirements to be fulfilled for graduation.
We expect very few students, if any, to find themselves in the position of having failed the MFT twice. Thus far, no one has been unsuccessful on their second attempt at making a passing score. During the fall semester the Department provides review sessions for seniors preparing for the MFT. The whole experience of the review sessions and exam-taking serves as a mechanism for students to rethink and synthesize mathematical concepts as well as to perfect skills that were covered in their mathematics courses.
Senior Presentation: Prior to our assessment plan, students rarely participated in departmental colloquia, but will now assume an integral and vital part. Beginning in the 1998-99 academic year each senior will be required to make an oral presentation. The topic for the senior presentation will be selected by the student with the approval of the advisor for senior presentations. The advisor will provide each senior with a list of steps that are helpful in preparing a talk. After a student chooses a topic, she must submit a written draft for a thirty-minute lecture. The advisor will provide feedback on the draft and set up a practice session for the student.
Questionnaires: Questionnaires and surveys are critical elements of the assessment process. Exit questionnaires are sent to graduating seniors about three weeks prior to the end of the spring semester and exit interviews are scheduled. An alumni survey will be administered every five years. This will help us track the placement of students in various jobs and professional programs. Between the years scheduled for alumni surveys, alumni questionnaires are mailed to graduates one year after graduation. These forms are much shorter and do not cover topics in the breadth and depth as does the alumni survey.
Findings
We have completed two years with the new assessment plan. The first steps proceeded fairly well. The freshman advisors created portfolios for the freshmen, the chair passed out entry profiles, and advisors collected samples of work for the portfolios throughout the year. There were very few freshmen in the second year and thus the job of the creating portfolios was easier, but that of recognizing a class profile was more difficult.
Although our experience with portfolios is very limited, portfolios have been useful. Not only is there an evolving picture of mathematical growth and progress, but also evidence of the content, level of difficulty, and teaching philosophy in the mathematics courses for majors. They provide information on the manner in which a course is taught which may not be obtained from course evaluations. Thus, a more thorough understanding of the overall curriculum is likely as more students and courses are tracked.
In 1996, seniors took the MFT exam and each met the departmental goal. In 1977 two of ten seniors did not meet the goal. They subsequently retook the exam and passed.
Although seniors are not yet required to make formal presentations, there have been presentations by the juniors and seniors in the departmental colloquium series for the last two years. Several students who participated in the 1996-97 Mathematics Modeling Competition gave talks which were very well attended by mathematics and computer science majors. Freshmen have been strongly encouraged and sometimes required to attend the colloquia.
Almost all majors in the classes of 1996 and 1997 returned their exit questionnaires and had an exit interview. The data from both 1996 and 1997 were similar and consistent with the information gathered in our alumni survey, which was carried out in the fall of 1995. Although these students and alumni have overall favorable impressions of the program and faculty, they raised issues that must be addressed. In particular, some felt that there should be more electives and fewer required courses, while others suggested that specific courses be added to or removed from the curriculum. A few thought that there should be a capstone course. Comments about the discrete mathematics course were generally negative and the merits of particular electives repeatedly surfaced.
There was substantial feedback on instructors, the use of technology in courses, and the availability of resources (particularly computers) in the exit questionnaires. In particular it was remarkably clear whom students classified as the very good and the not-so-good instructors and the reasons for their choices. Experiences with MAPLE, the computer algebra system employed in the calculus and linear algebra sequences during the first two years, drew mixed reviews. Students in Calculus III liked using MAPLE; a substantial number in Calculus I and Linear Algebra did not. Moreover, with the increasing use of computer labs on campus, accessibility to computer equipment is an issue that must be continually addressed.
The questionnaires and alumni survey also indicated a need for more counseling about career options. Prior to these results, most advisors believed that students were aware of the career opportunities that are available to them. This has proved to be false. Some students professed little knowledge about career opportunities and some had little idea of what they would do after graduation.
Use of Findings
The Department is responding to the issues that have surfaced from assessing the mathematics major. There are three main areas of focus. The one that has prompted the most scrutiny is the curriculum for the mathematics major itself. Some alumni who are currently teaching secondary mathematics remarked on the value of the elective Survey of Geometry that had been offered, but was dropped several years ago when the list of electives was streamlined. Some alumni and seniors suggested that statistics should be a requirement for all mathematics majors. In addition, many students questioned the intent of discrete mathematics and its contribution to their overall development. As a result of these findings, the Department will first thoroughly review the content in the discrete mathematics and the computer science courses in the freshman year. Then we will examine the cycle of electives. We are open to revising any aspect of the curriculum.
The use of MAPLE is the second area of concern. When there was a very heavy emphasis on MAPLE in a course, there was a considerable number of negative comments. In such cases, seniors clearly expressed the need for more balance in the use of technology. Many felt that the intense use of MAPLE left too little time for a deep understanding of basic concepts. This message was communicated to all departmental members, especially those named in the exit interviews and questionnaires. Moreover, the expansion of the use of MAPLE outside the calculus sequence has created a few problems of accessibility of computers.
The area of counseling is the third area of concern. To help students make better decisions about courses and career options, we are giving students more information early. In particular, during preregistration in the summer, the "fact sheet" on the mathematics program is given to all incoming majors. This brochure is put together jointly by the Department and the Office of Admissions (and revised annually). It contains the goals of the program, a description of the mathematics major, a recommended sequence of courses for the mathematics major, and information about careers in mathematics. In the fall semester, a packet containing additional information about careers from other sources will be given to freshmen majors.
It is quite apparent from the exit questionnaires and interviews which faculty members students perceive to be the best professors in the Department and the reasons for the acclaim. It is just as apparent which courses are the most difficult. Such information provides the chair with valuable information when composing a schedule of classes. The chair will speak to the individual faculty members whenever there is feedback which merits special attention.
The exit questionnaires are available to all members of the Department. Information in summary form is also made available to the Department and discussed in a department meeting. Thus, the Department is kept apprised of students' reactions.
Success Factors
Trends, problems, and successes have been fairly evident using data acquired from the groups of students with similar experiences at approximately the same time. Unfortunately, we have no means of requiring students to participate in the interviews and questionnaires, which are key elements of assessing the program for majors. Of course, students must take the MFT in Mathematics and make a senior presentation. These are now printed in the current University catalog. We hope to impress upon students the importance of their participation in the entire assessment process.
Assessing the program for majors is an ongoing project that will demand a commitment of time and energy from many department members. It is not difficult to foresee how the enthusiasm of some may wane and how those upon whom most of the work falls may become disgruntled. Because there are many aspects of the plan to monitor, it would be easy to let one or more measures slide from time to time. How the next chair will respond to the tasks is also an unknown. Hopefully, the Department will continue to assess the mathematics major beyond the "first steps" with the interest and energy that currently exist.
References
[1] Sons, L. "Portfolio Assessment of the Major," in
this volume, p. 24.
|
||
|
|
|
|