 |
||
 |
 |
 |
 |
The focus of this article is on assessing student learning for a segment of the undergraduate mathematics majors: those talented students who are prospective mathematics graduate students. At this private, liberal arts college in the Northeast there are three aspects of the undergraduate mathematics experience outside the standard curriculum which are described in this paper: a senior seminar (required of all majors), an Honors thesis and a summer undergraduate research experience. All three combined are used to assess how well the department is preparing students for graduate school.
Background and Purpose
Williams College is a small liberal arts college in rural Massachusetts with an enrollment of 2000 students. The Mathematics Department has 13 faculty members covering about 9.5 FTE's. Williams students are bright and very well prepared: the median mathematics SAT score is 700. For the past 7 years, annual mathematics enrollments have averaged about 1050, about 30 students have majored in mathematics from each class of 500, an average of 3 or 4 students have gone on to graduate school in mathematics or statistics, and about half to two-thirds of those have completed or are making progress toward the PhD.
Three aspects of the mathematics experience outside the standard curriculum at Williams are particularly relevant to students considering graduate school: the Senior Colloquium, the honors thesis, and the SMALL Undergraduate Summer Research Project. The colloquium and honors thesis lie within the formal structure of the major; both have existed for decades. The SMALL project is a separate program which began in 1988. These three experiences help cultivate in students some of the qualities necessary for success in graduate school such as independent learning, creative thinking on open problems, and the ability to present mathematics well orally and in writing. How well our students respond to these undertakings in part measures how well the rest of the major prepares them for graduate work. Which students choose to do an honors theses or afpply to SMALL is another measure of how appropriately we inspire our students to consider graduate school.
The Senior Colloquium and Honors Program have existed at Williams for many years. The former is required of all majors, a shared experience with many challenges and benefits. The latter is a special opportunity appropriate only for the most able and ambitious students. Each is a barometer in some obvious ways of how well we prepare students for graduate school. The ability of students to prepare clear and engaging talks on demanding topics they research with little faculty assistance is a sign of good preparation, not only for graduate school but for many other undertakings. Students who produce interesting results in their theses, written and presented orally in mature mathematical style, further demonstrate that we are helping their development as mathematicians. Likewise the performance of students in the SMALL project is an obvious preparation indicator.
Method
The Senior Colloquium is a colloquium series that meets twice a week throughout the school year. It is advertised and open to the entire community. The talks are given mainly by seniors, with occasional outside speakers, to an audience of mostly mathematics majors and faculty. Presentations last 30-40 minutes, followed by soda, cookies, and informal discussion. Senior majors are required to give one talk each, prepared under the guidance of a faculty member on a subject they have not studied in a class. After a few minutes of mingling with students, faculty members evaluate the talk together on a pass/fail basis. This private discussion lasts only a few minutes. The advisor verbally conveys the result to the student immediately, along with comments on the presentation. In an average year at most one or two failures occur. Seniors who fail the talk are usually required to give a second talk on a new topic. Seniors must also meet an attendance requirement to pass the Colloquium. Students do not receive formal credit for the Senior Colloquium, but it is a requirement for the major.
Evaluation of the Senior Colloquium from the student point of view takes place in two ways: the Mathematics Student Advisory Board and the senior exit interview. The Mathematics Student Advisory Board (MSAB) is a set of six students (three seniors, two juniors, and one sophomore) who meet with job candidates, help plan department social events, and serve as a sounding board for student concerns throughout the year. The senior and junior members are mathematics majors elected by their peers. The sophomore is appointed by the department. As with many such matters, the MSAB is routinely consulted about the Senior Colloquium, especially when a particular concern arises. Though seniors, both through the MSAB and through their exit interviews, are almost always the source of specific concerns or suggestions about the colloquium, the resulting issues are nearly always discussed with the MSAB as a whole. The input of underclass students is often very helpful.
Very rarely seniors will approach their colloquium so irresponsibly that their talks need to be canceled (the student, the advisor, or both may come to this decision). In such cases the student is usually required to give an extra colloquium talk. In general, we tend to deal with recalcitrant seniors on an ad hoc basis. Fortunately they are rare, and their circumstances often convoluted enough, that this approach is most effective and appropriate.
The Senior Colloquium can be a particularly valuable experience for students considering graduate school. The independent work required is good practice for more demanding graduate-level courses. The effort required to prepare a good talk alerts students to the demands of teaching.
The Senior Honors Thesis is an option in all majors, and is typical of many schools' honors programs. Because it usually involves original research, the thesis is of tremendous benefit to students going on to graduate school. The thesis is one of a senior's four regular courses in the fall semester and the sole obligation during the 4-week January term. More ambitious seniors may extend their theses into the spring term as well, if their advisor approves. Successful completion of an honors thesis along with some other accomplishments give the student the degree with Honors in Mathematics.
The honors thesis culminates in a formal paper written by the student and a thesis defense given orally. The paper gives the student's results along with appropriate background and bibliography. Students present their results in the talks, during and after which faculty members may ask questions or make suggestions. Revisions to the thesis may result though they are usually minor. The student receives a regular course grade determined by the faculty advisor. The department as a whole determines whether the thesis warrants Highest Honors, Honors, or, in rare cases, neither. Some students' work is exceptional enough to publish. Clearly the senior thesis is a good foretaste of research in graduate school. The Honors Program is evaluated in basically the same way as the Senior Colloquium: through student interviews, discussions with MSAB, and faculty discussion at department meetings.
The SMALL Undergraduate Research Project is a 9-week summer program in which students investigate open research problems in mathematics and statistics. The largest program of its kind in the country, SMALL is supported in part by a National Science Foundation grant for Research Experiences for Undergraduates and by the Bronfman Science Center of Williams College. Over 140 students have participated in the project since its inception in 1988. Students work in small groups directed by individual faculty members. Many participants have published papers and presented talks at conferences based on work done in SMALL. Some have gone on to complete Ph.D's. Due to funding constraints, about 60% of SMALL students come from Williams; the rest come from colleges and universities across the country. (To answer the obvious question, "SMALL" is an acronym for the faculty names on the original grant.)
SMALL was initiated in order to give talented students an opportunity to experience research in mathematics. We hoped those with particularly strong potential who were inspired by the experience would then choose to attend graduate school and be that much better prepared for it. (I should point out that during the early years of SMALL there were many other efforts undertaken in the department to increase interest in mathematics in general. The best evidence of our success is the increase in majors from about 10 per year to an average over 30, and the increase in enrollments in mathematics courses from around 600 to over 1000 per year.)
Evaluation of SMALL has two in-house components: one by student participants and one by faculty. Each summer the students conduct their own evaluation in the last two weeks of the project. Two or three students oversee the process of designing the questionnaire, collecting responses, and assembling a final report in which comments are presented as anonymously as possible. Each faculty supervisor gets a copy of the evaluation. (The students themselves each receive a written evaluation from their faculty supervisor.) Student comments cover many aspects of SMALL, including the structure of research groups, amount of direction from faculty, number and type of mathematics activities, length of the program, and various extracurricular matters such as housing and social life. Some of the changes motivated by these comments will be presented in the next section.
Faculty evaluate SMALL more informally through discussions during the summer and school year. We have debated many issues, including how to attract and select participants, expectations and work structure for research groups, and the value of attending conferences and other activities.
Because participants work on open problems, SMALL is an excellent means for students to test their graduate school mettle. Though the circumstances are somewhat idealized (summer in the Berkshires, no classes or grades), how one responds to doing mathematics all day every day is a very basic factor in the decision to go to graduate school. More important is one's response to the challenge, frustration, and, hopefully, exhilaration that comes from tackling a real research problem.
Both the SMALL Project and the Honors Program also provide feedback on graduate school preparation in a more indirect but still important way. The number of our most talented and dedicated students who choose to apply to SMALL and go on to do an honors thesis reflects how well we prepare, advise, and encourage students regarding graduate study. Of course, there will always be variations from year to year in the number of honors students and SMALL applicants, and Williams mathematics majors often have strong interests in other fields, but if we notice particularly talented students choosing not to pursue these options, we are forced to examine our role in such a decision. Did we miss the opportunity to create a department mentor for the student? Did the student have some negative experience we could have avoided? Is the student well-informed about the benefits of both SMALL and the Honors Program even if they are unsure about their commitment to mathematics? There may be many reasonable explanations; not all our strong students will choose graduate school or our programs in undergraduate research. In fact, given the current job market, we must be cautious and honest with our students about graduate study. Nonetheless, it is extremely helpful for us and for our students to have programs in which participation is instructive in so many ways.
Findings
Near the end of the spring term, seniors are interviewed by a senior faculty member about their experience as a mathematics major. A question like "What do you think of the Senior Colloquium?" is always included. Most seniors consider the colloquium a highly valuable experience; some feel it is one of the best parts of the major, even though it lies outside the formal curriculum.
Evaluation of the Senior Colloquium from the faculty point of view occurs mainly through discussions at department meetings. Over the past several years, we have discussed balancing advising work among faculty, responding to recalcitrant students (ill-prepared, canceling talks, etc.), evaluating talks (in a different sense than that above), and the process by which we select the best colloquium (for which we have a prize each year).
We feel our rather informal evaluation procedures are more than adequate in large part because the overall quality of colloquium talks has increased. Failures have always been rare, but there used to be several (3-4) borderline cases each year. Now it's more likely to have only one or two such cases, and they are almost always not poor enough to fail but rather just weak or disappointing.
As mentioned previously, the department has grown a lot in the last ten or more years. In particular, the number of students involved in research and the level to which we encourage such activity has increased dramatically. Where most honors theses used to be expository, now most involve original research. Not every student produces interesting results, but the experience of doing mathematics rather than just learning about mathematics is still significant. We believe this change is due in large part to the existence of the SMALL Project and the department's conviction that undergraduates can indeed do research in mathematics.
Several external agencies provide a more indirect evaluation of SMALL. The NSF and other granting agencies expect annual reports. More significantly, by renewing the REU grant three times over the last 9 years, the NSF has signaled its approval of our project. Otherwise, the NSF has provided no specific findings resulting in changes to SMALL. Further indirect evaluation comes from those journals to which research results are submitted for publication. (Journals publishing SMALL papers include The Pacific Journal of Mathematics, The American Mathematical Monthly, and the Journal of Undergraduate Research.)
Use of Findings
In discussion with students about the Senior Colloquium, concerns raised over the years include attendance, how talks are evaluated, and the procedures for preparing a talk. On the first matter, keeping track of attendance has been awkward and unreliable in the past. We now have a system that works very well. At the front of the room we hang a poster-sized attendance sheet for the entire semester with the heading "Colloquium Attendance Prize." (We do give a small prize at the end of the year to the student who attends the most talks.) The prize is a reasonable cover for the attendance sheet which, being posted publicly, encourages students to be responsible about coming to talks. We have also relaxed the requirement to specify a certain number of talks each semester, rather than one each week as we used to require.
Faculty used to evaluate a talk immediately afterwards, leaving students to chat over tea. Though usually we would disappear into another room for only a few minutes, by the time we returned most students would be gone, leaving only the speaker and a few loyal friends. Now we delay our meeting a bit, talking with students about the colloquium or other matters. Though the speaker has to wait a little longer for our verdict, we all benefit from the relaxed time to interact over cookies and soda. On the preparation of talks, wide variations in faculty approaches to advising a talk used to confuse many students. We now have, to some extent, standardized the timetable under which talks are prepared, so students know, for example, that they have to pick an advisor at least four weeks before the talk and give at least one practice talk.
Some of the issues that continue to be revisited include balancing colloquium advising work among faculty and responding to recalcitrant students (ill-prepared, canceling talks, etc). In some years, by their own choice students seem to distribute themselves pretty well amongst different faculty. In other years, however, a few faculty members have found themselves deluged with requests to advise colloquium talks. It's hard to say no, especially for junior faculty. Rather than set specific limits on the number of talks one can advise, we prefer to remind ourselves annually that an imbalance may occur, and then keep an eye on each other and especially on nontenured colleagues, encouraging them to say no if they feel overburdened advising talks.
Our internal evaluations of the SMALL Project have resulted in some significant changes over the years. The project now lasts nine weeks instead of ten. In response to requests from both students and faculty for a more flexible schedule, there is now only one morning "convocation" per week for announcements and talks rather than daily gatherings. (Weekly afternoon colloquia and Friday afternoon teas are still standard.) A more significant change affected the project format. The original format had students work in two groups of four to five on problems supervised by two different faculty members. Under this structure students and faculty often felt that student efforts were spread too thin. Furthermore, fostering efficient and comfortable group dynamics among five or six students could be quite a challenge. Participants now work with only one faculty member in one group of three students. Though there is some risk that students might not find their one group engaging, for the most part students now seem more focused, productive, and, ultimately, satisfied.
The Senior Colloquium has affected some aspects of our teaching in a way not solely connected to graduate school preparation. Seeing the benefits students gain by giving a colloquium talk, some faculty have added a presentation component to their more advanced courses. This not only enhances students' experiences in that course, but can also in return help prepare them for the more demanding Senior Colloquium which may lie ahead.
Success Factors
Our Honors Program is typical of that found at many other schools. Many of the benefits of such a program have already been mentioned: gives talented students an experience in independent work and original research, offers students and faculty a chance for extended one-on-one work, fosters interest in graduate study, helps prepare students for some of the rigors of graduate study. The costs tend to be what one would suppose: the challenge of finding appropriate projects, the extra workload for faculty, the potential for meandered dabbling rather than the structured learning the student would find in a regular course. There is a bit of an ironic downside as well. We have learned through student interviews that sometimes majors who are not doing honors theses feel they are less interesting to the faculty or less important to the department. Such a circumstance reflects the broader challenge any department faces in trying to meet the needs of all its students.
A full-fledged REU program like the SMALL Project is rarer than an Honors Program. It requires a strong commitment by the department as well as the privilege of an NSF grant or other outside funding and, in our case, substantial institutional resources. The costs and benefits of such a program are extensive. On the plus side are many of the same benefits as an Honors Program as mentioned above. In terms of cost, a summer research program requires faculty who are both willing to commit most of their summer and able to engage students in research. While not all faculty at Williams do SMALL every summer, and those who do also spend time on other projects, the fact remains that working in SMALL can take away substantially from other work faculty may wish or need to do. The project also requires a large commitment from the College. Students get subsidized housing and over half of all participants have their stipends paid by the College. (Williams has a strong tradition of supporting student summer research, especially in the sciences. Many faculty receive outside funding and the College returns a portion of the resulting overhead to the sciences to help support student research.) Further information about SMALL may be found in [1].
Though the department is fully committed to it, the Senior Colloquium is an expensive undertaking. Advising a talk is time consuming. Typically majors require some assistance selecting a topic, two or more meetings to digest the reading(s) they have found or which the advisor has given them, further discussion on the design of the talk, and then at least one practice talk. Over the course of three or four weeks one can spend eight or ten hours advising a talk. It's not unusual for some faculty to advise five talks in a year, so the time commitment is nontrivial. Then there is the colloquium series itself in which visitors speak as well as seniors. All faculty try to attend at least one of the two talks each week, often both. Since we also have a weekly faculty research seminar, the time spend attending talks is also nontrivial. All of this effort would probably not be well-spent if its only benefit were to students preparing for graduate school. The overall benefit to all our majors, however, makes the effort well worthwhile. Seniors gain independence, poise, and confidence even as they learn a healthy respect for the effort required to prepare a good presentation. We think the Senior Colloquium is one of the best aspects of our major.
Reference
[1] Adams, C., Bergstrand, D., and Morgan, F.
"The Williams SMALL Undergraduate Research
Project," UME Trends, 1991.
|
||
|
|
|
|