David Bressoud, chair of CUPM, has written a series of ’Launchingsâ? describing aspects of implementation of the CUPM Curriculum Guide. Topics through August 2007 are Introduction, Who are we teaching?, Teaching Students to Think, Only Connect!, Math & Bio 2010, Computational Science in the Mathematics, On Sustaining Curricular Innovation and Renewal, Targeting the math-averse, The Challenge of College Algebra, Avoiding Dead-End Courses, How to find more majors, Teaching for Transference, Keeping the Gates Open, Preparing K-8 teachers, Transition to Proof, Statistics for the Math Major, Writing to Learn Mathematics, The Role of Technology, Geometry in the Mathematics Major, Learning to Think as a Mathematician, Expanding the Boundaries of the Mathematics Curriculum, Attracting and Retaining Majors, Preparing Secondary Teachers, Preparing Our Majors, What has happened to Modern Algebra and Real Analysis?, Return to College Algebra, The Crisis of Calculus, Holding on to the Best and Brightest, Reform Fatigue, The Dangers of Dual Enrollment, and What You Test is What They Learn.
Many of the articles from PRIMUS: Problems, Resources, and Issues in Mathematics Undergraduate Studies can be found on the Find Articles website.
1: Understand the student population and evaluate courses and programs
Mathematical sciences departments should:
- Understand the strengths, weaknesses, career plans, fields of study, and aspirations of the students enrolled in their courses;
- Determine the extent to which the goals of courses and programs offered are aligned with the needs of students and the extent to which these goals are achieved;
- Continually strengthen courses and programs to better align with student needs, and assess the effectiveness of such efforts.
Assessing Programs, Courses, and Blocks of Courses
Supporting Assessment of Undergraduate Mathematics (SAUM) is an MAA sponsored project to support mathematics departments in strengthening their courses and programs based on assessment information. The project supports faculty members and departments across a variety of institutions in efforts to assess student learning in individual courses, coherent blocks of courses, and entire degree programs, using a range of assessment tools. (Madison, 2001) Blocks of courses targeted by the project include the major in mathematics, courses for future teachers, college placement programs, and general education courses, including those aimed at quantitative literacy. Recognizing that much mathematics is learned outside mathematics courses, this last block addresses the mathematical and quantitative literacy achieved in entire degree programs. Institutions that use assessment for program improvement, including research on learning, are of special interest to the project.
A great deal of information to support assessment efforts can be found in the project’s initial volume, Assessment Practices in Undergraduate Mathematics, edited by Bonnie Gold, Sandra Keith and William Marion, which is available in its entirety on the website. The volume includes a series of articles that address assessment and evaluation from many perspectives, and it contains over seventy case studies of assessment at institutions across the U.S.
Since publication of this volume, the SAUM project has supported workshops to assist teams of faculty in developing and implementing assessment plans. Reports on the projects are posted on the SAUM website. One study, Undergraduate Mathematics Program Assessment ’ A Case Study, from American University examined the complete departmental mathematics program. Faculty found that their learning goals were difficult to assess and were not leading to informed program change. In response, they sent a team to the SAUM PREP workshop in 2003. This led to productive conversations within the department, revised learning goals that concentrated on results rather than process, and multiple means of assessment.
Another study, ’Assessing Allegheny College’s introductory calculus and precalculus courses â? focused on a block of courses. This study assessed the effectiveness of the introductory calculus and precalculus courses using analyses of grade data, conversations with client departments, and information about such courses at similar institutions. The initial assessment led to substantial revisions in the department’s course offerings.
The study ’Assessing Written and Oral Communication of Senior Projectsâ? from Saint Mary’s University of Minnesota evaluated the success of a single course. Through this assessment process, the mathematics department learned that while its majors perform well in oral presentations and write well on general topics, their communication about technical aspects of mathematics could be improved. The report describes the changes made to the Senior Seminar course in response to the assessment findings, and changes that were incorporated into the mathematics major program as a whole.
Faculty in the OberlinCollege mathematics department developed a set of objectives for the mathematics major that addresses both content and attitudes. To assess the success of their majors in accomplishing these objectives, and to inform the effort to improve their program, the department maintains a file of syllabi and major assignments for each course, conducts an annual survey of post-graduate Activity, collects a statistical profile of the mathematics majors in each class, and administers sequenced surveys to mathematics majors when they declare a major, when they graduate, and 5 years after graduation. For further information, contact Michael Henle.
St. Olaf's mathematics department conducted a faculty retreat as part of a department self-study. To prepare for the retreat, faculty completed a questionnaire covering the department's mission, its curriculum and programs, staffing, students and assessment, faculty, communication, "fun stuff," and technology, resources, and facilities. Questions included: What should the goals of the department be in the next 5 years? How well does the mathematics curriculum serve (a) math majors; (b) students completing general education requirements; (c) students majoring in other areas? How can we best gauge how well we serve our students? What kind of support do you need to engage in your own professional activities? Are you receiving it? Which department activities do you value most? Results of the questionnaire were included with other data in a summary document used by outside reviewers and by the department in connection with its self-study. Outcomes of the self-study continue to develop, but they include new emphasis on undergraduate research and interdisciplinary collaboration. Significant structural changes to the mathematics major are also under consideration. These changes would encourage students to view mathematics, its applications, and its connections with allied areas more broadly than before. For further information, contact Paul Zorn.
External Support for Assessing Undergraduate Mathematics
The National Science Foundation has supported a variety of professional development opportunities that provide hands-on experience in assessment for both faculty and graduate students, including the series of workshops organized in connection with the SAUM project. Although these workshops are no longer available to new faculty, there is an online guide that can be freely used and adapted. Another NSF-funded project is investigating the long-term impact that the use of technology in introductory college mathematics courses has on students in STEM (Science, Technology, Engineering, and Mathematics) disciplines. Individuals at six institutions collect data locally while working as a team with experts who provide training and support. For more information, contact project directors Susan Ganter and Jack Bookman.
As a result of a recent reorganization, the assessment of student achievement, including research on assessment and the development of assessment tools and practices, has been designated as one track of the National Science Foundation’s Division of Undergraduate Education (DUE) Course, Curriculum, and Laboratory Improvement (CCLI) program.
The Colorado School of Mines has produced a website, Assessment Resource Page, for departments developing their departmental assessment plans. Links are provided to departmental assessment plans that are publicly available on the worldwide web.
Assessment Tools for the Classroom
Part II of SAUM’s publication, Assessment Practices in Undergraduate Mathematics, entitled Assessment in the Individual Classroom, provides examples of specific classroom assessment practices that can be useful in attempting to understand students’ thinking and determining what they understand. As David Bressoud writes in an article in this section, ’No matter how beautifully prepared our classroom presentation may be, what the student hears is not always what we think we have said.â? A widely used technique based on Angelo and Cross’s Classroom Assessment Techniques (Angelo & Cross, 1993), is the One-Minute Paper, in which students are given the last few minutes of class to write the answer(s) to one or two questions, such as ’What was the most important point in the lecture?â? or ’What is the slope of the graph of a function at a point?â? or ’How comfortable do you feel asking questions?â? or ’How clear was today's lecture for you?â? Additional classroom assessment techniques are organized into the categories Testing and Grading, Classroom Assessment Techniques, Reviewing Before Examinations, What Do Students Really Understand?, Projects and Writing to Learn Mathematics, Cooperative Groups and Problem-Centered Methods, Special-Needs Students, and Assessing the Course as a Whole.
The monograph Keeping Score by Ann Shannon discusses a variety of issues involved in designing assessment tasks, especially those that aim to evaluate a broad range of mathematical skills and abilities. An executive summary is available from the publisher, The National Academy Press. The article ’Mathematics performance assessment: A new game for studentsâ? by Ann Shannon and Judith S. Zawojewski (Mathematics Teacher, 88(9), 752’757) considers how to teach students to understand and benefit from new forms of assessment that may initially seem strange to them.
Colleges and universities frequently use placement exams to gather information about entering students’ mathematical abilities. Assessment Practices in Undergraduate Mathematics edited by B. Gold et al. contains two articles on placement exams. One describes the placement procedures at St.OlafCollege, and the other describes the methods at the University of Arizona. At St.OlafCollege there are three levels of placement exam (basic, regular and advanced) to respond to the varied mathematical backgrounds of incoming students. Each exam includes subjective questions about students’ mathematical motivation, background, calculator experience, and plans for college mathematics study. At the University of Arizona there are two levels of placement exam (intermediate algebra and college algebra/ trigonometry). Both schools report that a significant commitment is needed from the department and its faculty to complete the placement testing and assign students to appropriate courses. But they also report that these efforts bear dividends, as students enrolled in appropriate courses tend to be more successful. At the University of Arizona placement exam data have also been used to analyze the mathematics program, inform future decisions on course offerings, and improve testing procedures.
Norma G. Rueda & Carole Sokolowski, MerrimackCollege, wrote Mathematics Placement Test: Helping Students Succeed, which describes their study comparing the performance of students who took the course recommended by their mathematics placement exam score and students who did not take the recommended course. They found that students who followed the recommendation did much better than those who took a higher-level course or did not take the placement exam. Because of the careful statistical nature of the study, its results have been useful in convincing students to follow placement exam guidelines.
Assessing student background is especially important in ’open-doorâ? institutions, where any high school graduate can be admitted. For example, the CBMS2000 survey (p. 141) found that in two-year colleges, ’diagnostic or placement testing [was] â?¦ almost universal in availability.â?
The Transition Mathematics Project is a collaborative project of K-12 schools, community and technical colleges and baccalaureate institutions to assist with the transition from high school to college/university mathematics in the State of Washington. Its Resource Center contains information about placement tests and placement test issues, as well as much other information.
Alvin Baranchik and Barry Cherkas conducted a study of placement exams taken by more than 1000 students at an urban four-year college and published the results in ’Differential Patterns of Guessing and Omitting in Mathematics Placement Testingâ? (in Dubinsky et al. (Eds.), Research in Collegiate Mathematics Education, II, 1996, AMS, 177-193). They found that the tendency to guess, omit answers, or not finish the exam varied by gender, ethnicity, first language, and birthplace, in ways that could not be fully explained by prior mathematical skill. The authors concluded that scoring by number of correct answers reduces the representation in gateway courses of certain cultural and gender groups for reasons unrelated to mathematical skill. They recommend that colleges either employ ’formula scoring,â? which assigns a small penalty for guessing or a small bonus for omitting questions, or that they provide the following instructions at the beginning of the test: ’Because of the way this test is scored and interpreted, to be fair to yourself you must answer all questions, even if you must guess. You will not be penalized for guessing or incorrect answers.â?
In January 2005 Derek Bruff, Harvard University, posted a list of resource articles about placement exams on the discussion list for the Special Interest Group of the MAA on Research in Undergraduate Mathematics Education (SIGMAA-RUME).
A mathematical autobiography is a written assignment in which the writer relates and reflects on memories of his or her experiences with mathematics. Reading students’ mathematical autobiographies can help instructors understand that their students’ mathematical experiences may be very different from their own. An instructor’s awareness and acknowledgment of these differences may facilitate classroom communication, especially in lower-division courses. According to M. A. Conway, the assignment is most productive when it prompts a purposeful, extended, memory-search and is rewarded (Conway, 1990). For example, it might receive a grade or extra credit or be used to replace a test score. A sample of such an assignment is from Shandy Hauk, University of NorthernColorado
Interacting with students through the advising process can help faculty understand students and their goals and improve the effectiveness of a department’s program. The University of Southern Mississippi has developed a comprehensive advising program from providing mentors for freshmen to conducting a survey of graduating students.
MountHolyokeCollege provides a webpage, Beginning the Study of Mathematics and Statistics at Mount Holyoke: A "User's Guide" to selecting a first course, to advise students who are starting their collegiate mathematics study. The page explains that most students begin with a calculus course, a quantitative reasoning or data analysis course, a seminar course, or a computer science course. The webpage then describes the options within each type of course and helps each student choose the most appropriate one for his or her mathematical background and interests.
The website for the mathematics department at the University of North Texas includes links for academic advising and placement, course information, student resources, and career information. The site provides information for students on choosing their first math class, preparing for the placement exam, and fulfilling the general education requirement. It also includes course descriptions and sequencing information, tips for success with mathematics courses, information about the mathematics laboratory and tutoring help, description of math club activities, and extensive program and career information for mathematics majors, including links to outside resources.
Students at KenyonCollege are offered ’Advice to New Students,â? about how their choice of a first mathematics course could fit their academic plans. For example, students who want only an introduction to mathematics or a course to satisfy a distribution requirement can select from Elements of Statistics, Surprises at Infinity, Quantitative Reasoning, Pre-Calculus, Calculus A, and Introduction to Computer Science.
The University of Michigan at Dearborn publishes an ’Advising Newsletterâ? each semester. The newsletter is sent to all declared mathematics majors and contains information on courses, careers, competitions, scholarships, and on-campus jobs for math majors. It is available through the mathematics department's link Information for Students.
Redesigning Courses and Programs in Response to Information about Students
High drop-out and failure rates for certain groups of minority students in calculus at the University of California at Berkeley, despite their relatively high mathematics entrance scores, prompted Uri Treisman to investigate the study habits of both successful and unsuccessful students. Based on his findings, he introduced a mathematics workshop supplement for the Berkeley calculus course, recruiting African-American and Latino students who had relatively high SAT Mathematics scores and/or intended to major in a mathematics-based field. Key elements of the workshop involved:
* the commitment to help minority students excel at the University rather than merely avoid failure;
* collaborative learning and the use of small-group teaching methods; and
* faculty sponsorship, which has nourished the program and enabled it to survive. (Treisman, 1985, pp. 30’31)
The program at Berkeley ’replaces regular calculus discussion sections with workshop-style discussion sections, in which the students collaborate on non-textbook, non-routine problems.â? During these work sessions (which meet for larger blocks of time than traditional classes), ’[students] begin working the problems individually, then, when things get tough, in collaboration with one another. These experiences lead to a strong sense of community and the forging of lasting friendshipsâ? (Gillman, 1990, p. 8). The Berkeley program has been so successful that it has been adapted by universities and colleges throughout the country. Treisman has stressed that the program is not remedial ’ and should not be ’ and care is taken to prevent implementations from reverting to remedial programs.
Eric Hsu’s website at San FranciscoStateUniversity contains a number of resources for Treisman-style workshops, including a database of workshop problems and a handbook developed for instructors and teaching assistants in the Berkeley program. Additional information available on the Internet includes a history of the programs, summaries of evaluations (p.18), and details of one evaluation. Another detailed evaluation is in An Efficacy Study of the Calculus Workshop Model (Bonsangue, 1994). A few websites for existing Treisman-style programs are University of Wisconsin-Madison, Wayne State University, and University of Texas at Austin.
The Mathematical Thinking and Problem Solving course offered by Alan Schoenfeld, University of California Berkeley, was designed in response to research findings about student beliefs, attitudes, and problem solving abilities. More discussion of his course is in Part 1, Section 2.