SAUM

• Getting Started
• News
• About SAUM
• FAQ
• Online Workshop
• Supporting Assessment
• MAA Notes #49
• Links
• Bibliography

MAA Notes #49 – Table of Contents

Introduction

• Book Dedication, Preface, and Acknowledgements
• Introduction: Assessing Assessment, Lynn Arthur Steen
• Assessment and the MAA, Bernard L. Madison
• How to Use the Book
• Teaching Goals Inventory, Self Scorable Version
  

  The Teaching Goals Inventory is reprinted from Classroom Assessment Techniques, by Angelo and Cross. We suggest that you, the reader, take this inventory (with a specific course in mind, as the authors suggest) in part for your own information, and also because it can help you locate your own teaching styles and priorities in the context of this book.

Part I: Assessing the Major

Introduction, William A. Marion

Portfolios, Capstone Courses, and Focus Groups

• Assessing a Major in Mathematics, Laurie Hopkins

  At a small, private women's liberal arts college in the South student portfolios have become the principal means for assessing the major. Unique to this program, certain courses are designated as portfolio development courses and in these courses the student is asked to reflect in writing on the connection between the material included in the portfolio and the department goals.

• Portfolio Assessment of the Major, Linda R. Sons

  A Midwestern, comprehensive university which has five different programs in the mathematical sciences - General, Applied Mathematics, Computational Mathematics, Probability and Statistics, and Mathematics Education - requires students to maintain assessment portfolios in courses which are common to all five of the emphases. The portfolios of those who have graduated during the year are examined by a department assessment committee shortly after the close of the spring semester.

• An Assessment Program Built around a Capstone Course, Charles Peltier

  A full-year senior capstone course has evolved at a small, private women's liberal arts college in the Midwest to become the principal tool for assessing the major. Within this two-semester seminar each student has to develop an independent study project, known as the comprehensive project. Preliminary work on the project begins in the first semester and oral and written presentations of the completed project are given in the second semester.

• Using a Capstone Course to Assess a Variety of Skills, Deborah A. Frantz

  At a mid-sized, regional university in the East each student must complete a one-semester senior seminar in which a variety of assessment methods are used to assess student learning in the major. These methods include: a traditional final exam, a course project, an expository paper, reading and writing assignments, a journal and a portfolio - all of which are designed to assess a variety of skills acquired throughout a student's four years.

• The Use of Focus Groups within a Cyclic Assessment Program, Marie P. Sheckels

  An entirely different approach to assessing the mathematics major has been developed at a state-supported, coeducational, liberal arts college in the Midsouth. Graduating seniors participate in focus group sessions which are held two days prior to graduation. These are informal sessions with a serious intent: to assess student learning in the major.

Comprehensive Examinations

• Assessing the Major Via a Comprehensive Senior Exam, Bonnie Gold

  At a small, private men's liberal arts college in the Midwest, a comprehensive examination, known as comps, has been a tradition for seventy years. It has evolved into the assessment technique which the department uses to assess student learning. Comps are taken by seniors over a two-day period just prior to the start of the spring semester. The exam consists of two parts: a written component in the mathematics major and an oral component over the liberal arts.

• A Joint Written Comprehensive Examination to Assess Mathematics Processes and Lifetime Metaskills, G. Daniel Callon

  A rather unique approach to giving a comprehensive exam to seniors is described in this article by a faculty member at a small, private co-ed college in the Midwest. The exam is taken by seniors in their fall semester and lasts one week. It is a written group exam which is taken by teams of three to five students. Currently, the exam is written and graded by a faculty colleague from outside the college. As part of a college-wide assessment program, the Department of Mathematical Sciences developed a departmental student learning plan, detailing the goals and objectives which students majoring in mathematics or computing should achieve by the time of graduation. For mathematics, there were three major goals. The first goal related to an understanding of fundamental concepts and algorithms and their relationships, applications, and historical development. The second centered on the process of development of new mathematical knowledge through experimentation, conjecture, and proof. The third focused on those skills which are necessary to adapt to new challenges and situations and to continue to learn throughout a lifetime. These skills, vital to mathematicians and non-mathematicians alike, include oral and written communication skills, the ability to work collaboratively, and facility with the use of technology and information resources.

• Undergraduate Core Assessment in the Mathematical Sciences, John W. Emert

  In this article a "less" comprehensive exam to assess student learning in the core courses taken by all under-graduate mathematics majors at a regional, comprehensive university in the Midwest is discussed. We are guided through the process involved in developing the assessment instrument which is used in all four tracks of the mathematical sciences program: Actuarial Science, Mathematics, Mathematics Education and Statistics.

Overall Programs

• Outcomes Assessment in the B.S. Statistics Program at Iowa State University, Richard A. Groeneveld and W. Robert Stephenson

  Although the statistics program at this large Ph.D.-granting university in the Midwest is not housed within a mathematical sciences department, the wide variety of measures used to assess the statistics program can serve as one model for assessing the mathematics major. All of the assessment measures used are described with particular emphasis on surveys of graduates and surveys of employers of graduates.

• Assessment of the Mathematics Major at Xavier: First Steps, Janice B. Walker

  In an evolving assessment program at a private, medium-sized, comprehensive university in the Midwest a variety of assessment techniques are being developed to assess student learning. How two of them - exit interviews and the Educational Testing Service's Major Field Test in Mathematics - are part of the fabric of the mathematics program's assessment cycle is described in this article.

• Assessing Essential Academic Skills from the Perspective of the Mathematics Major Mark Michael

  At a private, church-related liberal arts college in the East a crucial point for assessing student learning occurs midway through a student's four year program. A sophomore-junior diagnostic project which is part of the Discrete Mathematics course, taken by all majors, is the vehicle for the assessment. Each student in the course must complete a substantial expository paper which spans the entire semester on a subject related to the course.

• Department Goals and Assessment, Elias Toubassi

  At a large, research university in the West, a major effort over the past 10 to 15 years has been underway to reform the entry level mathematics courses which the department offers. Assessment has been at the heart of this process. Focusing on all first year courses through assessment has had a positive effect on the undergraduate mathematics major and the courses in that major. This article describes the process and the ongoing assessment of student learning.

Special Programs Within the Major

• Analyzing the Value of a Transitional Mathematics Course, Judith A. Palagallo and William A. Blue

  At a large, regional university in the Midwest a specific course (Fundamentals of Advanced Mathematics) at the sophomore level provides a transition for student majors from the more computationally-based aspects of the first year courses to the more abstract upper-division courses. Surveys have been developed to measure the effects of this course on upper level courses in abstract algebra and advanced calculus. In addition, these surveys provide information about student learning in the major.

• Assessing the Major: Serving the Needs of Students Considering Graduate School, Deborah Bergstrand

  The focus of this article is on assessing student learning for a segment of the undergraduate mathematics majors: those talented students who are prospective mathematics graduate students. At this private, liberal arts college in the Northeast there are three aspects of the undergraduate mathematics experience outside the standard curriculum which are described in this paper: a senior seminar (required of all majors), an Honors thesis and a summer undergraduate research experience. All three combined are used to assess how well the department is preparing students for graduate school.

Part II: Assessment in the Individual Classroom

Introduction, Bonnie Gold

Testing and Grading

• What Happened to Tests?, Sharon Cutler Ross

  For students to believe that we expect them to understand the ideas, not just be able to do computations, our tests must reflect this expectation. This article discusses how this can be done.

• Interactive E-Mail Assessment, Michael D. Fried

  This article discusses how internet technology can be harnessed to give students semi-automated individualized help. The intended reader appreciates how little problem solving guidance students get in class, and how they are on their own to handle giving meaning to learning an overstuffed curriculum.

• Flexible Grade Weightings, William E. Bonnice

  Using a computer spreadsheet to compute grades, it's easy to let students (even in a large class) focus on their strengths by choosing what percent (within a range selected by the instructor) of their grade comes from each required activity.

Classroom Assessment Techniques (A La Angelo & Cross)

• The One-Minute Paper, David M. Bressoud

  This quick technique helps the instructor find out what students have gotten out of a given day's class, and works well with both large and small classes.

• Concept Maps, Dwight Atkins

  By drawing concept maps, students strengthen their understanding of how a new concept is related to others they already know.

• If You Want to Know What Students Understand, Ask Them John Koker

  To gain insight into how much students actually understand, and what they have learned by working the problems, have them discuss the evolution of their ideas as they work on homework sets.

• Improving Classes with Quick Assessment Techniques, John W. Emert

  This article discusses three quick techniques which can alert the instructor to potential problems. The first helps students understand course goals, the second evaluates the effectiveness of group work, and the third is a general way of finding out how things are going in time to make changes. As I continue to experiment with various informal classroom assessment techniques, I have come to favor assessment tools which use no more than ten minutes of class time and require no more than an hour after class to tabulate and form a response. The three techniques included here provide quick ways to find out what is going well and what is not, and allow me to address any problems in a timely manner.

• Creating a Professional Environment in the Classroom, Sandra Z. Keith

  This article describes four activities which can help students develop a more professional attitude toward their coursework: looking directly at their expectations, revising goals after a test, finding applications of the course in their chosen field, and preparing a résumé.

Reviewing Before Examinations

• True or False? Explain! Janet Heine Barnett

  One way to find out what students understand is to ask them true/false questions, but have them justify their answers. These justifications bring out confusions about the concepts, but are also the beginning, for calculus students, of writing mathematical proofs.

• Define, Compare, Contrast, Explain ..., Joann Bossenbroek

  These short writing assignments help students clarify concepts, and show the instructor where more work is needed.

• Student-Created Problems Demonstrate Knowledge and Understanding, Agnes M. Rash

  Having students write problems shows the instructor where the gaps in their understanding are at the same time that it has students review for an upcoming test. Further, it can help students find the relevance of the course to their own interests.

• In-Depth Interviews to Understand Student Understanding, M. Kathleen Heid

  Students' answers on tests don't always show their true level of understanding. Sometimes they understand more than their answers indicate, and sometimes, despite their regurgitating the correct words, they don't understand what they write. This article discusses a method to probe what they actually understand.

Projects and Writing to Learn Mathematics

• Assessing Expository Mathematics, Annalisa Crannell

  This article gives many helpful hints to the instructor who wants to assign writing projects, both on what to think about when making the assignment, and what to do with the projects once they're turned in.

• Assessing Modeling Projects in Calculus and Precalculus: Two Approaches, Charles E. Emenaker

  Once you've assigned a writing project and collected the papers, how are you going to grade it without spending the rest of the semester on that one set? This article outlines two grading scales which can make this more efficient.

• Assessment During Complex Problem-Solving Group Work in Class,Brian J. Winkel

  During student efforts to attack and solve complex, technology-based problems there is rich opportunity for assessment. The teacher can assess student initiative, creativity, and discovery; flexibility and tolerance; communication, team, and group self-assessment skills; mathematical knowledge; implementation of established and newly discovered mathematical concepts; and translation from physical descriptions to mathematical models.

• Student Assessment through Portfolios, Alan P. Knoerr and Michael A. McDonald

  Reflective portfolios help students assess their own growth. Project portfolios identify their interests and tackle more ambitious assignments.

• Using Writing to Assess Understanding of Calculus Concepts, Dorothee Jane Blum

  Student write expository papers in an honors, non-science majors' calculus course to integrate the major ideas they're studying.

• Journals: Assessment without Anxiety, Alvin White

  Using journals for formative assessment encourages students to explore topics they might have been intimidated by if they were being graded.

• Assessing General Education Mathematics Through Writing and Questions, Patricia Clark Kenschaft

  General education students can learn to read mathematics more thoughtfully and critically by writing questions over the reading, and short response papers-without enormous investment of faculty time grading.

Cooperative Groups and Problem-Centered Methods

• Combining Individual and Group Evaluations, Nancy L. Hagelgans

  One frequent concern of faculty members who have not yet tried cooperative learning is that giving the same grade to a whole group will be unfair both to the hard workers and the laggards. This article addresses this issue.

• Group Testing, Catherine A. Roberts

  From reviewing before a test to various ways for students to take tests collaboratively, this article looks at ways that groups can be used to evaluate student learning while increasing that learning.

• Continuous Evaluation Using Cooperative Learning, Carolyn W. Rouviere

  This article discusses several cooperative learning techniques, principally for formative assessment. These include ways to help students learn the importance of clear definitions and several review techniques (comment-along, teams-games-tournaments, and jigsaw).

• Collaborative Oral Take-Home Exams, Annalisa Crannell

  Giving Collaborative oral take-home examinations allows the instructor to assess how well students handle the kind of non-routine problems we would all like our students to be able to solve.

• Assessment in a Problem-Centered College Mathematics Course, Sandra Davis Trowell and Grayson H. Wheatley

  When using a problem-centered teaching approach, the instructor needs new methods of assessment. This article explores one such approach.

Special Need Students

• Assessing Learning of Female Students, Regina Brunner

  The author discusses assessment techniques which she has found to be particularly effective with female students.

• Strategies to Assess the Adult Learner, Jacqueline Brannon Giles

  Adult students are often better motivated than traditional-age students, but many have not taken an examination for many years. Thus, finding appropriate methods of assessment poses a challenge.

• The Class Mission Statement, Ellen Clay

  To set the tone of a course at the beginning, develop with the class a "class mission statement," which can be revisited as the course progresses to assess progress toward meeting course goals.

• Early In-Course Assessment of Faculty by Students, William E. Bonnice

  Students learn to take responsibility for their learning by giving input on how the class is going and what needs to be changed. This works in classes of all sizes.

• Student Feedback Teams in a Mathematics Classroom, David Lomen

  A student feedback team is a subset of the class which gives the instructor feedback on how the class is doing with new material.

• Early Student Feedback, Patricia Shure

  In introductory courses in mathematics at the University of Michigan, an instructional consultant visits the class one-third of the way into the semester. This observer holds a discussion with the class, in the absence of the instructor, about how the course is going, and provides feedback to the instructor.

• Friendly Course Evaluations, Janet Heine Barnett

  By having students write a letter to a friend about your course, you can get useful information both on what the students have learned and on what they thought of your course.

• The Course Portfolio in Mathematics, Steven R. Dunbar

  The teaching portfolio is an alternative to the standard course questionnaire for summing up a course. The instructor collects data throughout the semester into a course portfolio, which can then be used by that instructor or passed on to others teaching the course.

Part III: Departmental Assessment Initiatives

Introduction, Sandra Z. Keith

Placement and Advising

• Administering a Placement Test: St. Olaf College, Judith N. Cederberg

  A small college nurturing a large calculus clientele in a flexible calculus program recognizes the need for careful placement, and studies the effectiveness of its efforts withstatistics to derive a formula for placement.

• A Mathematics Placement and Advising Program, Donna Krawczyk and Elias Toubassi

  Placement was the issue at this large, comprehensive school. This article explains, among other initiatives the department took to improve its accessibility to students, a Mathematics Readiness Testing Program. Statistics measuring the reliability of the testing are included.

• A Comprehensive Advising Program, Stephen A. Doblin and Wallace C. Pye

  A large, budding university concentrates on the importance of advising students at all levels. From providing mentors for freshmen to surveying graduating students and alumni, this department operates with continuing feedback.

• A Quantitative Literacy Program, Linda R. Sons

  The author helped create MAA Guidelines for Quantitative Literacy and here spells out how this document was used at her large university. This school tested students and graded results in a variety of courses. Results led to curricular changes.

• Creating a General Education Course: A Cognitive Approach, Albert D. Otto, Cheryl A. Lubinski, and Carol T. Benson

  What is the point of teaching students if they're not learning? Here a general education course operates from a learning-theoretic mold; mathematics education instructors become involved to help students construct their own learning.

• Using Pre- and Post-Testing in a Liberal Arts Mathematics Course to Improve Teaching and Learning, Mark Michael

  This pre- and post-testing system in a liberal arts mathematics course raises interesting questions about testing in general, and asks why students may sometimes appear to go backwards in their learning.

• Coming to Terms with Quantitative Literacy in General Education: or, the Uses of Fuzzy Assessment, Philip Keith

  This article presents an administrative look at the ramifications of accommodating various departments' views of how quantitative literacy is to be defined. The issue is: what are the students telling us-how do we interpret the answers they provide to the questions we've asked? The value of "fuzzy" assessment is discussed in the interpretation of a simple survey which helps move a collective bargaining institution on track.

• Does Developmental Mathematics Work? Eileen L. Poiani

  An inner city school with a diverse, multicultural clientele is deeply committed to raising students' mathematical abilities. The school has operated with grants that are now drying up, but that help authored some assessment studies over a ten-year period. They ask: does developmental mathematics help, or is it a hopeless cause?

Mathematics in a Service Role

• Let Them Know What You're Up to, Listen to What They Say, J. Curtis Chipman

  Effective Relations Between a Department of Mathematical Sciences and the Rest of the Institution How can a mathematics department please its client disciplines? This department finds a solution in establishing a web of responsible persons to establish goals for students and instructors, course leaders, committees, and faculty liaisons, for placement of students into courses and for the content of those courses.

• Have Our Students with Other Majors Learned the Skills They Need? William O. Martin and Steven F. Bauman

  A large university begins by asking teachers in other disciplines not for a "wish list" but for a practical analysis of the mathematical knowledge required in their courses. Pretests for students reflect these expectations, and discussion of results encourages networking.

• A TEAM Teaching Experience in Mathematics/Economics Marilyn L. Repsher and J. Rody Borg

  Opening a course to both mathematics and business faculty teaching as a team creates public dialogue about problems that straddle two departments.

• Factors Affecting the Completion of Undergraduate Degrees in Science, Engineering, and Mathematics for Underrepresented Minority Students: The Senior Bulge Study, Martin Vern Bonsangue

  Commissioned by the California State University's Chancellor's Office, this study looks at transfer students and suggests key areas for reform.

• Evaluating the Effects of Reform, Richard West

  West Point turned an entire department around. Using an in-depth assessment study with careful attention to the needs of client disciplines, the department created a brand new curriculum, and continues to study it with the "Fullan model" which the author investigated in his dissertation.

• A Comprehensive, Pro-Active Assessment Program, Robert Olin and Lin Scruggs

  A large technical institute using the author as assessment coordinator, creates a broad new assessment program, looking at all aspects of the department's role. Statistical studies guide improvements in curriculum, teaching and relations with the rest of the university.

Assessing Reform Courses

• Assessment in One Learning Theory Based Approach to Teaching, Ed Dubinsky

  In this discussion piece, the author explains an approach to teaching based on Learning Theory, particularly examining a Calculus course to ask how assessment can best feed back into the learning environment.

• An Evaluation of Calculus Reform: A Preliminary Report of a National Study, Susan Ganter

  This is a preliminary report of a study at NSF dealing with what NSF has looked for, what it has found, and directions for future study. One such direction will be to shift from "teaching" to "student learning" and the learning environment.

• Increasing the Dialogue About Calculus with a Questionnaire, A. Darien Lauten, Karen Graham, and Joan Ferrini-Mundy

  A practical survey is provided here which might be found useful for departments looking to initiate discussion about goals and expectations in a Calculus course.

• Assessing Student Attitudes and Learning Gains, Nancy Baxter Hastings

  This assessment program stresses breadth - pre- and post-testing, journals, comparative test questions, student interviews and questionnaires, and more.

• Does Calculus Reform Work? Joel Silverberg

  This large study examines data over a long period of time regarding a calculus "reform" course. Grades and attitude are studied, with advice for novices at assessment.

• Assessing Student Attitudes and Learning Gains

  This assessment program stresses breadth - pre- and post-testing, journals, comparative test questions, student interviews and questionnaires, and more.

• Assessing the Effectiveness of Innovative Educational Reform Efforts, Keith E. Schwingendorf

  This study explains the creation of a calculus reform program, its objectives, and philosophy and provides an in-depth comparison of reform-trained students with traditional students.

• The Evaluation of Project CALC at Duke University, 1989-1994, Jack Bookman and Charles P. Friedman

  This in-depth study analyzes a Calculus reform program. It looks not only at analytical gains in student understanding but affective gains as well.

Part IV: Assessing Teaching

Introduction, William A. Marion

Assessing Teaching

• Departmental Assistance in Formative Assessment of Teaching, Alan P. Knoerr, Michael A. McDonald, and Rae McCormick

  At a small liberal arts college in the West a department-wide program has been developed to help faculty assess and improve their teaching while courses are in progress. Descriptions of what led to this effort, the steps already taken, the resources involved and plans for the future are presented.

• Assessing the Teaching of College Mathematics Faculty, C. Patrick Collier

  At a regional, comprehensive university in the Midwest the mathematics faculty have been grappling with what it means to be an effective teacher and how to evaluate such effectiveness. Their conclusion is that student evaluations should not be the primary means of evaluating teaching. Hence, the department is in the process of articulating a statement of expectations for teaching from which appropriate assessment instruments will be developed.

• Using Video and Peer Feedback to Improve Teaching, Joel David Hamkins

  This article discusses a program at Berkeley of using videotaping of actual classes, and peer feedback, to improve teaching. While the program was aimed at graduate students, it can be adapted to use with faculty members.

• Exchanging Class Visits: Improving Teaching for Both Junior and Senior Faculty, Deborah Bergstrand

  A peer visitation program for both junior and senior faculty at a small, liberal arts college in the East has been put into place to help improve the quality of teaching. Every junior faculty member is paired with a senior colleague to exchange class visits. This program is designed to foster discussion of teaching, and the sharing of ideas and to provide constructive criticism about the teaching effectiveness of each member of the pair.

• Peer Review of Teaching, Pao-sheng Hsu

  At a land-grant university in the Northeast one mathematics faculty member has begun experimenting with a peer visitation program in which a team of faculty visits her class at least twice a semester. What's unique about this process is that the team, usually three in number, consists of faculty both within and without the department and all visit the same class at the same time. This technique provides the instructor with a diversity of views on her teaching effectiveness. evaluations, since these are often mandated by university administrations. In an effort to generate discussion and broaden the perspective on evaluation of teaching, I initiated the experiment of peer review of my classes. I have experimented with inviting faculty members who have experience in ethnographic research1 and are from outside of my own discipline, as well as colleagues from my own department.

Appendix, Further Reading, and Bibliography

• Assessment of Student Learning for Improving the Undergraduate Major in Mathematics, Updated by CUPM on August 5, 2005.
  

  Originally the Apendix to the Book. A Reprint of "Assessment of Student Learning for Improving the Undergraduate Major in Mathematics." Prepared by The Mathematical Association of America, Subcommittee on Assessment, Committee on the Undergraduate Program in Mathematics. Approved by CUPM at the San Francisco meeting, January 4, 1995.

• Suggestions for Further Reading
• Bibliography