The Course Portfolio in Mathematics: Capturing the Scholarship in Teaching

The teaching portfolio is an alternative to the standard course questionnaire for summing up a course. The instructor collects data throughout the semester into a course portfolio, which can then be used by that instructor or passed on to others teaching the course.

Background and Purpose

The course portfolio documents course and classroom activity in a format open to reflection, peer review, standards evaluation, and discussion. The course portfolio can demonstrate the intellectual development in the students of the concepts in the course and documents that a transformation of students occurred because of the intellectual efforts of the instructor.

Bill Cerbin, at the University of Wisconsin - La Crosse was one of the first to form the idea of the course portfolio. The American Association for Higher Education (AAHE) Teaching Initiative Project "From Idea to Prototype: The Peer Review of Teaching" promoted the portfolio in a menu of strategies [2, 3].

According to a report from the Joint Policy Board on Mathematics, a major obstacle to including evaluation of teaching in the reward system is the absence of evaluation methods. The course portfolio is one response to the need for better tools to document and evaluate teaching. It puts faculty in charge of monitoring, documenting and improving the quality of teaching and learning. It provides a vehicle for the faculty member to reflect on what's working and what's not, and for communicating that reflective wisdom and practice to colleagues.

Peer feedback on the course portfolio provides professional accountability, and improves the process of teaching. Both the presenter and the reader learn new pedagogical methods to improve their teaching.

I have created a course portfolio for a course on Principles of Operations Research. Our department teaches this course yearly in one or two sections to both senior mathematics majors and as a service course for senior students in actuarial science. From a prerequisite of linear algebra and probability, the course covers linear optimization, queuing theory, and decision analysis.

Method

The word "portfolio" conjures up a vision of a collection of papers, or a binder of whatever was available, perhaps assembled in haste. But this is not what I mean by the "course portfolio." The course portfolio starts with the course syllabus, with explicit mathematical goals for students in the course. The instructor measures progress toward the goals through a pre-course student background knowledge probe; informal course assessment by students; homework exercises, solutions, and scores; in-class active-learning exercises; tests; labs or projects; formal student course evaluations; and a post-course evaluation by the instructor.

A "course portfolio" is not a "teaching portfolio." A teaching portfolio is a larger, longer document that establishes over a period of time and a range of courses that you are an effective teacher.

Here are some questions to address in a portfolio:

• What were you trying to accomplish?
• Why were these goals selected?
• Did the course meet the goals? How do you know?
• What problems were encountered in meeting the goals?
• How did you conduct the course?
• How did you challenge students?
• What changes in topics, materials and assignments are necessary?

For the course portfolio on Principles of Operations Research, I began with 5 prerequisite skills for the course, including the ability to solve systems of equations by row reduction, and computing conditional probabilities. I listed 5 specific goals of the course, including formulating and solving optimization models, and formulating and analyzing queuing models. All prerequisite skills and goals of the course appeared on the course information sheet distributed to students on the first day of class. I selected mathematical modeling as a major focus of the course, since I believed that most senior mathematics majors were already adept in solving formulated problems, but needed practice in turning operations research situations into a mathematical model. The course information sheet became the first item in the course portfolio.

For the first day of class, I created a "background knowledge probe" testing student knowledge of both the prerequisite skills and the goals of the course. Problems on the background knowledge probe were of the sort that would typically be on the final exam for the course or its prerequisites. A copy of the background knowledge probe together with an item analysis of student scores went into the course portfolio. Comparing this with the scores on the final exam gives a measurement of student learning and progress.

I included homework assignments, copies of exams, and course projects in the course portfolio as evidence of the direction of the course toward the goals. Occasional minute papers and in-class assessments measured student progress through the term. I collated and summarized information from the minute papers and included this in the course portfolio as the "student voice." I also did an item analysis of the scores from tests as a further refinement on measuring student gains in knowledge. Angelo and Cross [1] have samples and ideas for constructing the background knowledge probe, lecture assessments, and other assessment instruments that become evidence in the course portfolio.

The final element of the course portfolio was a "course reflection memo." In this memo created at the end of the course, I reflected on the appropriateness of the goals, the student progress on the goals, and recommendations for the course. I included the course reflection as the second element in the course portfolio, directly after the course information sheet. There is no recipe, algorithm, or checklist for preparing a course portfolio. In practice, the course portfolio will be as varied as the faculty preparing it, or the courses taught.

Preparing a course portfolio requires time that is always in short supply. Nevertheless, a course portfolio is not such a daunting task since we already keep course notes from the last time we taught a course. My experience is that the course portfolio requires about an average of one hour per week through the term of the course. The time is not evenly distributed, however, since careful preparation of the goals and organization of the course takes place at the beginning of the term. Assessing student learning as a whole is a large part of the portfolio that occurs best at the end of the term. Thoughtful reflection on the course and its conduct and results naturally occurs at the end of the term. Fortunately these are the parts of the term when more time is available. During the term, collection of course materials and evidence of student learning is easy and automatic.

Findings

In the course on Operations Research that I documented with a course portfolio I found that students generally came to the course with good prerequisite skills. This factual evidence was encouraging counterpoint to "hallway talk" about student learning. I was able to document that students improved their homework solution writing through the course of the semester. The classroom assessments showed that the students generally understood the important points in my lectures, with some exceptions. The assessments also showed that the students generally were not reading the material before coming to class, so I will need to figure out some way to enforce text reading in the future. Analysis of the scores on the background knowledge probe, the exams, and the final exams documented that students learned the modeling and analysis of linear programming problems acceptably well, but had more slightly more trouble with queuing models and dynamic programming. The course portfolio was generally able to document that student learning had taken place in the course.

Use of Findings

One immediate benefit of the portfolio is a course record. A course portfolio would be especially useful to new faculty, or to faculty teaching a course for the first time. The portfolio would give that faculty member a sense of coverage, the level of sophistication in presenting the material, and standards of achievement. The course portfolio also gives a platform for successive development of a course by several faculty. This year, two other instructors taught the course on Operations Research. At the beginning of the term, I turned over my portfolio to them as a guide for constructing their course. The course portfolio informed their thinking about the structure and pace of the course, and helped them make adjustments in the syllabus.

Having a course portfolio presents the possibility of using the material for external evaluation. Peers are essential for feedback in teaching because they are the most familiar with the content on a substantive level.

Success Factors

The AAHE Peer Review Project [2] has some advice about course portfolios:

• Seek agreement at the outset about the purposes of portfolios, who will use the information, who owns it, what's at stake.
• Be selective. The power of the portfolio comes from sampling performance, not amassing every possible scrap of information.
• Think of the portfolio as an argument, a case, a thesis with relevant evidence and examples cited.
• Organize the portfolio around goals, your own goals or departmental goals.
• Include a variety of kinds of evidence from a variety of sources.
• Provide reflective commentary on evidence. Reflective commentary is useful in revealing the pedagogical thinking behind various kinds of evidence and helping readers to know what to look for.
• Experiment with various formats and structures, be deliberate about whether portfolios are working and how you can refine the structure.

It may be useful to include the actual course syllabus, to show what the instructor actually covered in the course in comparison with what the instructor intended to cover.

A variant is the "focused portfolio," which concentrates on one specific aspect or goal of a mathematics course and the progress made toward the goal. For example, a course portfolio might focus on the integration of graphing calculators or on the transition of students from "problem-solving" to "proof-creating." In any case, there should be a cover letter or preface to the reader, briefly explaining the purpose and methods of the portfolio.

An important addition to the portfolio is the "student voice," so the portfolio clearly represents not only what the instructor saw in the course, but also the students got out of it. Informal class assessments, formal student evaluations and samples of homework and student projects are ways of including the student voice.

References

[1] Angelo, T.A., and Cross, K.P.Classroom Assessment Techniques, second edition. Jossey-Bass, San Francisco, 1993.

[2] Hutchings, P. From Idea to Prototype: The Peer Review of Teaching. American Association of Higher Education, 1995.

[3] Hutchings, P. Making Teaching Community Property: A Menu for Peer collaboration and Peer Review. American Association for Higher Education, 1996.