Assessing Allegheny College’s introductory
calculus and precalculus courses

Ronald Harrell and Tamara Lakins
Allegheny College
December 30, 2003

Abstract:  With the goal of improving student learning, Allegheny College assesses the effectiveness of its introductory calculus and precalculus courses by analyses of grade data, conversations with client departments, and information regarding such courses at similar institutions.  The initial assessment led to substantial revisions in its offerings.

Background and goals:  What did we hope to accomplish?

Allegheny College, a national liberal arts college located in Meadville, PA, has an enrollment of approximately 1800 students.  The mathematics department teaches approximately 550 students per year in its introductory calculus and precalculus courses, which from 1990 to 2003 consisted of Math A (Intermediate Algebra), a one-year sequence Math 155/156 (Calculus/Precalculus), and a traditional calculus course Math 160 (Calculus I).

The original intent of Math A was to prepare students for the sequence Math 155/156.  It also became a specific prerequisite for the following courses in client departments:  introductory chemistry, introductory computer science, and the research design and statistics course in the psychology department.  The original intent of the sequence Math 155/156 was to provide an alternate entry point into the regular calculus sequence for students with weaker precalculus backgrounds.  In particular, the sequence covered selected precalculus topics in addition to the calculus topics traditionally covered in Calculus I and was designed to prepare students for Calculus II.

At the beginning of the fall semester, entering first-year students are placed in a mathematics course based on an algebra-trigonometry-precalculus-based placement exam and/or consultation with a member of the mathematics department.  The placement is a non-binding recommendation.

The goals of our assessment project were to determine whether the intermediate algebra course and precalculus/calculus sequence were addressing the needs of our students and to make any needed changes to the courses, which in any case would include the addition of a regular assessment program.  The intermediate algebra course had existed for over 20 years, and the precalculus/calculus sequence had been used for 13 years.  In neither case had we ever assessed their effectiveness, and we had anecdotal evidence from members of the mathematics department that the courses were not preparing students at the level we expected for subsequent courses.  It was time to take a hard look at both.

Description:  What did we do?

During the 2001-02 academic year the mathematics department assessed the effectiveness of Math A (Intermediate Algebra) and Math 155/156 (Precalculus/Calculus) by reviewing data regarding student performance in these and subsequent courses, by conducting conversations with faculty in client departments, and by reviewing information about precalculus and introductory calculus offerings at the 26 colleges and universities identified by Allegheny College as in our comparison group. 

Data on the distribution of grades were examined for the 359 entering first-year students who enrolled in Math A during a fall semester from 1997 to 2000 and who went on to take introductory chemistry, Math 155, introductory computer science, or the psychology statistics course.  We omit the discussion of students who took the introductory computer science course, since their number was too small to draw viable conclusions.  In addition, a second comparison of grade distributions was made for students taking these subsequent courses, based on mathematics placement level and regardless of whether the students took Math A.  For this second comparison, grades in Fall 1997 through Spring 2001 were considered, and the sample consisted of students who entered Allegheny in a fall semester from 1997 to 2000.

Data on the distribution of grades were examined for the 310 entering first-year students who enrolled in Math 155 in a fall semester from 1997 to 1999 and who subsequently took Math 156 (the second course in the precalculus/calculus sequence) and Math 170 (Calculus II) at Allegheny College.  These latter students were compared to the 224 entering first-year students who began in Math 160 (Calculus I) in a fall semester from 1997 to 1999 and who subsequently took Math 170 at Allegheny.

All grade data considered were obtained from the Allegheny College Registrar in electronic format.  Student names were removed from the data, and fictitious identification numbers were used in order to ensure student anonymity. 

We consulted 1-2 faculty members  (typically the chair or faculty who teach courses requiring quantitative or mathematical skills) in our client departments regarding Math A and our calculus offerings.  Client departments include biology, chemistry, computer science, economics, environmental science, geology, mathematics, physics, and psychology.    In the case of Math A, we wished to specifically learn why Math A was a prerequisite for the four earlier mentioned courses, and which skills currently taught in Math A were considered essential for these courses.  In the case of calculus and those departments that require it for their major, we wanted to know specifically why calculus was required and which skills were considered essential.

Data from the colleges and universities in our comparison group were obtained by researching the college and university web sites; when necessary, details were clarified by email and phone calls to the appropriate department chair, or another faculty member designated by a department chair.  In particular, we wished to determine the following information about each school in our comparison group:  its precalculus and introductory calculus course offerings and the mechanisms by which it enables students with deficiencies in their mathematics preparation to prepare for calculus.

Initially, we planned to review the high school backgrounds of a sample of our students.  This would have required reading folders of individual students to determine which math courses had been taken and which grades had been obtained. Organizing that information would have been complicated by the variety of math courses now taught in high schools. So given our resources, we decided to drop this part of the study. 

Insights:  What did we learn?

The analysis of the data, on student performance in Math A (Intermediate Algebra) and the courses for which it became a prerequisite, indicated that Math A did not seem to adequately prepare students for any of the subsequent courses, except possibly the psychology statistics course.  (Year-by-year details of all the grade information discussed here can be found in Appendix A.)  In particular, of the 359 entering first-year students who enrolled in Math A during their first semester at Allegheny College, 246 (69%) earned a successful grade of C or higher.  Of these 246 students, 112 went on to take introductory chemistry with 74 (66%) earning a grade of C or higher, 183 took Math 155 with 111 (61%) earning a grade of C or higher, and 54 took the psychology statistics course with 46 (85%) earning a grade of C or higher.

Even more striking was that our second comparison showed that, of all students who enrolled in introductory chemistry or Math 155 from Fall 1997 to Spring 2001, students who placed in Math A and who did not take it prior to enrolling in one of these courses often fared as well as, or better than, students who placed in Math A and took it first.  In particular, 49 of the 74 students (66%) who placed in Math A but did not take it or Math 155 prior to enrolling in introductory chemistry earned a grade of C or higher in chemistry, while 85 of the 141 students (60%) who placed in Math A and took it or Math 155 prior to taking introductory chemistry earned a grade of C or higher in chemistry.  In the case of Math 155, 67 of the 107 students (63%) who placed in Math A but did not take it prior to taking Math 155 earned a grade of C or higher in the course, while 124 of the 231 students (54%) who placed in Math A and took it prior to taking Math 155 earned a grade of C or higher in the course.  Thus, it appeared that Math A was ineffective in preparing students for subsequent courses. 

The analysis of the data on student performance in the one-year precalculus/calculus sequence (Math 155/156) indicated that a smaller number of these students went on to take Calculus II (Math 170) than one might expect.  Of the 310 entering first year students who enrolled in Math 155 during a fall semester from 1997 to 1999 only 52 (17%) went on to take Calculus II (either at Allegheny or elsewhere) by Spring 2001, while of the 446 entering first year students who enrolled in Math 160 during the same semesters, 248 (56%) went on to take Calculus II (either at Allegheny or elsewhere) by Spring 2001.  Furthermore, students in Allegheny’s Math 170 who began in Math 155/156 were generally not as successful as those who began in Math 160.  Of the 47 first-year students who began in Math 155 during a fall semester from 1997 to 1999 and who went on to take Math 170 at Allegheny by Spring 2001, 29 (62%) earned a grade of C or higher.  On the other hand, of the 224 students who began in Math 160 and went on to take Math 170 at Allegheny by Spring 2001, 168 (75%) earned a grade of C or higher.  Thus the Math 155/156 sequence was not serving students as well as we had hoped.

The conversations with faculty in client departments revealed that Math A was serving more than one purpose.   The chemistry and mathematics departments required Math A, or placement out of Math A, as a prerequisite for entry level courses which require students to possess traditional algebra skills.  However, faculty in the biology, computer science, environmental science, and psychology departments emphasized wanting their students to possess general quantitative skills (such as good problem solving skills, the ability to translate word problems into an appropriate mathematical model, being able to work with and interpret data, and having good number sense), rather than specific kinds of algebra skills.  Clearly at the level below calculus, something more than a course that only reviews algebra was needed.

In the case of Math 155/156 the chemistry, mathematics, and physics departments expected a thorough treatment of the concepts of calculus and a good knowledge of computational skills.  The other client departments required at least some calculus for their major programs and expected students to be able to understand and apply the concepts of calculus.  Often these same departments wanted calculus courses to be less theoretical and were primarily interested in having their students learn how to do only the more elementary computations.  Thus there were two kinds of clientele for calculus courses.

Finally, an examination of course offerings at the 26 schools in our comparison group indicated that we were only one of 2 institutions that offer a course at the level of Math A, and only 2 institutions offer a course at the level of college algebra.  On the other hand, a total of 15 colleges offer precalculus, 7 offer a combined precalculus and calculus sequence similar to our Math 155/156, and 5 colleges offer no course which directly prepares students for calculus.  Finally, 8 institutions offer a one-semester alternative to the traditional first calculus course (usually a course which emphasizes applications from the social and/or life sciences).

Redesigning: What did we do? 

The above findings indicated that our lower level course offerings were not diverse enough to meet students’ needs, and some courses did not accomplish their intended purpose.  After much discussion, the mathematics department replaced Math A (Intermediate Algebra) and Math 155/156 (Precalculus/Calculus), which were hierarchically designed to prepare students for the regular calculus sequence, with four courses that provide students with three options for beginning the study of mathematics, depending on their individual goals.  The new courses are briefly described below; more detailed descriptions and course goals are given in Appendix B.  Important in the design of the new courses was meeting the needs expressed by client departments.

The first option, Math 110 (Elementary Mathematical Models), replaced Math A.  Math 110 is an elementary algebra-based modeling course that emphasizes the study of real world problems and models, and rates of change.  Algebra is reviewed as needed.  The course is for those students who need a mathematics course but not a calculus course.  The intended audience consists of humanities and social science students, who take it to fulfill a graduation requirement or who find it useful in a major field, such as economics, environmental studies, political science, or psychology. 

The second option, Math 150 (Precalculus), is a standard college level course on the subject, intended only for those students who need to take the regular calculus sequence, but who also need to brush up on precalculus topics before doing so. Topics covered in the course were formerly taught in the sequence Math 155/156.

The third option is the sequence Math 157/158 (Calculus I and II for Social/Life Sciences), which replaces the Math 155/156 sequence.  The sequence is for those students who need calculus, but not the thorough and more rigorous treatment presented in the regular calculus sequence.  The emphasis is on the concepts of calculus and how they occur in problems from the life and social sciences.  Topics in both single and multivariate calculus are covered. This option serves primarily biology, economics, and environmental science students.  

The mathematics placement exam, which was previously used to determine placement in either Math A, Math 155, or Math 160, is still used by the department.  The department now requires a particular score on that exam in order to recommend placement into Math 160.  Students who do not achieve the target score may enroll in Math 110, Math 150, or Math 157, depending on high school background, confidence, and intended major.  These latter three courses have no formal prerequisite.  While Math 110 is a terminal course and is not intended to prepare students for Math 157, some students may opt to take it before attempting Math 157.

Students who have already received college credit for a calculus course may not take Math 110, 150, or 157 for credit.  Furthermore, students who begin in the Math 157-158 sequence and later change to a major requiring the ordinary calculus sequence Math 160-170 are treated on an individual basis.  A student may take Math 160 for credit after receiving credit for Math 157 but not after receiving credit for Math 158.  (Such a student may still take Math 160, but will not receive credit toward graduation for the course.)  Students who wish to take Math 170 after Math 158 receive individual advising.

In addition to creating these new courses and options, we also created a way to have an ongoing assessment of each in order to monitor their effectiveness.  Content goals for each course are assessed using selected questions on final exams to gauge how well students have mastered the material.  For the goal

• students will be able to communicate mathematical information in written form,

• students will be able to choose, implement, refine, and interpret appropriate mathematical models for various real-world problems,

which pertains to Math 110 and the sequence Math 157/158, the assessment consists of short writing assignments, projects, or appropriate homework where writing is emphasized.  Thus the assessment data consists of scores on selected final exam questions and instructors’ impressions of the writing assignments and/or projects during the semester.

At the end of each semester the instructors for each course meet briefly to review and discuss the assessment data for that semester.  They then submit a report of their findings to the department chair, who makes the contents of the report available to the entire department.  It is hoped that, by reviewing a small amount of assessment data each year, the department will be able to maintain an ongoing and accurate picture of the effectiveness of the courses and our assessment methods, while at the same time not placing too great a burden on the faculty.  Periodically, perhaps every three to five years, we will do a wider assessment, similar to the one reported here, that determines how well these courses prepare students for subsequent courses, not only in mathematics, but in other areas.

Other Comments and Recommendations.  After Lakins compiled and analyzed the data, the department spent several weeks of intense discussion creating the above four replacement courses.  The two major sticking points were the exact nature of the replacement courses and finding a reasonable ongoing assessment plan for each course.  Some faculty questioned whether ongoing assessment plans were needed, and getting them to see the usefulness and benefits of such plans was a hard sell.  To help students and advisors the department also made up documents explaining the new courses in detail and indicating which courses would benefit which students.

Acknowledgements

The authors wish to acknowledge the support and assistance of Allegheny College’s Associate Dean Richard Holmgren in planning and implementing this assessment project. Funds from Dean Holmgren’s Arthur Vining Davis Foundation grant supported the 2001-02 salary of Tamara Lakins, as well as the travel costs of both authors to three SAUM Assessment in Undergraduate Mathematics workshops.  We also acknowledge the assistance and advice of our workshop team leader Bonnie Gold.