The Four A's: Accountability, Accreditation,
Assessment, and Articulation
By Peter Ewell and Lynn Arthur Steen
Team members from different colleges discuss assessment issues at a recent SAUM workshop at Towson University.
Most MAA members understandably devote their primary professional energies to teaching and research. Indeed, support of education and scholarship in the mathematical sciences ("especially in the collegiate field") is central to the mission of the Association. It used to be that the primary pressures on college and university mathematicians came from their peers and partners - from the practitioners and users of mathematics. But as higher education has become more important, more prevalent, and more costly, those who pay the bills - administrators, trustees, legislators, parents - are beginning to ask for evidence of value. Mathematics, the argument goes, is too important to be left to mathematicians.
Recognizing that most mathematics departments have little experience in answering the evidentiary questions that public agencies tend to ask (e.g., What mathematics do all students learn? Why aren't more students preparing to be mathematics teachers?), MAA sought and received a $500,000 grant from the National Science Foundation for the project "SAUM: Supporting Assessment in Undergraduate Mathematics." (See sidebar for details.) Discussions at section meetings and SAUM workshops confirm that most MAA members are still focused on mathematics teaching and scholarship without being fully aware of significant changes underway in the ecosystem in which mathematics departments operate. Thus our topic: the four As.
Supporting Assessment of Undergraduate Mathematics (SAUM)
An MAA project supported by a $500,000 grant from the National Science Foundation, SAUM is designed to help departments effectively assess one or more goals common to undergraduate mathematics departments, especially (a) the mathematics major, (b) preparation of future teachers; (c) college placement programs; (d) mathematics in mathematics-intensive programs, and (e) general education courses, including those aimed at quantitative literacy. The project is directed by Bernard L. Madison, Professor of Mathematics and former Dean of the College of Arts and Sciences at the University of Arkansas.
The central activity of the SAUM project has been a series of workshops for teams of faculty who work with each other, with other teams, and with project leaders to develop plans to implement on their own campuses. Each team is preparing a case study documenting the assessment issue they were trying to resolve, and reporting on progress or results over the 2-3 years of the project activity. These case studies, and others, will be assembled into a report together with syntheses of common issues that will be distributed to every department of mathematics in the United States.
MAA's leadership in assessment has a relatively long history- especially compared with other disciplines. In 1990 CUPM established a subcommittee on assessment, chaired by Bernard Madison. In 1995 this committee published guidelines for departments to use in establishing a cycle of assessment aimed at program improvement. Subsequently, in 1999, MAA published Assessment Practices in Undergraduate Mathematics (MAA Notes #49). This volume contains over seventy case studies of assessment at institutions across the US; the CUPM subcommittee's 1995 report is reprinted as an appendix. SAUM extends MAA's history of energetic assessment support one step further, concluding with a new volume of case studies to help departments identify models they may wish to adapt (as well as mistakes they may wish to avoid).
In addition to project director Bernard L. Madison, other senior SAUM personnel are Bonnie Gold (Monmouth University), William E. Haver (Virginia Commonwealth University), William A. Marion, Jr. (Valparaiso University), and Lynn A. Steen (St. Olaf College). Peter Ewell (NCHEMS) serves as Project Evaluator and Michael Pearson (MAA Associate Director for Programs) manages the project at the MAA's Washington office.
Further information can be found on the SAUM website.
Accountability. "No Child Left Behind," the signature legislation that George Bush used to inaugurate his presidency, imposes unprecedented federal requirements on the K-12 system to use regularly-administered standardized tests to document annual improvements in all student ethnic and socioeconomic sub-populations. Many feel that higher education's turn is next.
During this session Congress must reauthorize the Higher Education Act (the source of student loans, among other things). Opening discussions have been more vigorous than in previous years as the administration seems to be suggesting that the logic of "No Child Left Behind" - annual improvements in every subpopulation - should apply to higher education just as it does to K-12. However, since no one appears to know how to measure learning outcomes in higher education, the leading proposal is to focus on graduation rates. Many state legislatures are moving in the same direction. Mathematics will be in the crosshairs of this movement since one of the leading reasons for students not graduating is failure to pass required mathematics courses.
Accreditation. Unique among nations, the United States relies on private associations of schools and colleges - rather than a government agency - to validate a college's standing as a destination of government-funded student aid. Most academic members of the MAA can recall periodic episodes of special reports associated with visits of accreditors to their campuses. In 1992 defaults on student loans were politically hot, so that year in the Reauthorization of the Higher Education Act Congress required accreditors to validate key financial and socioeconomic indicators associated with loan defaults. Knowledgeable observers believe that this time Congress will further tighten the rules for accreditation by imposing specific learning, retention, or graduation criteria.
Accreditors were first mandated to look at learning outcomes as a condition of recognition in Department of Education rules established in 1989, but these directives were not very specific. Now they are at the top of the list of things Congress is grilling accreditors about. Consistent with the logic of "No Child Left Behind," moreover, accreditors are increasingly being asked not just about whether they examine student learning outcomes in the light of institutional mission, but also why they don't establish and enforce common standards of learning that all must meet. In this way the "mathematics for all" movement that has swept K-12 is likely to take root in higher education.
The potential federal stick is to disenfranchise accreditors from their gatekeeping role. Indeed, a bill to decouple accreditation from gatekeeping has already been drafted and may be deployed. So while Reauthorization is unlikely to bring higher education a federally-mandated, test-based, accountability "solution" as in K-12 (largely because of lack of money and expected fierce resistance), the attempt may very well be to achieve this goal indirectly by leaning much more heavily on accreditors to enforce standards of learning.
Assessment. "Standards of learning" brings us to the central focus of SAUM: assessment. How do mathematics departments know whether students are achieving the goals they have in mind and the standards they set? Not only Congress but even more so state legislatures and boards of trustees are now focused on evidence of outcomes. Reports like Measuring Up  highlight what seem to many policymakers to be an appalling lack of information about results of higher education. One consequence is the prominence of K-16 rhetoric that appears to call for a test-based response, at least through grade 14. Funding schemes based on measurable performance statistics are increasingly popular with state legislatures. These create a problem of particular significance to mathematics, namely, the lack of alignment between newly mandated high-school exit tests, college entrance exams, and college placement tests.
Assessment that is persuasive and helpful requires actual evidence of student learning. Not only do external stakeholders increasingly require such evidence, but it is vitally important internally for educational improvement. Too often assessment practices reveal only "compliance behavior" where assessment is seen as "for" somebody else. That posture ensures that the results of assessment remain unconnected to decisions that matter. Moreover, it suggests to those responsible for students' overall education - deans, provosts, trustees - that academics don't see the educational revolution taking place in front of them.
Articulation. Underlying all these educational and political factors are significant changes in the way people are going to school. A majority of students now attend two or more institutions to complete programs (and almost a fifth attend three or more).  Rapid increases in technology-based or distance education raise new issues of quality and confound standard methods of academic accounting (courses and credits). Especially noticeable and problematic in mathematics is the increasing overlap between high school and college illustrated by remedial courses in college, advanced placement courses in high school, and dual enrollment courses in both. 
Because these new enrollment patterns are educationally non-traditional, they resist accounting that relies on traditional time-based course credits. And since they are also attractively economical, they increase budget managers' attention to certified outcomes and end-product assessment. Articulation is thus moving from friendly agreements among neighboring mathematics departments who wish to ensure transfer of credits to a hot button state-wide issue that leads to some kind of legislatively enforced outcomes-based accountability "solution."
Mathematicians tend to see their discipline as uncompromisingly hierarchical, so they focus considerable attention on setting prerequisites for courses and planning curricula to ensure that students who enter courses are "prepared." But the increased "swirling" of students among different institutions renders many of these careful articulation plans ineffective if not meaningless.
We are in a new era that demands new approaches. Fortunately, as the accompanying box illustrates, the MAA has a distinctive track record of leadership on these issues that is well recognized outside the bounds of the mathematical community. Members and departments who may only now be awakening to the seriousness of the four A's should not hesitate to build on the work that has already been done.
National Center for Public Policy and Higher Education. Measuring Up 2002: The State-by-State Report Card for Higher Education. Available Online at: http://measuringup.highereducation.org/2002/reporthome.htm.
Adelman, Clifford. 1999. Answers in the Tool Box: Academic Intensity, Attendance Patterns and Bachelor's Degree Attainment. Washington D.C.: U.S. Department of Education, Office of Educational Research and Improvement.
Clark, Richard W. Dual Credit: A Report of Programs and Policies that Offer High School Students College Credit, Pew Charitable Trusts, 2001.
Peter Ewell is a Senior Associate at the National Center for Higher Education Management Systems (NCHEMS). Lynn Arthur Steen is Professor of Mathematics at St. Olaf College and a former president of the MAA. Both are members of the SAUM Steering Committee.