 Membership
 MAA Press
 Meetings
 Competitions
 Community
 Programs
 Students
 High School Teachers
 Faculty and Departments
 Underrepresented Groups
 MAA Awards
 MAA Grants
 News
 About MAA
What does this mean for mathematics departments and for those of us who teach mathematics at the college or university level? First, we believe that our profession will be called upon to define standards for degrees and courses. We may well see a new structure for our degree programs. Second, we feel we will see significant changes in classroom practices, especially with regard to assessment. Third, we foresee structural changes that will affect placement, advising, and the way students advance through a major. Our purpose here is to discuss these changes. We will give a general overview of standardsbased education and then detail some of its possible implications in concrete terms.
What are Standards? In education the word "standard" is used in many different ways^{3}. Some of the ones relevant to our discussion are:
Content Standards: Content standards describe the knowledge and skills that students should attain in a class or course of study. (Example:"understand the concept and be able to apply matrix algebra to compute the affect of a linear transformation on a finitedimensional real vector space.") Curriculum Standards: Curriculum standards describe general goals or ways in which classes or programs should be organized and taught. (Example: "Recognize mathematical problems that can be linearized and apply techniques of linear algebra to solve them.") Competency Standards: Competency standards set required or desirable levels of performance on specified activities or tests. These are also called benchmarks, proficiencies, and performance standards. (Example: "compute the matrix representation of an algebraically defined linear transformation from R^{m} to R^{n} in a given pair of ordered bases.") Performance Standards: Performance standards include competency standards but are more general. For example, measures of institutional success are often called performance standards. (Example: "At least 80% of students should be able to demonstrate competency in computational linear algebra by the end of their junior year.") 
Voluntary national mathematics tests^{4} and standardized tests such as the SAT's represent competency standards. The NCTM standards provide examples of both content and curriculum standards. Institutional and professional accreditation boards are starting to request information on graduation rates and employer satisfaction with graduates. These are performance standards in the more general sense.
There is often confusion among different types of standards. This stems in part from the way in which they interact. Tests like the SAT or GRE are really competency standards, but they also define a body of knowledge that must be mastered by a student wishing to score well. As such, they implicitly define content standards. Similarly, standardized tests dictate curriculum standards for teachers who wish to adequately prepare their students. "Teaching to the test" may be viewed as the practice of translating competency or performance standards into content and curriculum, if not into content and curriculum standards.
Conversely, it makes little sense to set standards for what students should learn in a class without having a reliable way to determine whether students have learned it. The practice of writing exams is that of translating content standards into assessments. But students appreciate knowing more than just the time, place, and topics covered on an exam; they also want to know what types and level of problems will be on the exam and what constitutes satisfactory performance. Answering these questions requires translating content and curriculum into competency and performance standards.
Criteria and Norms. Standards are often linked to specific assessment methods. When considering these we must consider how the output of an assessment is interpreted. There is a significant difference between comparing the results of an assignment or test to those of other students and interpreting the results in terms of independent criteria. Standards that reference population norms are called normreferenced while those that define independent criteria for levels of performance are called criterionreferenced.
Tests graded on "curves" are examples of normreferenced assessments. Lower level undergraduate mathematics courses often appear to have rules (written or unwritten) about the rough percentages of students who may be awarded A?s and B?s. In this case, grades are normreferenced standards. At the other end of the spectrum, a fundamental standard for awarding a doctorate in mathematics is a demonstrated ability to do independent and original research of publishable quality. This is a criterionreferenced standard.
Standardsbased Education. Standardsbased education consists of two interrelated practices. The first is that of explicitly defining content and, if appropriate, curriculum standards. In more common language, this means explicitly defining learning objectives. The second is that of defining associated criterionreferenced performance standards and assessments. In other words, it is necessary to explicitly say what it means for students to have achieved the learning objectives and at least suggest ways in which this can be measured.
A final important aspect of standardsbased education is that the standards are public. They do not depend on individual teachers and are supposed to be intelligible to all informed and relevant participants, including the students. This means that every student should understand the expectations for satisfactory performance. There is also a policy point to be made. Educational policy makers often site wide disparities among the curricula and content offered by public education to different socioeconomic classes. An A from one high school might not be the same as an A from another. Standardsbased education attacks this problem by uniformizing and publicizing standards that apply to all students in the same educational system^{5}.
Degree Programs. What does it mean to complete an undergraduate major in mathematics? Most departments define their major in terms of course work. It is common to have a series of different tracks or options, a core curriculum, and/or a senior thesis, but the essential requirement is the successful completion of a set of courses^{6}. The standardsbased approach demands that a major be defined in terms of what students know and can do. It asks us to define assessments and performance measures by means of which students will demonstrate competence. And it requires that these standards and measures be public. Colleagues from other institutions should be able to look at our standards, examine the evidence collected for our students, and make consistent determinations as to which students have fulfilled our requirements.
What would it take to define such standards? Content and curriculum standards usually start with objectives that are few in number and general in nature. These are then refined in several stages until they reach the level of performance standards and assessments. As an example, we might decide that one of several requirements for obtaining a degree in mathematics is that a student
demonstrate the ability to communicate mathematics orally and in writing. 
This sounds great, but what does it mean? One could argue about practically every word: What does it mean to demonstrate, what counts as mathematics, and why should we require both oral and written communication? More fundamentally, what are we really trying to capture? Does this refer to formal proofs? Is it enough to be able to describe mathematical ideas or does this include correct use of notation? If so, at what level of sophistication? Are we really trying to say that students should be able to produce results and communicate them as part of a team?
One way to address these questions is to define what are called a set of criteria for proficiency. Thus we might say that to demonstrate the ability to communicate mathematics orally and in writing a student should

This is just an example, and is not hard to imagine debating at length whether these are appropriate criteria, to what extent they must all be met, whether other criteria should be added, and so on. But even were we to accept these as criteria, we would still be a long way from defining both how students can demonstrate they have met these criteria and setting an appropriate level of performance. The next step is to define what are often called indicators that move towards performance criteria. For example, we might say that a student who can state the basic theorems of undergraduate real analysis on demand has given evidence that she has met the first criterion, and we might then go on to give examples of acceptable and unacceptable statements of some theorems. But this would undoubtedly be only one of a long list of indicators intended to frame the meaning of 1, 2, and 3 in terms that can be consistently used to make judgments by our colleagues in mathematics and consistently understood by our colleagues in other departments and by the students themselves.
Benefits and Motivation. The process of defining standards for a degree program is hardly simple but it has some tangible benefits. Assuming that the process took place at the department level in a single institution, it would necessarily engage a department and its constituents in the important discussion of what we really want our students to take away from a mathematics degree. We suggest that this discussion will provide a starting point for rethinking what we need to do to get our students to where we want them to go. That is not to say that what we do now is terrible, but every program should undergo periodic review and we feel that a mathematics major designed by starting with a clear set of overall objectives and performance standards has the potential to be more efficient and effective than one put together from a collection of courses.
But the real motivation for thinking about standardsbased degrees is that the world is changing.
Accreditation. If we do see a move towards standardsbased degrees, one possible mechanism is that of professional accreditation. The idea of accrediting mathematics programs is one that surfaces periodically. We do not wish to argue its merits or demerits point out that the accreditation process itself can effect significant changes in teaching and assessment practices. This is illustrated by ABET, the accreditation board of engineering and technology, which switched from a totally coursebased set of requirements to an outcomesbased set of requirements for all engineering degrees (see references in note 10). This is resulting in fundamental revisions of engineering programs nationwide.
As far as we know, there are no large standardsbased degree programs in mathematics. But there is experimentation going on with standardsbased courses in mathematics^{9} and other disciplines^{10}. What do we mean by a standardsbased course and how is a standardsbased course designed?
Just as was described for a degree program, the design of a standardsbased course starts with a definition of what students should know and be able to do at its conclusion (content standards) and criteria that define satisfactory performance (performance standards). In its simplest form this is the practice of specifying a set of learning objectives. These objectives, however, be "public" and must be measurable on the basis of tests, projects, and other evidence produced by the students.
To test if learning objectives are public, it is good to do two thought experiments. The first is to imagine teaching your class, gathering evidence of student performance, and having a colleague at another university evaluate your students using only this evidence and your written list of objectives and criteria. You should feel confident that your colleague would make judgments consistent with your own. The second experiment is to ask yourself to what extent students can reliably evaluate their own work on the basis of your stated objectives and criteria. Students should have a clear picture of where they are headed and be able to recognize when they have arrived^{11}.
The design of a standardsbased course continues by choosing topics, activities, and a syllabus that target the objectives. It is important to note that the topics covered in a course and the learning objectives of a course are not necessarily the same thing. If the point of a course is getting the right answer on problems of a specific form or type, then that should be an explicit objective. Indeed, the students will appreciate knowing what they need to do in order to succeed. But individual techniques and derivations are often steps on the way to something more synthetic and should be included in the course topics but not in the course objectives.
Standardsbased Assessment. As we hope is by now clear, the process of defining course objectives is one part of a feedback loop. The other part is assessment. The requirement that objectives be assessed and that students generate assessable evidence limits the number objectives we can realistically expect to reach and demands rethinking what students do in and outside of the classroom. This might mean more time spent on projects, more writing assignments, a different style of lecture, using the Web for drill type quizzing  we each must find what works best for us. It also means taking a new look at some familiar things such as the way in which we grade tests.
Standardsbased assessment focuses on the holistic determination of whether or not a student has met the demands of a particular standard. The practice of scoring individual test questions and totaling the scores does not really address this; it does not separate the score into component objectives and is largely justified by the need to create a class norm and to compare students to each other. Partial credit is another example of a practice that is frequently normreferenced and not criterionreferenced. Students might appreciate knowing that they got a problem partly right and seeing where they went wrong, but discussions of partial credit tend to center around issues of fairness and accuracy for the sake of comparative evaluation.
An alternative method of grading an exam is to review the entire exam paper in light of each course objective. The idea is to assign a simple score of "failed to meet", "met", or "exceeded" for each objective and not grade individual problems. This is called using a "scoring guide". One could argue for a finer scale, but we find advantages in the discipline imposed by not being able to equivocate by saying that a student has met some fractional part of an objective . Does it really do much good for a student to be able to almost get the right answers when he or she takes the next course?
What about multiple choice tests? What about projectbased and other means of assessment? Many alternative means of assessment, especially those that involve student writing, seem naturally suited for a standardsbased system. On the other hand, resources often limit or preclude their use. For example, we looked at the college algebra course taught in a large lecture/recitation format at one of our institutions. A quick computation showed that there are less than 30 minutes of graduate assistant grading time available for each student for an entire term. This appears to say that computergraded multiple choice tests will remain a major part of the grading scheme. This does not, however, mean that multiple choice tests need to be graded by assigning points to each problem and computing the total. It would make a lot more sense to identify clusters of problems that address each course objective and report the scores on each objective. If computers deliver the exams, it is possible to go beyond that to an adaptive testing scheme. If a student misses a question on a certain objective, another one is delivered, and if a student correctly answers questions on a topic, the program either drops the topic or delivers more challenging questions.
Example. As an example, learning objectives for a purely computational matrix algebra course might be stated as
Learning ObjectivesTo successfully complete this course students must demonstrate the ability to
The student must demonstrate the ability to do these both by hand and by using a computer when appropriate. 
Note that other topics, such as matrix notation, vectors, row reduction, the span of a set of vectors, and possibly complex numbers, will have to be covered in class and might appear on the syllabus. The objectives only include the overall skills that students are supposed to take away from the course and that will be assessed. All of these could be tested in fairly traditional ways or tested in the context of more involved projects. In the language of standards, the content standards are in boldfaced type and the statements in regular type are performance criteria or indicators. The italicized statement at the bottom is a performance criterion that applies to all of the objectives. To complete this description, we should include examples of the types of problems we expect students to accurately solve.
The course described above is computational. What if we also feel that students should also understand the notions of a vector space, linear independence, a vector space basis, and dimension, even just as applied to subspaces of R^{n}? We would then need to add new learning objectives, but it makes little sense to have "understanding" as an objective without knowing how students will demonstrate their understanding. Should students be able to define, compute, manipulate definitions, derive easy consequences, give examples and counterexamples, and prove theorems? Should they be able to do more or is this too much too ask? We need to decide and make our decision explicit. A learning objective that captures some ways in which a student can demonstrate an understanding and says what a student knows upon successful completion of the course might look like:
Define, compute, and derive first consequences from the definitions of a vector space, basis, and the dimension of a vector space as applied to subspaces of R^{n}: State the definitions of vector space, basis, span, linear independence, and dimension; prove results that follow directly from these definitions; determine if a set of vectors is a basis for a given subspace of R^{n} ; compute bases and dimensions for vector subspaces of R^{n} defined by algebraic properties or as spaces associated with matrices; use dimension to compare subspaces and draw conclusions about the linear dependence or independence of a set of vectors. 
The last set of implications we will discuss deals with the manner in which students advance as they progress through the educational system. This involves a use of standards more familiar from technology than from education: the use of standards to define the interface and guarantee interoperability between two parts of a system.
College Admissions. For us, the first point at which two parts of the educational system meet is at the point of college admission. Admission standards have been in place for a long time. The traditional criteria used are high school grade point averages, subject requirements (also called Carnegie units), and standardized test scores (such as the SAT and ACT). The validity and biases of standardized tests is a more controversial issue, but there can be little doubt that grades and subject requirements do not mesh well with the standardsbased educational reform occurring in our nation's school systems. Grades are inherently normreferenced and subject requirements lose their relevance as progress through school is measured by means external to classes.
Regardless of the reason, the mathematical community simply does not believe that high school transcripts are adequate indicators of what students know and can do. Our evidence for this statement is the ubiquitous placement exam given to entering freshmen. It also seems clear that the data attached to students entering college in the near future will be new and different. For example, the new proficiencybased admissions system in the state of Oregon promises to provide different information on entering students. In that system students will need to demonstrate proficiency in a variety of content areas using public criterionreferenced standards.
Advising. Data coming out of standardsbased educational systems is new and different and will need to be integrated into our placement and advising systems. If it is sufficiently good, we may be able to eliminate placement exams. In Oregon, mathematics comprises seven of thirtythree individual proficiencies, so we might expect good data. By the same token, if more emphasis is placed on integrative abilities such as critical thinking and communication skills, we may have less data on specific content knowledge and be faced with an increased need for placementtype exams in mathematics or other areas.
Graduate Admissions. The second point at which two parts of the educational system meet is at the point of graduation. We have already discussed the notion of a standardsbased degree in mathematics, but we have not talked about what this means for graduate admissions. In our opinion, it means a lot of good things. If standards for mathematics majors and especially for mathematics courses are public and professionally viable, we should have a much easier time making admissions decisions and, more importantly, advising new graduate students as to what classes they need or do not need to take.
Prerequisites. The final point of interface the is internal interface between courses and their prerequisites. It seems logical that if we have standards for the completion of a course, we might also have the same type of standards for the commencement of a course. If courses have entrance standards then it should be possible to determine whether a student is ready for a course by checking that the entrance standards for the desired course are a subset of the objectives for the courses the student has successfully completed.
Naturally, some flexibility would be needed, but taking this point of view might accomplish some useful things. Currently, there is no way for a transfer student to know if the courses she has taken at one institution really satisfy the prerequisites for a course at a second institution. In fact, a faculty member often has trouble making this determination. Publicly defined standards for entering and completing courses would alleviate this problem. They are also significant for a future in which courses are available from multiple institutions via the Internet and in which students follow nonlinear career paths that have them stopping in and out of higher education.
Might any of this happen or is this all a theoretical exercise? We admit that we do not know. On the other hand, standardsbased reform is happening now in secondary schools, virtual universities are touting competencybased degrees, universities and professional programs are being asked to define standards for learning outcomes, and other forces at work in changing the nature of higher education point to standardsbased education. Our educational system will most certainly need to transform as we move from the industrial age into the information age and standardsbased education seems a good candidate for one such transformation. We therefore feel that the mathematical community should seriously consider its and be prepared for it if and when it comes along. We presume there will be no shortage of debate and differing points of view. We hope that we have started that debate on a positive note by emphasizing some of the more interesting and potentially helpful aspects of standardsbased education.
1  In 1893 the Committee on Secondary Schools Studies (better known as "the committee of ten") was chaired by Charles W. Eliot, president of Harvard University and had five university professors among its members. Its work defined what we would now recognize as a standard academic high school curriculum. The Carnegie Foundation for the Advancement of Teaching established the Carnegie unit in 1914 "when colleges began to worry about how to assess high school transcripts" and the Educational Testing Service was established in 1947 by the College Board (established in 1900), the American Council of Teachers, and the Carnegie Foundation with impetus coming from several Ivy League colleges. See http://www.carnegiefoundation.org/history.htm. 
2  See Albjerg Graham, Richard W. Lyman, and Martin Trow , Accountability of Colleges and Universities, Columbia University Office of the Provost, Columbia University, New York, New York. Available online at http://www.columbia.edu/cu/provost/acu/index.html for a discussion of accountability and Strategic Planning for Tulane: Executive Summary (August, 1988) by the Tulane University Strategic Planning Framework Committee, available online at http://www.tulane.edu/~strplan/scan.htm, for a discussion of all of these factors. The many sides of globalization and the Internet can be seen by looking at the online summary notes from the American Association of College Registrars and Admissions Officers 1997 Virtual Learning Environments: World Conference and Summit available online at http://www.merit.edu/~lmp/aacraorep.html. 
3  See Standards: A Common Language at http://www.mcrel.org/standardsbenchmarks/docs/chapter3.html. 
4  Information on proposed (but unfunded and unimplemented) voluntary national mathematics tests may be found at http://www.nagb.org/. 
5  See, for example: Lynn Olson, Achievement Gap Widening, Study Reports, Education Week, Dec. 4, 1996, and Kathleen Kennedy Mannzo, Weighted Grades Pose Dilemmas in Some Schools, Education Week, June 17, 1998. 
6 
Some online references that illustrate this are:
and many more. 
7  In 1997 the American Federation of Teachers reported that every state with the exception of Iowa and the District of Columbia either had or were planning to implement state standards. A more recent AFT report with qualitative analyses of state standards is summarized in Making Standards Matter 1998 available from the AFT Web site at http://www.aft.org/edissues/standards98/index.htm. 
8  Higher education is being called upon to serve an everincreasing portion of the population. According to the Census Bureau, in 1940, 24.5 percent of the population over the age of 25 had completed four years of high school or more. By 1996 this had risen to 81.7 percent. In the same group, 4.6 percent had completed four years of college in 1940 and 23.6 percent had completed four years of college in 1996. In his 1997 State of the Union speech President Clinton listed as the one of his ten principles in his "Call to Action for American Education" that "we must make the 13th and 14th years of education  at least two years of college  just as universal in America by the 21st century as a high school education is today, and we must open the doors of college to Americans." 
9  There is at least one example that has been around for over 25 years, Alverno College, a small urban catholic women's college in Milwaukee, Wisconsin. Some writeups, together with some research about the effectiveness of their "abilitybased program", can be found in the American Association for Higher Education Bulletin, Volume 36, Number 6, published February, 1984 by the AAHE, Washington, DC. Other information is available directly from http://www.alverno.edu/educators/educators.html. 
10  Such experimentation is occurring on the authors' campuses, and a few hours of Web browsing turned up truly standardsbased courses in writing, psychology, health sciences, and other disciplines at a number of universities. The work that is probably most significant to mathematics is the work being done in Engineering as a result of new accreditation criteria being used by the Accreditation Board of Engineering and Technology (ABET). ABET 2000 is outcomesbased and represents a radical change from previous curriculum based criteria. See http://www.abet.org/eac/eac.htm and How Do You Measure Success? Designing Effective Processes for Assessing Engineering Education. (1998). Professional Book, American Society for Engineering Education, Washington, DC. 
11  When asked to produce learning objectives, faculty typically phrase the objectives in terms of what a student should "learn", "understand", "appreciate", and so on. These are good descriptive terms and may even be somewhat welldefined among professional colleagues, but how useful are they without a clearly defined notion of how the student is supposed to demonstrate understanding, learning, or appreciation? How many times have we encountered a student who did poorly on the test but was nonetheless confident that he or she had understood, learned or appreciated the material? 
Robby Robson is an associate professor of mathematics and the university education reform coordinator at Oregon State University . He is working on a standardsbased university admissions policy and supporting faculty efforts at standardsbased education across the campus. His mathematics tends to be algebraic in nature and his current other passion is Webbased pedagogy.
M. Paul Latiolais is a professor of mathematics at Portland State University. His mathematical research focus has been in Algebraic Topology/ Combinatorial Group Theory. More recently he has been studying systemic change in Higher Education, as well as working on an environmental Statistics book and trying to figure out how to teach proficiencybased courses.
Both Paul and Robby have worked extensively with the Proficiencybased Admissions Standards System (http://passous.uoregon.edu) in writing mathematics standards for college admissions and in experimenting with standardsbased approaches in their own classes.
You can reach Paul and Robby by email at paul@math.pdx.edu and robby@orst.edu.
MAA Online is edited by Fernando Q. GouvÃªa (fqgouvea@colby.edu).