1. What is assessment?
2. Why is assessment done?
3. Why is assessment called assessment?
4. What is the difference between assessment and evaluation?
5. How does assessment differ from testing and grading?
6. How does assessment differ from measurement?
7. How does one get started with assessment?
8. Where can I find descriptions of assessment programs?
9. What are examples of assessment programs that are carried out by many mathematics departments?
10. Does it help to know about assessment in disciplines other than mathematics?
11. How do you finance an assessment program?
12. What is classroom assessment?
13. What is summative assessment?
14. What is formative assessment?
15. What is alternative assessment?
16. What is large-scale assessment?
17. What is outcomes-based assessment?
18. What is an assessment cycle?
19. What could cause an assessment cycle to be short-circuited? Are there any epicycles?
20. When is assessment finished?
21. How does one establish learning goals?
22. How does one decide what assessment tools to use?
23. What data should be gathered and what kinds of records should be kept?
24. What do you do with the data from assessment?
25. What are common uses of assessment data?
26. How do you get your colleagues to help with assessment?
27. What should students know about assessment programs?
28. What should faculty members know about assessment programs?
29. What is a rubric?
30. Where can I find information on constructing rubrics?
31. Are there example rubrics available?
32. What is the difference between a learning objective and a learning outcome?

What is assessment?

Assessment can apply to various components and activities of colleges and universities. As used in SAUM, assessment applies to student learning and is comparing student learning with the learning goals of an academic program or curricular block of an academic program.

The MAA's CUPM guidelines on assessment define assessment as the "process of gathering and interpreting information about student learning." One source* expands this definition and emphasizes assessment's roots in program evaluation and improvement: "as a systematic process of gathering, interpreting, and using information about student learning, assessment is a powerful tool for educational improvement."

Both of the above statements emphasize the fact that assessment can be applied to both individual students and to academic programs.

*American Association Of Higher Education (AAHE), Principles Of Good Practice For Assessing Student Learning, Washington DC: AAHE, 1992.



Why is assessment done?

Improving student learning -- for both current and future students -- is the best reason to do assessment. However, assessment is often done to satisfy some mandate from within an institution or from some governing or accrediting unit outside the institution



Why is assessment called assessment?

The process that is called assessment in higher education had early roots at Alverno College. An account* of those early developments of assessment contains the following.

"Though the word assessment did not emerge from classroom or campus, it derives from an idea important to educators -- that of sitting down beside or together (from late Latin ad+sedere). In the seventeenth century an assessor was one 'who sits beside' or 'who shares another's position.' Early uses of the word focused primarily on determining the worth or value of something in monetary terms, but underlying those uses was the idea of expert judgement made on the basis of careful observation. 'Assessment' was thus a word destined for the tongues of educators -- whether humanists or scientists."

For a lighter look at this issue, see "Assessment: The Burden of a Name" by Bernard L. Madison.

*Loacker, Georgine, Lucy Cromwell, and Kathleen O'Brien, "Assessment in Higher Education: To Serve the Learner," from Assessment in Higher Education, edited by Clifford Adelman, Office of Educational Research and Improvement, U. S. Department of Education, Washington, DC, 1986.



What is the difference between assessment and evaluation?

Evaluation and assessment are sometimes thought of as the same, but there are differences when applied to individual students or academic programs. Evaluation of students is often by grading and may use the results of assessments. Similarly, the results of assessments can be used as part of evaluations of programs. Evaluation is typically a broader concept than assessment as it deals with all aspects of a program including resources, staffing, organization, operations, and efficiency. Assessment is typically used to describe processes used to examine the student learning that results from academic programs. Assessment is an ongoing process aimed at improving student learning, more so than evaluation, which is usually a final (or summative) result. Improving learning from assessment sometimes occurs immediately in classrooms or later because of changes for future students.

For a discussion of this and related issues, see Ewell, Peter, "An Emerging Scholarship: A Brief History of Assessment" in Building a Scholarship of Assessment (Banta, Trudy W. et. al.). San Francisco: Jossey-Bass, 2002.



How does assessment differ from testing and grading?

Testing is a form of assessment, usually done in classrooms in individual courses. Grading is a form of evaluation and may use the results of assessment. The reverse is also true: assessment may use both test results and grades. Assessment often uses multiple methods, including testing, to allow students to demonstrate what they have learned and how they can use the knowledge. Assessment can address student learning in more than a single course, which testing and grading could do, but most testing and grading focuses on a single course.

Assessment also emphasizes consistency of judgment across individual students in a way that grading does not because grading is done by individual faculty members, each having his or her own standards.



How does assessment differ from measurement?

Very often in education one speaks of testing and measurement. Measurement likely refers to a process with a numerical or quantitative result while assessment is likely to yield a broader description that includes narrative analysis.

Here is how one source* contrasts assessment with measurement. "When we narrow testing to measurement, it answers the question "How am I doing?" with a quantitative response that says, "You did a certain percent of what was asked on a given occasion" or "You did as well as a certain percent of all those who tried or might try to do the same." Assessment answers the question with a descriptive account of precisely what the individual person has done on a given occasion.

*Loacker, Georgine, Lucy Cromwell, and Kathleen O'Brien, "Assessment in Higher Education: To Serve the Learner," from Assessment in Higher Education, edited by Clifford Adelman, Office of Educational Research and Improvement, U. S. Department of Education, Washington, DC, 1986.



How does one get started with assessment?

Start small and grow. Very often, successful assessment programs grow from classroom assessments by a single faculty member. Such efforts often lead to questions about how to improve student learning, and these questions provide ways to interest fellow faculty members. Remember, assessment is answering the student's questions: "How am I doing?" and "How can you help me to do better?"

It is also useful to inventory all of the information that you already have about students and their learning that has not been used before, and by carefully examining your curriculum for opportunities where additional information about students and their learning might be collected. Sometimes assessment merely involves keeping records systematically and then regularly taking time to reflect on their significance.

Sometimes, assessment programs begin with an external mandate that prompts creation of a structure for an entire institution. This approach often results in assessment activities that are separate from teaching and learning activities.



Where can I find descriptions of assessment programs?

The best source for descriptions of assessment programs in undergraduate mathematics is the SAUM website, http://www.maa.org/saum, where you will find numerous case studies of assessment programs, including more than seventy in MAA Notes #49, Assessment Practices in Undergraduate Mathematics. The bibliography on this site lists many sources that describe a single or multiple assessment programs or processes.

The following* consists of 82 documented cases of successful applications of assessment in a variety of disciplinary and campus settings, presented in a common format; cases are cross-referenced according to a number of topical variables to enable them to be compared. Other references are given in the bibliography on the SAUM website.

* Banta, Trudy W.; Lund, Jon P.; Black, Karen E.; and Oblander, Frances W. (1996). Assessment in Practice: Putting Principles to Work on College Campuses. San Francisco: Jossey-Bass.



What are examples of assessment programs that are carried out by many mathematics departments?

A college mathematics placement program is an assessment aimed at determining how entering students can use their school mathematics in college mathematics courses. Tools in such assessments include placement tests, high school records, student interviews, and standardized test scores (e.g. SAT or ACT).

Another example of a multi-dimensional comprehensive assessment activity is associated with most doctoral programs in mathematics. Assessing the learning of doctoral candidates usually employs multiple measures, including faculty and student interactions, seminar presentations, written and oral comprehensive examinations, and a major capstone experience -- the dissertation. Grades in courses are a part of this assessment, but usually not determining. The process often continues beyond the granting of the degree to include the scholarly publication record.



Does it help to know about assessment in disciplines other than mathematics?

Yes. The fundamental ideas and purposes of assessment are similar for most academic disciplines. Very often, assessment practices in one discipline are adaptable for other disciplines. Sometimes, assessment data gathered in other disciplines is relevant to assessment in mathematics. This is especially true when assessing general education. In mathematics, for example, the kinds of rubrics originally developed by English departments can be adapted to help judge student performance in writing about mathematics or in delivering oral presentations.



How do you finance an assessment program?

Assessment should be financed as part of the instructional program. Since organized assessment programs are relatively new to most academic programs, financing must be added to or carved out of existing resources. Financing that is clearly designated for assessment helps keep efforts focused.

Some campuses have found that small mini-grant programs can really help when launching an assessment program. Recipients of such grants can undertake a small project or attend a conference, then report back to their peers about lessons learned. Even small amounts of money will signal an administration's commitment, which is often critical in getting started.



What is classroom assessment?

Classroom assessment is assessment of student learning in an class during the class meeting time, undertaken by individual faculty to help improve their own teaching. See Angelo, Thomas. A. and Cross, K.P. (1993). Classroom Assessment Techniques: A Handbook for College Teachers, 2nd ed. Jossey-Bass, San Francisco, 1993. This book describes and provides samples of fifty in-class techniques that instructors can use to determine student reactions to teaching and engagement with classroom learning and activities.



What is summative assessment?

Summative assessment is assessment aimed at determining the learning outcomes of an academic program at the end of the program or at the end of a particular phase of the program. It is used to inform periodic "high stakes" judgments such as whether a student should move on or receive a credential, or whether a given academic program should be discontinued or awarded a mark of quality. This should be viewed in contrast with formative assessment, which is assessment to feed back into an ongoing program in order to improve it. Summative assessment can be undertaken by outside authorities, while formative assessment is usually undertaken by a program's own faculty.



What is formative assessment?

Formative assessment is assessment that provides feedback into an on-going academic program to be used to modify the program to improve student learning. This should be viewed in contrast with summative assessment which is used to inform periodic "high stakes" judgments such as whether a student should move on or receive a credential, or whether a given academic program should be discontinued or awarded a mark of quality.


What is alternative assessment?

Alternative assessment usually means assessment that does not use the usual tools of paper and pencil testing. An example might be a group presentation by students centered on their approach to a particular problem, judged by faculty using a pre-designed rubric or scoring guide.



What is large-scale assessment?

Assessment can be large-scale because it assesses learning of many students in an academic program that is usually more extensive that a single course. Large-scale assessment is often accomplished by assessing the learning of a sample of students in a program.

The actual term, large-scale assessment, is usually applied in k-12 settings to standardized testing programs designed to be part of evaluating the effectiveness of schools or districts. Large-scale assessment is virtually always "summative" assessment.



What is outcomes-based assessment?

Outcomes-based assessment focuses on where the student winds up, that is, what has been learned. Most assessment programs include outcomes-based assessment but also note what processes occur along the way. In an assessment program, one compares the outcomes with the learning goals and reflects on processes that might be changed so that more learning results.



What is an assessment cycle?

An assessment cycle is a sequence of steps or phases of an assessment process that is repeated once the sequence is finished. A simple version of such a cycle is answering three questions: What do we want our students to learn? How well are they learning it? What should we change so that they will learn more?
For a more detailed assessment cycle, see the CUPM Guidelines on http://www.maa.org/SAUM.



What could cause an assessment cycle to be short-circuited? Are there any epicycles?

At any point in an assessment cycle, there may be a need to modify a previous step before completing the cycle. For example, if you determine that a learning goal is flawed, you may not want to measure progress toward that goal. Return to the goal-setting stage and fix that before proceeding. Similarly, many have found that the process of more precisely defining learning goals in itself suggests changes in curriculum. For example, simply setting the explicit goal that students should be able to explain a concept in mathematics to a non-mathematician raises the question of where this skill is actually taught in the curriculum and where students get a chance to practice it.

In situations like the one above, one has smaller cycles -- epicycles -- inside the larger assessment cycle. These epicycles may not need to be repeated in future trips through the assessment cycle.



When is assessment finished?

Never. Assessment is an integral part of teaching and learning. Assessment for a particular purpose, say summative assessment at the end of a course for the purpose of evaluating students, is finished when the purpose is met. However, assessment for the purpose of program improvement and enhanced student learning should be an on-going process.



How does one establish learning goals?

The faculty who have stewardship of an academic program should articulate the goals of the program with due consideration for the program's clients and sponsors. Very often, an initial list of goals is long and overlapping. Creating that initial list is usually not difficult. Paring the list and making it efficient, understandable, and realistic usually requires considerable discussion and thought. As goals are developed, thought should be given to how progress toward the goals will be measured and what curricular and extracurricular strategies will be employed.



How does one decide what assessment tools to use?

Assessment tools should be chosen so that they will provide students with good opportunities to demonstrate their learning. For example, if students are expected to be able to communicate with others about mathematics, then the assessment tool should allow them to demonstrate communication. Consequently, the tool could be an interview, an essay, or an oral presentation. If students are expected to know certain derivatives, then the tool might be a written test.

One way to summarize the above is that assessment tools should promote valid inferences. In addition, assessment tools should be practical to use, balanced with their ability to measure what is worth measuring, not simply what is easy to measure.



What data should be gathered and what kinds of records should be kept?

The results of the application of any assessment tool should be recorded. Examples include test scores, faculty judgements of student portfolios, and student self-assessments. All data should be carefully identified as to purpose, date, and populations involved. Interpretations of data and any actions take as a result of those interpretations should be recorded. Complete and thorough records that can be used over time are essential. Much of assessment merely involves keeping records systematically and then regularly taking time to reflect on their significance.

One of the six guiding principles of assessment in the CUPM Guidelines is: "Data should be collected for specific purposes determined in advance, and the results should be reported promptly."



What do you do with the data from assessment?

You interpret assessment data in the context of learning goals and objectives and make curricular decisions about changes that the interpretations imply. If the data are to be used for evaluations or accountability, then they should be provided to the appropriate entity.

Any sharing of assessment data should respect relevant legal and privacy issues.

One of the six guiding principles of assessment in the CUPM Guidelines is: "Data should be collected for specific purposes determined in advance, and the results should be reported promptly."



What are common uses of assessment data?

The most important use is for improving academic programs and enhancing student learning. Other uses are in evaluation of students, programs, and institutions, often for accountability or accreditation.



How do you get your colleagues to help with assessment?

Developing good assessment tasks that truly reflect the broad goals of mathematics is difficult work that requires mathematical creativity. Further, assessment documents student progress toward those goals. Getting colleagues to understand this and that assessment is a part of learning and teaching will go a long way toward recruiting them to help.

Faculty members should know why assessment is done, how the results will be used, and that they are stewards of the process. Uses of assessment results for program or institutional evaluation and accountability should be viewed as secondary, much as analogous uses of faculty publications and grants are viewed.



What should students know about assessment programs?

One of the five principles of assessment in the CUPM Guidelines is: "Students and faculty should be involved in and informed about the assessment process, from the planning stages throughout implementation." This implies that students should know the purposes, the processes, and how the results will be used.



What should faculty members know about assessment programs?

One of the five principles of assessment in the CUPM Guidelines is: "Students and faculty should be involved in and informed about the assessment process, from the planning stages throughout implementation." This implies that faculty should know the purposes, the processes, how the results will be used, and that they are stewards of assessment.



What is a rubric?

Used in the context of assessment, rubric (often scoring rubric) refers to a scoring guide for some demonstration of student learning.

From Mathematics Assessment: A Practical Guide for Grades 9-12, Reston, VA: NCTM, 1999, "For practical purposes, we will define a rubric to mean a hierarchy of standards used to score student work. … Well-designed rubrics allow students to see descriptions of the requirement of their performance."

Rubrics are sometimes divided into two types: holistic and analytic. Holistic rubrics often use a scale such as 1 - 5 to capture the overall quality of a work while analytic rubrics assign values to parts of a work.

Rubric comes from Latin rubrica meaning red earth and Middle English rubrike red ocher, heading in red letters of part of a book. Rubric can refer to a title, heading or directions for a book or manuscript that is printed in red or otherwise distinguished from the rest of the text. The meaning of rubric consistent with its use in assessment is any rule of conduct or procedure.



Where can I find information on constructing rubrics?

Some of the case studies in MAA Notes #49, Assessment Practices in Undergraduate Mathematics have information about rubrics. The National Council of Teachers of Mathematics (NCTM) publishes two books on assessment that include information on and examples of rubrics.

Mathematics Assessment: Cases and Discussion Questions for Grades 6-12, Reston, VA: NCTM, 2000.
Mathematics Assessment: A Practical Guide for Grades 9-12, Reston, VA: NCTM, 1999



Are there example rubrics available?

Yes. Some can be found in the following:

MAA Notes #49, Assessment Practices in Undergraduate Mathematics (http://www.maa.org/saum/MAANotes49/index.html)

Mathematics Assessment: A Practical Guide for Grades 9-12, Reston, VA: NCTM, 1999

The AP Program of the College Board creates rubrics for all free-response items on the annual AP examinations. The scoring rubrics for the AP Calculus free-response items (9-10 per year) are analytic rubrics and are released each year after the examinations are scored. Some of the AP courses use holistic rubrics. (See http://www.apcentral.collegeboard.com/)



What is the difference between a learning objective and a learning outcome?

When used in the context of assessment, learning goal and learning outcome are usually synonymous, but more general than learning objective. A learning goal may be very general. For example, a learning goal of the mathematics major might be that graduates will be able to apply mathematics to solve real world problems. In order to measure progress toward that goal and to design curricular strategies for achieving the goal, objectives need to be developed. One such objective might be to use definite integrals to model real world problems. That objective might be reached early in the mathematics major, and would be a step along the way to reaching the general goal.

Goal, objective, and outcome are used in different ways in different contexts and by different people. Consequently, making precise distinctions is difficult.

If you have any questions or comments please direct them to Dr. Bernie Madison