Figure 1. A proposed solution evaluated using the analytic scale.
Once you've assigned a writing project and collected the papers, how are you going to grade it without spending the rest of the semester on that one set? This article outlines two grading scales which can make this more efficient.
Background and Purpose
Projects that require students to model a real-life situation, solve the resulting mathematical problem and interpret the results, are important both in enabling students to use the mathematics they learn and in making the mathematics relevant to their lives. However, assessing modeling projects can be challenging and time-consuming. The holistic and analytic scales discussed here make the task easier. They provide the students with a set of guidelines to use when writing the final reports. They also provide me with a structured, consistent way to assess student reports in a relatively short period of time.
Currently I use these scales to assess projects given in precalculus and calculus classes taught at a two-year branch of a university. I have used these scales over the last seven years to assess projects given in undergraduate as well as graduate-level mathematics courses. The scales have worked well at both levels.
Method
I provide each student with a copy of the assessment scales the first class meeting. The scales are briefly discussed with the students and care is taken to point out that a numeric solution without the other components will result in a failing grade. I review the scale when the first group project is assigned. This helps the students to understand what is expected of them.
The analytic scale (p. 117, adapted from [1]) lends itself well to situations where a somewhat detailed assessment of student solutions is desired. When the analytic scale is used, a separate score is recorded for each section: understanding, plan, solution, and presentation. This allows the students to see the specific strengths and weaknesses of the final report and provides guidance for improvement on future reports.
The holistic scale (p. 119, adapted from [2]) seems better suited to situations where a less detailed assessment is required. It often requires less time to apply to each student report and can be used as is or adjusted to meet your individual needs and preferences. For some problems it might be useful to develop a scale that has a total of five or six points possible. This will require rewriting the criteria for each level of score.
Findings
One of the easiest ways to demonstrate how the scales work is to actually apply them. Figure 1 contains a proposed solution to the problem given below. The solution has been evaluated using the analytic scale.
The Problem: You want to surprise your little brother with a water balloon when he comes home, but want to make it look "accidental." To make it look accidental you will call, "Watch out below!" as you release the balloon, but you really don't want him to have time to move. You time the warning call and find that it takes 1 second. You have noticed that it takes your brother about 0.75 seconds to respond to warning calls and you know sound travels about 1100 feet per second. If your brother is 5 feet tall, what is the greatest height you could drop the balloon from and still be certain of dousing him?
Using the holistic scale on p. 119 to assess this proposed solution results in a score of "3a." The "a" is included to provide the students with guidance regarding the shortcoming(s) of the report.
Use of these scales, especially the analytic scale, has led to reports that are much more consistent in organization and usually of a higher quality. Students have repeatedly made two comments regarding the use of the scales. First they appreciate the structure. They know exactly what is expected of them. Second they are much more comfortable with the grade they receive. On more than one occasion students have expressed concern regarding the consistency of assessment in another class(es) and confidence in the assessment based on these scales.
Use of Findings
One issue I focus on is how well the students are interpreting what is being asked of them. After reading many reports where the students are first asked to restate the problem, it has become clear that I need to spend time in class teaching the students how to read and interpret problems. They may be literate, but they often lack the necessary experience to understand what is being described physically.
I also focus on the students' mathematical reasoning. In the problem presented above, a number of students run into difficulty developing an equation for the brother's reaction time. They will add the times for the warning call and the reaction while completely ignoring the time for sound to travel. Although it does not significantly alter the solution in this problem, it is important for the students to realize that sound requires time to travel. Prior to giving this assignment, we now spend time in class looking at situations where the time needed for sound to travel affects the solution.
Success Factors
When using the holistic scale, I include a letter (the "a" of "3a" above) to indicate which criterion was lacking. Students appreciate this information.
When new scales are developed it is well worth remembering that as the number of points increases so does the difficulty in developing and applying the scale. In the case of the holistic scale, it is often best to start with four points. As you gain experience, you may decide to develop more detailed scales. I have always found five or at most six points sufficient for holistic scales.
To avoid problems with one or two students in a group doing all of the work, I have them log the hours spent on the project. I warn the class that if a student in a group spends 20 minutes on the project while each of the other members averages four hours, that person will receive one-twelfth of the total grade. There has only been one case in seven years where this type of action was necessary.
A final note about using these scales. It often seems a huge task to evaluate the reports, but they go rather quickly once started. First, there is only one report per group so the number of papers is reduced to one-third of the normal load. Second, having clearly described qualities for each score reduces the time spent determining the score for most situations.
An analytic scale for assessing project reports (Portions of this are adapted from [1]. See Figure 1 for a sample.)
Understanding
3 Pts The student(s) demonstrates a complete understanding of the problem in the problem statement section as well as in the development of the plan and interpretation of the solution.
2 Pts The student(s) demonstrates a good understanding of the problem in the problem statement section. Some minor point(s) of the problem may be overlooked in the problem statement, the development of the plan, or the interpretation of the solution.
1 Pt The student(s) demonstrates minimal understanding of the problem. The problem statement may be unclear to the reader. The plan and/or interpretation of the solution overlooks significant parts of the problem.
0 Pt The student(s) demonstrates no understanding of the problem. The problem statement section does not address the problem or may even be missing. The plan and discussion of the solution have nothing to do with the problem.
Plan
3 Pts The plan is clearly articulated AND will lead to a correct solution.
2 Pts The plan is articulated reasonably well and correct OR may contain a minor flaw based on a correct interpretation of the problem.
1 Pt The plan is not clearly presented OR only partially correct based on a correct/partially correct understanding of the problem.
0 Pt There is no plan OR the plan is completely incorrect.
Solution
3 Pts The solution is correct AND clearly labeled OR though the solution is incorrect it is the expected outcome of a slightly flawed plan that is correctly implemented.
Figure 1. A proposed solution evaluated using the analytic scale.
2 Pts Solution is incorrect due to a minor error in implementation of either a correct or incorrect plan OR solution is not clearly labeled.
1 Pt Solution is incorrect due to a significant error in implementation of either a correct or incorrect plan.
0 Pt No solution is given.
Presentation
1 Pt Overall appearance of the paper is neat and easy to read. All pertinent information can be readily found.
0 Pt Paper is hard to read OR pertinent information is hard to find.
A Holistic Scoring Scale
(This is an adaptation of a scale from [2].)
4 Points: Exemplary Response
3 Points: Good Response
Exactly one of the following characteristics is present.
a The answer is incorrect due to a minor flaw in plan or an algebraic error.
b The explanation lacks clarity.
c The explanation is incomplete.
2 Points: Inadequate Response
Exactly two of the characteristics in the 3-point section are present OR
One or more of the following characteristics are present.
a The answer is incorrect due to a major flaw in the plan.
b The explanation lacks clarity or is incomplete but does indicate some correct and relevant reasoning.
c A plan is partially implemented and no solution is provided.
1 Point: Poor Response
0 Points: No Response
References
[1] Charles, R., Lester, F., and O'Daffer, P. How to Evaluate Progress in Problem Solving, National Council of Teachers of Mathematics, Reston, Va, 1987.
[2] Lester, F. and Kroll, D. "Evaluation: A New
Vision," Mathematics Teacher 84, 1991, pp.
276-283.
 |
||
 |
 |
 |
 |