You are here

DRILL 3.1 - Results and Limitations

Vadim Ponomarenko

Previous versions of DRILL offered only one exam, covering the topic of common algebra errors made by calculus students. It has been offered to students in first-semester calculus four times, dating back to 1998. Students were required to take the untimed exam at least once, and a small incentive was provided for successful completion. Students were free to take it wherever, whenever, and as many times as they wanted. Some students took advantage of this freedom to take the exam hundreds of times. Generally all exam-related activity took place within the first two weeks of the course.

A total of 145 students have taken that exam at least once, of which 68 completed it successfully. The others did not manage to receive a passing score, which was an unforgiving 100%. We distinguish here two groups of students -- those who went on to earn less than B in the entire course (moderate to poor calculus performance), and those students who went on to earn less than C (poor calculus performance). Naturally, we want to decrease the size of each of these groups, which would correspond to more students doing well. The following table shows aggregate data -- details are available upon request.

% of students who
passed the exam 
 % of students who
did not pass the exam
Earned < B 43 68
Earned < C 19 44

We can offer two explanations for the dramatic semester-long differences between groups that did or did not complete a single exam. Likely, both are true to some degree.

  • First, DRILL acted as a diagnostic tool. Though we did not do this during the four semesters of this experiment to avoid skewing the data, one could use the information provided by DRILL to identify weaker students and target them with additional help and resources.
  • Second, DRILL acted as an educational tool. Those students who completed the exam thoroughly reviewed their algebra skills, improved their weak points, and were fully prepared for the semester to come. Those who did not complete the exam found their algebra skills hampering them and fell behind. Though probably rather weak, this effect would be more pronounced in those students who spent ten or more hours taking and re-taking the exam.

DRILL still has a limited set of question models -- certain combinations of skills and objects are not fully implemented. By making additional question models, it could be useful in other courses as well.


Journal of Online Mathematics and its Applications