What I Learned… about Online Assignment Management

Online assignment-management (OAM) systems allow instructors to create assignments or exams that are posted on the web, taken by students on their own time, and computer-graded to provide instant feedback for students and a record for instructors. Interest in these systems can be measured, perhaps, by the number of talks at mathematics meetings on issues related to their use.

The Department of Mathematics at the University of Nebraska-Lincoln uses on OAM system called EDU, from Brownstone Learning; however, most of these principles apply equally well to other systems. (EDU, as well as Maple T. A. and several publisher-hosted products associated with textbooks, is founded on common code architecture developed by Professor John Orr of our department.)

I have been using OAM long enough to compile an impressive list of mistakes and bad choices as well as a smaller list of good practices. In the hope that others might profit from my errors without having to repeat them, I have collected here what I believe are the ten most important principles for OAM use. Some of the items in the list are specifically for exams or for daily assignments, while others apply equally to both. Here, then, are the top ten things I’ve learned about Online Assignment Management:

10. Use matched sets of questions

When several different cases occur in a problem class, it is good to require students to work a problem from each case. An example would be two problems of the form “Find the (exact) x coordinate of the global minimum of f(x) = 3x³ + bx² + cx on [-1,1]," where b and c are chosen so that the global minimum occurs at a critical point for one of the questions and at an endpoint for the other one.

9. Use a mastery protocol

In a mastery protocol, students are required to successfully work one problem from each of several categories. This means that students repeat only those problems they have not yet worked correctly, and it also allows the instructor to set up question hierarchies (see item #5). This is qualitatively different from the case where a student must get a certain number correct to pass and has multiple attempts at passing. In the latter case, students who do not pass must repeat all of the questions on their next attempt. Mastery protocols do not work well for exams, but they are ideal for short assignments.

8. Choose the right material for online instruction

Online instruction is fine for routine computation and ideas that can be developed through examples, but it is no substitute for a live class meeting for the teaching of nuanced ideas, problem-solving strategies, or techniques in which individual steps need a separate presentation. The typical homework problem from a textbook is too complicated to be of instructional value as an all-or-nothing computer-graded problem.

7. Write good questions

Many of my students try to learn the material only when they have exhausted all other ways to pass an exam. It is good instructional practice to help students reach this point quickly. Randomized parameters allow for an enormous variety of answers to problems that look almost identical, eliminating any hope of passing an exam by memorizing the answers. Grouping problems by category has a more subtle effect. Many students who fall short of a passing mark on an exam will quickly repeat the exam, hoping to get an easier set of questions. The more closely different settings of an exam are related, the more likely students will come to appreciate that trying to get an easy version is hopeless. This desirable uniformity of appearance and difficulty comes from having an exam constructed by random selection of one problem each from several standardized categories.

6. Set high standards and allow retakes

Our standard educational system is built around one-shot exams. These allow us to categorize students by level of achievement, but they encourage students to work towards performance rather than learning (so that the goal is to complete assignments and get good grades rather than to learn the material), and may let them move on without mastering fundamental material. An alternative plan is to set high performance standards and require students to repeat an exam until they achieve the standards. This plan does not distinguish the stronger students from the weaker, but (see point #8) it is not necessary to do that on material that is well suited to online testing. Another objection is that allowing retakes can encourage students not to study for an exam. I solve this problem by basing the students’ grades on the date at which they pass the exam and limiting them to one attempt per day. Students have three days in which they can receive the full 30 points for passing my exam. After that, the number of points they get decreases by one each day. This policy is very effective at motivating students to pass the exam quickly.

5. Use a question hierarchy

Students learn best from success that build on previous success. Success rates can be improved by using a hierarchy of questions that allow students to progress from easy questions to hard ones. For example, consider the problem of finding the derivative of . Experienced students should find this problem to be straightforward; however, it does require students to successfully combine the quotient rule, the chain rule, and the formulas for the derivatives of the trig functions. Students who have just learned one of these components are not likely to be able to do this problem successfully. I use a hierarchy of problems in which students must first differentiate , , and before tackling the desired problem. The average student will probably have to work seven problems to get through the four-question hierarchichal assignment. The same student would probably also need seven tries to succeed at the corresponding single-question assignment, but she will have got only one question right in the process instead of four.

4. Encourage students to rework missed problems before trying an exam or assignment again

Students who take the time to analyze their mistakes show much greater improvement in their next attempt than students who don’t; however, few students use this seemingly obvious method of study. I suspect that the problem is the all-too-common performance orientation. To students whose goal is to complete the assignment rather than to learn the material, time spent studying is time not spent on the task of completing the assignment. Even students who have a learning orientation often have not developed good study techniques. It is worth spending a little bit of class time teaching students how to learn mathematics.

3. Use a short time window

Mathematics lectures generally build on previous material. The standard calculus course includes daily lectures, a small amount of daily or weekly homework, weekly quizzes, and monthly exams. Most students fall behind during the first week and only catch up when studying for an exam. This means that most lecture material is delivered to students who have not learned the background material from the previous class meeting. Online instruction has the potential to ameliorate this difficulty, provided assignments are timed so that each is due before the next class meeting. There is no harm in extending the short time window as needed, but the standard practice should be to require online work to be done in time to prepare the students for the next class.

2. Minimize instructor commitment

No matter how useful OAM is, most faculty are not going to use it if they perceive it as a large time commitment. I set up an administrative structure that utilizes EDU’s facility to have class folders created as copies of existing folders, with the daughter folders inheriting any materials kept in the parent folder. I have a master course folder that contains the question banks, the single online examination used by all sections, and the non-credit assignments used by students to practice their skills or study for exams. I also have an enhanced master course folder that is a copy of the master folder with the addition of a set of assignments-for-credit used by some of the course instructors. These course folders can be reused each semester just by changing the assignment dates. Each individual class has its own folder that is copied from the appropriate master folder. I find that a one-hour training session is sufficient to teach instructors how to do the only tasks that they must actually perform themselves: changing assignment dates, observing and re-grading student work, and downloading the gradebook.

1. Give minimal credit

Few of my students will take the time to do an assignment that does not count for credit, no matter how interesting or valuable the assignment. Yet the same students will do an inordinate amount of work for minimal credit. Last semester, I gave two points for each of my 30 web assignments, out of 600 total points for the course; my students completed 75% of the total number of assignments possible in the course. My colleague offered the same assignments for no credit, and his students completed only 2% of possible assignments. The credit I gave for the assignments had only a minimal direct impact on student grades; not a single student passed because of the availability of a few easy points. The indirect impact was much larger: my students were better prepared for class and were able to get more out of the limited class time available for the course.

Glenn Ledder teaches at the University of Nebraska-Lincoln. He can be reached by email at gledder@math.unl.edu.