Question 2. Proof, Validation, and Trains of Thought

RESEARCH QUESTION

2. Proof, Validation, and Trains of Thought

September 19, 2000

by Annie and John Selden

When a mathematician reads a proof of a theorem in order to be certain it contains no errors and actually proves the theorem, he/she typically does much more than read in the ordinary sense. Questions are asked and answered. Some of these are explicit in that they are clearly articulated in inner speech. Others might be outside her/his main focus of attention and consist of recognizing a familiar situation and judging whether it has been properly handled. For example, when something is to be proved about every number x and a portion of the proof begins "Let x be a number . . . ," x should neither have occurred before nor depend on anything that has occurred before. Most mathematicians would notice this, but not focus on it or explicitly ask themselves (in inner speech), "Is x arbitrary?" However, any hint that x might not be arbitrary, would lead to a careful examination of that portion of the proof. In addition to asking and answering questions, a mathematician might recall definitions and theorems used in the proof, and their application to the proof might be examined. Occasionally, they also use visualization or construct entire subproofs.

This process may be entirely mental, and hence not directly observable by someone else. Furthermore it seems to require considerable focus of attention and much short-term memory (Baddeley, 1995). Thus even a mathematician's self-observations regarding details may be unreliable and fade quickly from memory.

We think most mathematicians are familiar with this process and clearly distinguish it from ordinary reading, but they may not have a name for it. We call it validation. When one attempts to articulate the validation process explicitly, the result is often much longer than the original proof. We have done this for a short calculus proof (see Appendix I, Selden and Selden, 1995).

Because validating a proof for oneself appears to be an essential part of constructing it, we view validation as part of the implicit curriculum. Furthermore, the reasoning involved appears to be similar to that required for checking the correctness of problem solutions. Moreover, problem solving is emphasized in the NCTM Standards for school mathematics. Thus, acquiring the ability to reliably validate proofs should be an important part of the education of preservice teachers, especially secondary teachers.

There are a number of interesting, uninvestigated questions regarding validation. Here we focus on just one. Simply put: When does a mathematician stop reading? When is he/she satisfied that the proof is correct? The validation process may end when the validator becomes conscious of a "feeling of understanding and correctness." However, some mathematicians do not experience this, and do not conclude the validation process, until they have reviewed the entire proof in a single train of thought. They will go over the proof from the beginning several times until they are satisfied. We mean by a single train of thought that the validator focuses on the proof, or a supplemented version of it, in an uninterrupted way from beginning to end. Activities such as constructing subproofs or recalling complex definitions could break the train of thought and lead to starting over. However, a familiar, more or less automated action, such as drinking a cup of coffee, probably would not.

Our conjectured explanation for this phenomenon is that the validator is trying to be sure there are no errors in various, possibly overlapping, "chunks" of the proof. This often cannot be done in a simple linear reductive way, e.g., dismissing the first half of the proof as finished and requiring no further attention, because although some possible errors are restricted to a small part of the proof, others are more global in nature. We expect this process would require almost all of one's short-term memory. A break in one's train of thought could put an additional, and unnecessary, burden on short-term memory. This would increase the chance of errors of omission, i. e., neglecting to check something. We are not suggesting that mathematicians consciously justify their personal style of validating proofs according to this explanation. Rather we suggest that through experience they may come to conclude their validations by reviewing proofs in an unbroken train of thought because this provides increased reliability.

Question I. Do many mathematicians commonly end validations as we have described?

A partial answer might be obtained by interviewing one, or more, collections of mathematicians. By describing them and the results of the interviews, "intellectual group portraits" could be produced. For example, one might interview the mathematicians of a small department, or those that are producing research papers, or those that teach courses requiring them to evaluate student proofs. In order to get independent information from such mathematicians it might be good if they did not discuss validation before the interviews. Also, as subjects occasionally shift their comments towards what they think an interviewer expects to hear, it might be helpful to avoid describing the exact purpose of an interview until its conclusion. Given the opportunity, mathematicians might even spontaneously describe the way they conclude validations and this could be followed up in subsequent interviews. Finally, because fine details of one's own thinking are hard to notice or remember, it might be helpful to ask each mathematician to validate a proof shortly before the interview.

Question II. Assuming mathematicians conclude their validations with an unbroken train of thought, do students at various levels do something similar?

This question might also be partially answered using interviews, perhaps starting with middle-level undergraduates at the end of a transition course. A "think aloud" validation might also be useful, but the length or complexity of the proof might influence whether multiple passes through the proof were deemed necessary.

Question III. Does concluding a validation by reviewing the entire proof in an unbroken train of thought provide more reliability? Does a mathematician or student who concludes a validation in this way find more errors?

One approach to partially answering this question would be to collect a set of purported "proofs" containing a variety of errors, and then to search for students who can find differing numbers and kinds of these errors. It might turn out that students who conclude their validations with an unbroken train of thought find more errors, or more errors of a certain kind. The students should be fairly accomplished and the "proofs" moderately long or complex because there are many situations, unrelated to this, in which inexperienced students fail to find errors. Another approach would be to have students do think aloud validations and look for errors they discover when reviewing the proof from beginning to end. Such errors might not have been found had such a review not been undertaken.

The explanation conjectured above is reminiscent of some recent observations made by those studying speech-driven computer interfaces. Word-processor users (i.e., typists) can often work considerably faster when giving voice commands such as "italic," than when using a mouse. However, when asked to reproduce some mathematical symbols, one group of typists was slower and had to look back more often when asked to view the symbols, say "page down" to the computer, and then type the symbols. Speech, it turns out, is a heavy user of short-term memory. Even speaking two words interfered with remembering the mathematical symbols (Schneiderman, 2000). Since much of validation involves inner speech, one might similarly expect an (inner speech) interruption in one's train of thought to interfere with short-term memory and make a review of earlier work necessary.

The ideas of trains of thought and other aspects of consciousness have been investigated (James, 1910), but largely neglected for some time. Psychologists, in particular, seem to have regarded such ideas as difficult to investigate, but they appear to be amenable to the kinds of qualitative techniques used in mathematics education research. For an example of a more recent paper, including a discussion of "fringe," i.e., the phenomena in consciousness that are not focused upon, see Mangan (1993).

References

• Baddeley, A. (1995). Working memory. In M. S. Gazzaniga (Ed.), The Cognitive Neurosciences (pp. 755-764), Cambridge, MA: MIT Press.

• James, William. (1910). The Principles of Psychology. New York, NY: Holt.

• Mangan, B. (1993). Taking phenomenology seriously: The "fringe" and its implications for cognitive research. Consciousness and Cognition 2, 89-108.

• Selden, J. and Selden, A. (1995). Unpacking the logic of mathematical statements. Educational Studies in Mathematics 29, 123-151.

• Schneiderman, Ben. (2000). The limits of speech recognition. Communications of the ACM, 43:9, 63-65.

Return to the list of Research Questions or the Research Sampler column