Reviewing Education Research Papers

Why Mathematics Education Research Papers
are Accepted or Rejected

by Annie and John Selden

November, 1998

Return to the Bibliography or the Research Sampler column.

This annotated bibliography provides information on what mathematics education journal editors and reviewers (i.e., referees) look for in research papers. It was prepared as a supplement to our talk "Where's the Theorem? Where's the Proof? An Analysis of Why Math Ed Research Papers Get Rejected" given at the RUMEC Conference on Research in Mathematics Education, South Bend, Indiana, September 1998.

Donmoyer, R. (1996). Educational research in an era of paradigm proliferation: What's a journal editor to do? Educational Researcher, 25(2), 19-25.

Musings of a newly-appointed journal editor regarding the wide variety of research paradigms used in education today and the fact that peer reviews often provide conflicting recommendations.
Hanna, G. (1997). Evaluating research papers in mathematics education. In A. Sierpinska & J. Kilpatrick (Eds.), Mathematics Education as a Research Domain: A Search for Identity, An ICMI Study, Vol. 2 (pp. 399-407). Dordrecht, The Netherlands: Kluwer Academic Publishers.

Written by an editor of Educational Studies in Mathematics, this chapter gives questions reviewers are often asked to address. E.g., What was the theoretical framework? How is this paper related to others? What does it add? Was the data gathering systematic? Was the analysis appropriate? Do the conclusions follow? Is this paper likely to interest readers? Of special interest are quotes taken from actual reviews regarding originality (highly valued), usefulness, readability, etc.
Hart, K. (1997). Basic criteria for research in mathematics education. In A. Sierpinska & J. Kilpatrick (Eds.), Mathematics Education as a Research Domain: A Search for Identity, An ICMI Study, Vol. 2 (pp. 409-413). Dordrecht, The Netherlands: Kluwer Academic Publishers.

Written by the director of the Shell Center, this chapter gives and elaborates on a PME (Int'l Group for the Psychology of Mathematics Education) discussion group's list of essential and minimum criteria for research (a.k.a. disciplined inquiry): 1. There is a problem. 2. There is evidence/data. 3. The work can be replicated. 4. The work is reported. 5. There is a theory.
Henson, K. T. (1997). Writing for publication: Some perennial mistakes. Phi Delta Kappan (June 1997), 781-784.

While very general, this article has some good advice: Resist the temptation to send your manuscript to the most prestigious journal. Inquire about a journal's acceptance rate, backlog, and turnaround time. Consider submitting to theme issues (where competition is less intense). Don't automatically assume all refereed journals are better than all nonrefereed journals. Adhere to the journal's specific reference style (often APA for education journals). Do revise and resubmit - chances of acceptance are much greater the second time round.
Lester, F. K., Jr., Kehle, P. E., & Birgisson, G. (1996). In pursuit of practical wisdom in mathematics education research. A revised version of a paper presented by the first author at the NCTM research presession at the annual meeting in San Diego on April 23, 1996. (Available from Frank K. Lester, Jr., Faculty of Education, Indiana University, Bloomington, Indiana.)

While ostensibly about communication between researchers and practitioners, this paper also considers how claims are justified in mathematics education research. It points out that data do not speak for themselves, that the researcher's assumptions should be made clear, and there should be a reasoned argument from (both of) these to the conclusions drawn.
Lester, F. K., Jr. & Lambdin, D. V. (1997). The ship of Theseus and other metaphors for thinking about what we value in mathematics education research. In A. Sierpinska & J. Kilpatrick (Eds.), Mathematics Education as a Research Domain: A Search for Identity, An ICMI Study, Vol. 2 (pp. 415-425). Dordrecht, The Netherlands: Kluwer Academic Publishers.

Written by a former editor of JRME, this chapter discusses criteria useful in evaluating "all aspects of the research process (conceptualization and design, question formulation, conduct of the study, data analysis and interpretation of reports, etc.)." These include: worthwhileness, coherence, competence, openness, ethics, credibility, as well as originality, conciseness, and connections with existing research.
Smith, M. U., Wandersee, J. H., & Cummins, C. L. (1993). What's wrong with this manuscript?: An analysis of the reasons for rejection given by Journal of Research in Science Teaching reviewers. Journal of Research in Science Teaching, 39(2), 209-211.

Twenty rejected manuscripts received during 1990 by JRST were randomly selected and analyzed using content analysis. Major reasons for rejection included: poor research design, weak literature review, and weak discussion/implications. An additional 36 manuscripts were rejected outright by the editor without going to reviewers for the following reasons: too general and not related to science, not research, and the theory base was missing.