The following paper by Ken Houston was published in
The Teaching and Learning of Mathematics at University Level: An
ICMI Study, (Derek Holton, Editor), Kluwer Academic Publishers:
The Netherlands, 2001, pages 407-422. (ICMI is the International Commission
on Mathematics Instruction.)
The paper is posted here with kind permission of Kluwer
Academic Publishers, editor Derek Holton, and author Ken Houston. The
volume containing the paper is volume 7 of the New ICMI Study Series,
co-editors Bernard R. Hodgson (Canada) and Hyman Bass (USA).
Derek Holton (Ed.), The Teaching and
Learning of Mathematics at University Level: An ICMI Study, 407-422.
© 2003 Kluwer Academic Publishers. Printed in the Netherlands.
ASSESSING UNDERGRADUATE MATHEMATICS STUDENTS
Any discussion of assessment must necessarily include a discussion
of the curriculum, how it is designed and organised, and what it contains.
It must examine the aims of the course that students are taking, and
the objectives set for that course and the individual modules that comprise
the course. (Here I am using terminology common in the UK. The 'course'
students take is 'the whole thing', the 'programme'. A course in this
sense consists of 'modules' or 'units', commonly called 'courses' in
the USA, so beware of confusion!) The discussion must consider who is
doing the assessing, why they are doing it, what they are doing and
how it is being done. It must consider how assessors become assessors
and how those assessed are prepared for assessment. And it must consider
if the assessment is valid and consistent, and if it is seen to be so.
It might also be useful at this stage to define what we mean by a 'mathematician'.
There is a real sense in which almost everyone could be described as
a mathematician in that they make use of some aspect of mathematics
- be it only arithmetic or other things learnt at primary/elementary
school. The term could be used of those who have taken a first degree
in mathematics and who use it in their employment. Or it could be reserved
only for those who have a PhD and who are doing research in pure mathematics
or an application of mathematics. We will use the middle of the road
term. In other words, a mathematician will be one who has studied the
subject at least to bachelors degree standard (and of course that varies
across the world!), and who is using some aspect of advanced mathematics
in their work. Such people could join a professional or learned society
such as the UK based Institute of Mathematics and its Applications.
So we are primarily concerned with the higher education of these people
who can rightly be considered to be professional mathematicians. But
also there are many disciplines wherein mathematics is an extensive
and substantial component of study. Examples are physics or electronic
engineering. The mathematical education of professionals in such fields
as these could also come under the remit of this article in that many
of the suggestions made could enhance the teaching, learning and assessment
of students in these fields.
Traditionally assessment in higher education was solely summative and
consisted of one or more time-constrained, unseen, written examination
papers per module. A typical, and in some places predominant, purpose
of assessment was to put students in what was believed to be rank order
of ability. Students were, perhaps, asked to prove a theorem or to apply
a result, or to see if they could solve some previously unseen problem.
Generally this method succeeded in putting students in a rank order
and in labelling them excellent, above average, below average or fail.
But was it rank order of ability in mathematics or rank order of ability
to perform well in time-constrained, unseen, written examination papers?
Sadly it was the latter, and while the two may coincide, this is not
guaranteed. Taking time-constrained, unseen, written examination papers
is a rite of passage, which students will never have to do again after
graduation and which bears little relationship to the ways in which
mathematicians work. While it is true that working mathematicians are
sometimes under pressure to produce results to a deadline, the whole
concept of time-constrained, unseen, written examinations is somewhat
artificial and unrelated to working life.
It is in this context that people started to think about change, change
in the way courses are designed and organised, change in the way course
and module objectives are specified and change in the way students are
assessed and in the way the outcomes of assessment are reported. It
is usually the case that 'what you assess is what you get', that is,
the assessment instruments used determine the nature of the teaching
and the nature of the learning. Learning mathematics for the principal
purpose of passing examinations often leads to surface learning, to
memory learning alone, to learning that can only see small parts and
not the whole of a subject, to learning wherein many of the skills and
much of the knowledge required to be a working mathematician are overlooked.
In time-constrained, unseen, written examinations no problem can be
set that takes longer to solve than the time available for the examination.
There are no opportunities for discussion, for research, for reflection
or for using computer technology. Since these are important aspects
of the working mathematician's life, it seems a pity to ignore them.
And it seems a pity to leave out the possibilities for deep learning
of the subject, that is, learning which is consolidated, learning which
will be retained because it connects with previous learning, learning
which develops curiosity and a thirst for more, learning which is demonstrably
useful in working life.
This is, of course, a caricature of 'traditional' assessment, but it
is not too far from the truth, and it brings out the reasons why some
people in some societies became unhappy with university and college
education. Consequently those who educate students now pay attention
to stating aims and objectives, to redesigning curricula and structures
and to devising assessment methods which promote the learning we want
to happen and which measure the extent to which it has happened. And
they pay attention to the need to convince students and funding bodies
that they are getting good value for their investment of time and money.
The discussion on course design and assessment is also tied up with
the discussion on 'graduateness'. What is it that characterises college
or university graduates and distinguishes them from those who are not?
Is it just superior knowledge of a particular topic, or is it more than
that? It is, of course, more than that. It is not easy to define or
even to describe, but it has to do with an outlook on life, a way of
dealing with problems and situations, and a way of interacting with
other people. (This is not to denigrate the learning that non-college
graduates get from 'the university of life', nor to suggest that they
are inferior as people. It is to do with considering the 'added value'
of college or university education.) Traditionally graduateness was
absorbed, simply through the university experience, but now that we
have systems of mass education in many countries of the world, we need
to pay attention to the development of graduate attributes in students
so that they do, indeed, get value for money. In many instances, and
mathematics is no exception, it is the 'more than' that is important
when it comes to finding and keeping employment. Subject knowledge is
important but so also are personal attributes. It is highly desirable
that students develop what have come to be known as 'key skills' while
they are undergraduates, and not just because employers are saying that
the graduates they employ are weak in this area. Innovative mathematics
curricula seek to do this by embedding the development of key skills
in their teaching and learning structures. (Key skills are often described
as employability skills or transferable skills. They include such skills
as written, oral and visual communication, time management, group-work
and team-work, critical reflection and self assessment, and computer
and IT, and aural skills.)
Who are the stakeholders in an undergraduate's education? First and
foremost are the students themselves. They are investing time and effort
and they want to know that they are getting a return on this investment.
Most of them realise that it is not enough for them to be given a grade;
they know that they have to earn it. So they need to know what performance
standards are required and they need to be able to recognise within
themselves whether they have achieved these standards or not. This raises
the question of self-assessment and ways of promoting self-assessment.
Giving 'grades that count' is one way of encouraging students to carry
The next stakeholder to consider are the teachers. It is their job
to enable learning and so they need to know what learning has taken
place. Financial sponsors of students are also stakeholders. They, too,
want to know if they are getting a good return on their investment.
Finally, in the stakeholder debate, there is a demand from society,
students themselves, universities, prospective employers, that students
be summatively assessed, ranked and labelled in such a way that they
may be measured, not just against what they are supposed to have learned,
but also against their peers across the world.
This chapter will consider all of these features, but will focus on
assessment, as that is its theme. It will look at the purposes and principles
of assessment and then it will move on to consider the aims and objectives
of courses and modules. Innovative methods of assessment will be reviewed
and discussed, and this will be the biggest part of the chapter. Ways
of disseminating information about new assessment practices will be
discussed, as will obstacles to change. Finally pertinent research issues
will be mentioned. The chapter will close with an annotated bibliography
of pertinent books and papers dealing with these issues.
2. PRINCIPLES AND PURPOSES OF ASSESSMENT
Perhaps the only principle that should be applied is 'fitness for purpose'.
To achieve this, assessment methods should be intimately related to
the Aims and Objectives of the Module under consideration. And it should
be born in mind that the assessment methods used will influence the
learning behaviour of students to a considerable extent.
There are a number of purposes of assessment that should be considered:
- to inform learners about their own learning.
- to inform teachers of the strengths and weaknesses of the learners
and of themselves so that appropriate teaching strategies can be adopted.
- to inform other stakeholders - society, funders, employers including
the next educational stage.
- to encourage learners to take a critical-reflective approach to
everything that they do, that is, to self assess before submitting.
- to provide a summative evaluation of achievement.
3. AIMS AND OBJECTIVES
Aims and objectives should be established both for a course and for
each of the modules that comprise the course. The aims of a course are
statements that identify the broad educational purposes of the course
and may refer to the ways in which it addresses the needs of the stakeholders.
Here are some examples; there are, of course, many more and each provider
must write their own:
- To provide a broad education in mathematics, statistics and computing
for students who have demonstrated that they have the ability or who
are considered to have the potential to benefit from the course.
- To develop knowledge, understanding and experience of the theory,
practice and application of selected areas of mathematics, statistics,
operations research and computing so that graduates are able to use
the skills and techniques of these areas to solve problems arising
in industry, commerce and the public sector.
- To develop key skills.
- To provide students with an intellectual challenge and the practical
skills to respond appropriately to further developments and situations
in their careers.
- To prepare students for the possibility of further study at post
graduate level, including a PhD programme or a teacher training programme.
It would be necessary to indicate how each of the modules selected
for a course helps to achieve the aims of the course. The aims of the
individual modules should 'map' to the overall aims of the course. Objectives
are statements of the intended learning outcomes that would demonstrate
successful completion of the course or module, and that would warrant
progression through the course and the eventual award of a degree. Module
objectives should identify the knowledge, skills and attributes developed
by a module, and course objectives should identify the knowledge, skills
and attributes developed by the totality of modules selected for the
course. Objectives may include reference to subject knowledge and understanding,
cognitive skills, practical skills and key skills. They should be clearly
relevant to fulfilling the aims and, above all, they should be assessable,
that is, we should be able to devise assessment instruments that allow
students to demonstrate that they have achieved the learning intended,
and, if appropriate, to what extent. Here are some examples of course
On completion of their studies graduates will have:
- an understanding of the principles, techniques and applications
of selected areas of mathematics, statistics, operations research
- the ability and confidence to analyse and solve problems both of
a routine and of a less obvious nature.
- the ability and confidence to construct and use mathematical models
of systems or situations, employing a creative and critical approach.
- effective communication skills using a variety of media.
- effective teamwork skills.
A course document should demonstrate how the aims and objectives of
the constituent modules contribute to the overall course aims and objectives.
Here is an example of the aims and objectives of a module, taken from
an introductory module on mathematical modelling. (These aims and objectives
are those of module MAT112J2, University of Ulster. Full details may
be read under 'Syllabus Outline' at http://www.infj.ulst.ac.uk/~cdmx23/mat112j2.html.)
Note that an indication of the method of assessment of each objective
Aims: The aims of this module are to:
- enable students to understand the modelling process, to formulate
appropriate mathematical models and to appreciate their limitations.
- develop an understanding of mathematical methods and their role
- study a number of mathematical models.
- develop in students a range of key skills.
It can be seen how these module aims help to meet the aims of the course
listed above. Thus this module contributes to developing mathematical
understanding, problem solving, and key skills.
Objectives: On completion of this module, students should be able to:
- Formulate mathematical models and use them to solve problems of
an appropriate level. (Assessed by coursework and written examination.)
- Solve simple differential equations using calculus and computer
algebra systems. (Assessed by written examination.)
- Describe and criticise some mathematical models. (Assessed by coursework.)
- Work in groups and report their work in a variety of media. (Assessed
- Work both independently and in support of one another. (Assessed
- Demonstrate other key skills. (Assessed by coursework.)
Again, it can be seen how these module objectives map to the course
objectives listed above. There are references to the assessment of mathematical
techniques, the construction and use of mathematical models, and key
Of course, aims and objectives are not created in a vacuum. They evolve
from the previous and present experiences of the lecturing staff who
design the course and its constituent modules, and they are reviewed
and modified from time to time as circumstances permit or demand. Nevertheless,
the objectives for each module and for the course as a whole should
be stated and essentially should be a form of contract between the lecturer
and the students. Furthermore, detailed assessment criteria should be
drawn up so that lectures have a well-defined framework in which to
work and students have clear guidelines to what they have to do in order
to succeed. This contractual arrangement does to some extent limit the
power traditionally wielded by lecturers. This is a necessary and desirable
consequence of the innovations described in this paper. It makes students
more powerful in the right context, namely their own learning, in that
it does require students to take more responsibility for themselves.
4. EXTERNAL ASSESSMENT OR EVALUATION
While this paper is primarily about the assessment of student learning,
it may be appropriate to mention current developments in the evaluation
of institutions, their courses and modules, and the teaching and other
staff who deliver these. In the UK, for example, the government agency,
The Quality Assurance Agency for Higher Education has a remit to review
the quality of provision of education by institutions. Mathematics courses
were reviewed between 1998 and 2000, along with several other subjects.
The whole of university life is covered in approximately six-year cycles.
Institutions are required to write an evidence-based self-assessment
document (the SAD), which a visiting team of reviewers will scrutinise
and make a judgement on. The SAD outlines the aims and objectives of
the provision and provides details of the physical and human resources
available. It then gives the institution's own assessment of its quality
of provision under six headings:
- Curriculum Design, Content and Organisation;
- Teaching, Learning and Assessment;
- Student Progression and Achievement;
- Student Support and Guidance;
- Learning Resources;
- Quality Management and Enhancement.
Evidence to support the claims made in the SAD must be provided and
may be found in documents and in observation of teaching.
This peer-review process (the reviewers are selected from other, similar
institutions) devours a considerable amount of academic time and energy,
and it remains to be seen if the improvements justify the cost. It is
part of the general move in society to satisfy the public demand for
public accountability of public funds. On the positive side, it has
encouraged institutions to think about their course provision in a way
that would be new to many.
5. METHODS OF ASSESSMENT
Once the learning outcomes or objectives have been articulated, suitable
assessment methods have to be selected. (In practice, the articulation
of objectives and the selection of assessment methods will proceed hand-in-hand.)
This should be done in such a way as to ensure that the assessment methods
are appropriate and allow students to demonstrate positive achievement.
There should be transparent assessment criteria, which should be explained
to students, if possible with examples of good work and not so good
work harvested from previous cohorts, or descriptions of excellent,
median and pass level performance. Ideally the assessment criteria should
be drawn up in debate with the students, without sacrificing the lecturer's
expertise and experience. Assessment should blend with the teaching
and learning pattern. This section will now review some assessment practices
that have been developed and used successfully.
5.1 Individual project work
Project work, both individual project work and group project work, is
used widely. This has been a feature of many undergraduate mathematics
courses for over twenty-five years, so it can hardly be described as
innovative. Individual projects are often given to final year students
and are substantial pieces of work. The very least the project is worth
is about one sixth of final year studies, but it may be worth more than
that. The topics set for investigation can be quite demanding and give
scope for considerable initiative and independent work by students.
Projects demand research and investigation and the production of a written
report and some sort of presentation, such as a seminar, a poster or
a viva voce examination. Students learn to conduct research and to organise
information and present it cogently.
But now comes the hard part - assessing it reliably and validly - and
ensuring that students know how it is to be assessed. If students know
the assessment criteria and have some idea of what constitutes good
or not so good work, then they are in a better position to assess their
own work before submitting it, and in a better position to assess one
another's work in a peer support activity. Project work like this is
a good method for assessing many of the objectives outlined above that
lead to the development of 'the way of life' of a working mathematician
in whatever guise they may find themselves.
Experienced project assessors can usually come to an accurate judgement
of a student's work fairly quickly and can defend that judgement to
their peers. But there still is an element of subjectivity in this and,
to remove as much of this as possible and to achieve consistent marking
by several assessors, consultation and training are necessary. The team
of assessors should develop assessment criteria, should trial their
use and should analyse and reflect on their judgements. In this way
hard markers and lenient markers will be identified, and all will learn
how to apply the detail of the criteria. Inexperienced assessors need
this sort of training exercise at the start of their careers.
5.2 Group project work
Group project work is often introduced at an earlier stage in a student's
career. Again this gives opportunity for encouraging research, investigation
and communication. But it also introduces students to group work and
the problems associated with that. Often the internal working of the
group can best be assessed by the members of the group themselves. This
can be accomplished through confidential self and peer assessment. Sometimes
it is more appropriate for the lecturer to observe the working process;
this method has the added advantage that the lecturer can intervene
in crisis situations. Assessors face dilemmas when assessing project
work carried out by groups. If the same grade is given to each member
of the group, then some may benefit and some may suffer from the work,
or lack of it, of their peers, and this could be considered to be unfair,
both by students and by society. Experience shows that this is a price
worth paying. The dilemmas can be overcome by including an element of
confidential, within-group, peer assessment, by observing group work,
and by ensuring that students experience a good mix of group and individual
assessment methods throughout their course. In working life, after all,
group leaders often carry the blame for the poor performance of their
group. By that stage, of course, they will be much more experienced
and will have more control over their staff, but it is a useful lesson
for students to discover the difficulties of working with other people,
provided it is not disastrous to the overall outcome of their time at
5.3 Variations on written examinations
Variations on the theme of written examinations have been tried. These
include open book examinations, seen examinations wherein students are
given the questions some time in advance and they prepare their answers
to them, examinations conducted in a computer laboratory with ready
access to computer algebra systems and other mathematical software,
and examinations which involve conceptual questions.
5.4 Comprehension tests
Some experiments on the use of Comprehension Tests have been carried
out. This method of assessment is widely used in other subjects. Students
are given an article or part of a book to read in advance. They study
it very carefully and then take an unseen written paper, which is designed
to explore the extent to which each student has comprehended the article.
This can be useful for assessing a student's understanding of mathematical
processes. Furthermore it encourages students to read critically and
reflectively, to try to get into the mind of the author, and to think
deeply about the topic of the article. It helps them to see that mathematics
is alive and active in some contemporary context.
5.5 Journal writing
Student journal writing through the course can be used to help diagnose
learning difficulties and to address these at an early stage. Students
may be given time at the end of a teaching session to reflect on their
learning during that session and to write down their thoughts and feelings,
their worries and concerns, what they have learnt and what they are
having difficulty with. Or they may be asked to do this overnight, thus
giving them at least a little time to digest the day's work. The journals
should be read frequently by the lecturer so that formative feedback
can be given in good time and appropriate intervention strategies introduced
Other strategies have been developed and used, such as brief, 'one-minute'
quizzes or student-written summaries of key points learned (or not)
at the end of class periods to provide feedback to instructors, particularly
at early stages of modules. Student portfolios, student lectures and
combined written-oral examinations are other strategies that have been
used to good effect
6. DISSEMINATION OF INNOVATIVE IDEAS AND CONCLUSIONS OF RESEARCH STUDIES
The impetus for change and innovation usually comes from individuals
who are dissatisfied with what they have been doing. They will have
experimented, preferably with the approval of their head of department
but sometimes covertly, and evaluated the effects of their ideas, and
then adopted or scrapped them. The wider mathematics community can be
informed about these developments in the same ways that research findings
are disseminated, that is, by word of mouth at seminars and conferences,
and by publication. It is helpful if papers relating to teaching and
learning research are included in mainstream mathematics conferences
and journals. Then lecturers who would never dream of attending a 'teaching'
conference or reading a 'teaching' journal might just be exposed to
these ideas and might be persuaded to accept them and adopt them.
Very occasionally a charitable foundation or a government agency will
fund the production and dissemination of material relating to teaching
innovation. Some examples are given in the annotated bibliography at
the end of this chapter.
7. OBSTACLES TO CHANGE AND STRATEGIES FOR OVERCOMING THEM
Ignorance and prejudice are, perhaps, the greatest obstacles to change.
Lack of resources is another. Many teachers in higher education will
not have attended a course on teaching as part of their pre-service
or in-service preparation for the job. (Most will have completed a PhD
or equivalent and will be well versed in research methodology.) And
so they will most probably teach as they themselves were taught. They
are ignorant (in the nicest possible sense of the word!) of new ideas
and new scholarship in student learning. Overcoming ignorance is relatively
easy but requires an extensive programme of dissemination of ideas targeted
particularly at new lecturers. One strategy being introduced in the
UK is the requirement of many institutions for new lecturing staff to
complete a post graduate certificate in university teaching. Usually
this will be a two-year, part-time course delivered by the lecturer's
own institution (or local consortium) and completing it successfully
is a requirement of probation. Courses will usually include modules
on generic and subject specific teaching and learning and the assessment
instruments will include a portfolio of work relating to the lecturer's
own teaching. Another strategy introduced in the UK is the recommendation
that all lecturers join, and maintain their membership of, the newly
constituted Institute for Learning and Teaching in Higher Education
(ILT). Membership of the ILT gives public recognition that the member
has had training as a teacher and continues to develop professionally.
It will function as a professional association. (For two years there
will be a special route to membership for experienced teachers, who
will not be required to undergo an initial training programme but will,
instead, submit a relatively short document outlining their experience
as teachers and including a section of critical reflection on their
work. See Mason, this volume, pp. 529-538.) It helps greatly if the
head of department and other senior officers in the institution are
sympathetic to the aims of such a programme, are knowledgeable about
developments in teaching and learning, and support and encourage their
colleagues to overcome their ignorance.
Prejudice is much harder to overcome. Prejudice is when a person, in
full knowledge of developments, still rejects them unreasonably and
out-of-hand, just because they are innovations or perhaps because once
they encountered badly written arguments for change or are suspicious
of research in education. This requires greater evangelistic effort.
Research findings must be carefully presented and arguments for innovation
persuasively written. Personal contacts, one-to-one over a meal or a
drink, are good opportunities for this. Again, encouragement from heads
of department and higher is very valuable.
Resource issues are important also. Recent years have seen resources
diminish in universities all over the world. Classes are bigger and
lecturers have conflicting and very strong demands on their time, particularly
the demand to carry out high quality research. Money to pay for professional
development is in short supply. There are no easy answers to the problem
of lack of resources. It is a matter of commitment and priority, particularly
on the part of heads of department and other resource managers in institutions.
If the leader of a unit is committed to innovation and development in
teaching and learning, then it is more likely to happen. One possible
help might come from the bodies that determine the resources given to
universities or departments to conduct research. If they were to allow
research into the teaching and learning of a subject to have equal status
with research into the subject itself, and if they were to allocate
funds for this, then it is likely that more attention would be paid
to teaching development. Also other carrots within institutions, such
as promotion criteria which include good teaching, would help to stimulate
the developments outlined in this paper.
Perhaps one of the more serious obstacles to be faced, as regards change
and innovation in assessment at the university level, is that there
may be genuine conflicts of interest between different stakeholders
and parties. For instance, there may be a clash between, on the one
hand, academics who tend to insist on the (in)formative purpose of assessment
and on the ensuing necessity of multi-faceted and complex assessment
instruments for validly capturing a fair range of knowledge and skills
with their students, and, on the other hand, institution heads and administrators
who tend to insist on the summative or ranking aspects of assessment
in order for the institution to live up to common expectations in the
environment or to make a positive appearance in a highly competitive
'university market'. However, even if this sort of clash is not present,
clashes may arise if, as is often the case, the innovative assessment
methods advocated or adopted by academics turn out to be considerably
more time consuming or resource intensive than the traditional methods.
At times when university funding is scarce heads and administrators
may be inclined to counteract the use of such methods, not because of
scepticism towards their relevance but simply because of the resources
they consume. Such sorts of clashes are of an objective nature and cannot
always be easily reconciled.
8. PERTINENT RESEARCH ISSUES
As mentioned above, there are sceptics in universities who are not
yet convinced that teaching innovations are necessarily a good thing,
but who may still be receptive to persuasive arguments and research
findings. So research which seeks to evaluate innovative teaching methods
and which demonstrates that aims and objectives are being met, is needed.
Of course, the teaching developments themselves must have a rationale
which is based on research into student learning. Another field of study
is the robustness of the assessment methods themselves.
There are a number of internationally renowned teams who have published
widely their research into student learning. Most of the work on assessment
has been carried out by the Assessment Research Group (ARG) in the UK
and some members of the International Community of Teachers of Mathematical
Modelling and Applications (ICTMA). The work of the ARG is reported
at length elsewhere in this volume (see Haines and Houston, this volume,
pp. 431-442). Their main work has been to develop, test and evaluate
robust methods for assessing several different forms of student project
work and associated communication skills. They were also the nucleus
of a group who received UK government funding to develop and disseminate
resource material relating to innovative learning and assessment. Since
some of this work related to teaching mathematical modelling, members
of ARG are also active in the ICTMA, and some original research is published
in the ICTMA series of books and conference abstracts.
But every teacher can be a researcher in their own classroom, picking
up good ideas, developing them, evaluating them and then telling the
world about them. This is actually quite an exciting thing to do and
people who do it get a buzz from the experience which invigorates themselves,
their teaching and their students.
- American Association of University Professors (1990). Mandated Assessment
of Education Outcomes. Academe, Nov./Dec. Discusses impact of mandated
assessment on traditional arenas of professorial autonomy; focuses
on five assessment issues (institutional diversity, skills, majors,
value-added, and self-improvement). Concludes with recommendations
for learning to live with mandated assessment.
- Angelo, Thomas A. and Cross, K. Patricia (1993). Classroom Assessment
Techniques: A Handbook for College Teachers, 2nd Ed, San Francisco:
Jossey-Bass Publishers. This text focuses on formative classroom assessment.
In addition to describing what classroom assessment is and how one
might plan and implement classroom assessment tasks, the authors present
50 different classroom assessment techniques, many of which can be
used in or modified for the mathematics classroom.
- Astin, Alexander, Banta, Trudy W., Cross, Patricia K., et al. (1992).
Principles of Good Practice for Assessing Student Learning. Washington,
DC: American Association of Higher Education. Nine principles for
assessing student learning developed by the long-standing annual Assessment
Forum of the American Association of Higher Education (AAHE).
- Ball, G, Stephenson, B, Smith, G, Wood, L, Coupland, M, and Crawford,
K. (1998). Creating a Diversity of Mathematical Experiences for Tertiary
Students. Int J Math Edu Sci Technol, 29, 827-841.
- Bass, Hyman (1993). Let's Measure What's Worth Measuring. Education
Week, October 27, 32. An 'op-ed' (opinion) column supporting Measuring
What Counts from the Mathematical Sciences Education Board (MSEB).
Stresses that assessments should (a) reflect the mathematics that
is most important for students to learn; (b) support good instructional
practice and enhance mathematics learning; and (c) support every student's
opportunity to learn important mathematics.
- Benjamin, Ernst (1990). The Movement to Assess Students' Learning
will Institutionalize Mediocrity in Colleges. Chronicle of Higher
Education, July 5. A brief op-ed column criticizing the indefensible
consequences for higher education of rapidly spreading accountability
systems that rely on narrow tests.
- Benjamin, Ernst (1994). From Accreditation to Regulation: The Decline
of Academic Autonomy in Higher Education. Academe, July/Aug., 34-36.
A worried analysis by the retired general secretary of the American
Association of University Professors (AAUP) concerning the impact
of increased regulation based on accountability measures that, Benjamin
believes, might distort traditional goals of the academy.
- Berry, J. and Haines, C.R. (1991). Criteria and Assessment Procedures
for Projects in Mathematics. Plymouth: University of Plymouth. The
first of four reports written by the UK Assessment Group (ARG). It
is a report of a workshop held in 1991, which aimed to review the
assessment schemes and assessment criteria currently in use in UK
universities and to begin to develop robust criteria-based assessment
procedures for a wide range of topics. It describes the use of the
FACETS data analysis package to consider the differing effects of
the criteria themselves and the ways in which the assessors used them.
- Berry, J. and Houston, K. (1995). Students using Posters as a means
of Communication and Assessment. Educational Studies in Mathematics,
29, 21-27. Gives a thorough literature review of the use of posters
by students in higher education generally, and suggests ways in which
student posters could be used and could be assessed in mathematics
- Bok, Derek (1986). Toward Higher Learning: The Importance of Assessing
Outcomes. Change, Nov./Dec., 18-27. A classic short essay by then
Harvard President Derek Bok outlining the benefits to higher education
of assessing the accomplishments and value of a college education.
- Burton, L. and Izard, J. (1995). Using FACETS to Investigate Innovative
Mathematical Assessments. Birmingham: University of Birmingham. The
last of four reports written by ARG. It reports on the 1994 workshop
and discusses continuing issues in assessing student project work.
It deals with reliability of marking schemes, self, peer and tutor
marked assignments, equity, and criteria for the assessment of students'
- Committee on the Undergraduate Program in Mathematics (CUPM) (1995).
Assessment of Student Learning for Improving the Undergraduate Major
in Mathematics. Focus: The Newsletter of the Mathematical Association
of America, 15:3 (June), 24-28. Recommendations from the Mathematical
Association of America (MAA) for departments of mathematics to develop
a regular 'assessment cycle' in which they (1) set student goals and
associated departmental objectives; (2) design instructional strategies
to accomplish these objectives; (3) select aspects of learning and
related assessments in which quality will be judged; (4) gather assessment
data, summarize this information, and interpret results; and (5) make
changes in goals, objectives, or strategies to ensure continual improvement.
- Edgerton, Russell (1990). Assessment at Half Time. Change, Sept./Oct.,
4-5. A brief summary of the political landscape of assessment in higher
education by the president of the American Association of Higher Education
(AAHE). Claims that state pressures for accountability will continue,
but that if institutions define assessment in worthy terms the faculty
will find the effort worthwhile.
- Ewell, Peter T. (1997). Strengthening Assessment for Academic Quality
Improvement. In Marvin W. Peterson, David D.Dill, and Lisa A. Mets
(Eds.), Planning and Management for a Changing Environment: A Handbook
on Redesigning Postsecondary Institutions, pp. 360-381. San Francisco:
Jossey-Bass Publishers. Historical survey of assessment efforts in
the U.S. during the last decade in the context of increased accountability
requirements, decreased financial resources, and increased experience
with assessment on college campuses. Discusses relation of assessment
to academic planning and syndromes to avoid.
- Ewell, Peter T. and Jones, Dennis P. (1996). Indicators of Good
Practice in Undergraduate Education: A Handbook for Development and
Implementation. Boulder, Co: National Center for Higher Education
Management Systems (NCHEMS). Intended to provide colleges and universities
with guidance in establishing an appropriate system of indicators
of the effectiveness of undergraduate instruction, and to build on
this foundation by cataloguing a range of exemplary indicators of
'good practice' that have proven useful across many collegiate settings.
- Ferren, Ann (1993). Faculty Resistance to Assessment: A Matter of
Priorities and Perceptions. Commissioned paper prepared for American
Association of Higher Education. Analyzes faculty priorities to help
understand why assessment is rarely valued by faculty. Argues that
assessment must derive from widely agreed goals, must be connected
to clear outcomes that the faculty see as beneficial, and must not
be simply added to already overburdened faculty loads.
- Frechtling, Joy A., 1995. Footprints: Strategies for Non-Traditional
Program Evaluation. Washington, DC: National Science Foundation. A
series of papers suggesting diverse strategies for assessing the impact
of funded programmes both short- and long-term, both intended and
- Gaither, Gerald H. (1995). Assessing Performance in an Age of Accountability:
Case Studies. San Francisco: Jossey-Bass Publishers. Case studies
from several states and public institutions about the shift from campus-based
assessment in the 1980s to state-based accountability systems in the
- Glassick, Charles E., Huber, Mary T. and Maeroft, Gene I. (1997).
Scholarship Assessed: Evaluation of the Professoriate. Carnegie Foundation
for the Advancement of Teaching. San Francisco: Jossey-Bass Publishers.
Companion to Ernest Boyer's widely-cited Scholarship Reconsidered:
Priorities of the Professoriate, this report outlines standards for
evaluating scholarship that transcend differences among disciplines:
clear goals, adequate preparation, appropriate methods, significant
results, effective presentation, and reflective critique.
- Gold, Bonnie, Keith, Sandra and Marion, William (Eds.), (1999).
Assessment Practices in Undergraduate Mathematics. Washington, DC:
Mathematical Association of America. A collection of over fifty brief
reports from dozens of different U.S. colleges and universities providing
a wide variety of methods of assessing the major, teaching, classroom
practice, the department's role, and calculus reform.
- Haines, C.R. and Dunthorne, S. (Eds.) (1996). Mathematics Teaching
and Learning-Sharing Innovative Practices. London: Arnold. This resource
pack is a collection of articles describing innovative practices in
teaching and assessment. It was written by mathematics lecturers from
a consortium of UK universities, including members of ARG.
- Haines, C.R. and Izard, J. (1995). Assessment in Context for Mathematical
Modelling. In C. Sloyer, W. Blum and I. Huntley (Eds.). Advances and
Perspectives in the Teaching of Mathematical Modelling and Applications
pp. 131-149. Yorklyn, Delaware: Water Street Mathematics. Credible
assessment schemes measure evidence of student achievement, as individuals
or within a group, over a wide range of activities. This paper shows
that item response modelling can be used to development rating scales
for mathematical modelling. It draws on the work of ARG and work in
- Haines, C.R., Izard, J. and Berry, J. (1993). Awarding Student Achievement
in Mathematics Projects. London: City University. The second of four
reports written by ARG. It investigates in depth the use of assessment
criteria for judging oral presentations by students of their project
work. It also proposes criteria for the assessment of written reports
of different types of student project work.
- Hilton, Peter (1993). The Tyranny of Tests. American Mathematical
Monthly, April, 365-369. Several suggestions for 'reducing the distorting
effect' which tests exert, principally on undergraduate mathematics.
- Houston, S.K. (1993). Developments in Curriculum and Assessment
in Mathematics. University of Ulster. This pamphlet contains papers
presented at a one day symposium at the University of Ulster following
the 1993 meeting of ARG.
- Houston, S.K. (1993). Comprehension Tests in Mathematics (I and
II), Teaching Mathematics and its Applications, 12, 60-73 and 113-120.
These are the original papers describing the use of comprehension
tests in mathematics.
- Houston, SK. (1995). Assessing Mathematical Comprehension. In C.
Sloyer, W. Blum, and I. Huntley (Eds.), Advances and Perspectives
in the Teaching of Mathematical Modelling and Applications, pp. 151-162.
Yorklyn, Delaware: Water Street Mathematics. This paper examines the
rationale for comprehension tests in mathematics, and outlines possible
aims and objectives. It describes the author's experiences in setting
and using such tests and outlines the extent of their use in secondary
and tertiary education in the UK and extends the work reported in
- Houston, S.K. (1997). Evaluating Rating Scales for the Assessment
of Posters. In S.K. Houston, W. Blum, I. Huntley and N.T. Neill, (Eds.),
Teaching and Learning Mathematical Modelling, pp. 135-148. Chichester:
Albion Publishing (now Horwood Publishing). Deals with the use of
posters by university students as a means of communication and as
a vehicle for assessment. There is a summary of a literature review
and a rationale for the activity. The author claims that it is an
enjoyable activity, which is beneficial for students. The main purpose
of the paper is to describe the development and evaluation of assessment
criteria and rating scales.
- Houston, S.K., Haines, C.R. and Kitchen, A. (1994). Developing Rating
Scales for Undergraduate Projects. University of Ulster. The third
of four reports written by ARG. It reports on the 1993 workshop, giving
details of assessment criteria (or descriptors) for the assessment
of written reports on projects in pure mathematics, mathematical modelling,
statistical investigations and investigations of a more general nature.
The report describes how the group members trialled the criteria and
how the data analysis led to the development of robust assessment
procedures. It also introduces the use of criteria for the assessment
of student posters.
- Izard, J. (1997). Assessment of Complex Behaviour as Expected in
Mathematics Projects and Investigations. In S.K. Houston, W. Blum,
I. Huntley and N.T Neill, (Eds.), Teaching and Learning Mathematical
Modelling, pp. 109-124. Chichester: Albion Publishing (now Horwood
Publishing). No single assessment method is capable of providing evidence
about the full range of achievement. This paper reviews the problems
faced in devising better assessments to monitor learning, and provides
practical suggestions for meeting these problems. The methods presented
are applicable to traditional examinations, project and investigation
reports, presentations and posters, judgements of performance and
constructed projects, observations of participation, collaborative
group work and ingenuity. The paper concludes with advice on monitoring
the quality of the assessment process.
- Joint Policy Board for Mathematics (1994). Recognition and Rewards
in the Mathematical Sciences. Providence, RI: American Mathematical
Society. Discussion of faculty expectations in relation to institutional
rewards. Findings include a general dissatisfaction with current methods
of evaluating teaching as well as uncertainty about the weight of
effective teaching in college expectations and rewards.
- Katz, Stanley N. (1994). Defining Education Quality and Accountability.
Chronicle of Higher Education, November 16, A56. An op-ed statement
by the president of the American Council of Learned Societies (ACLS).
Urges that colleges and universities heed the wake-up call of assessment
from elementary and secondary schools and figure out how to define
educational quality in terms that are worthy of higher education.
- Linn, Robert L. and Herman, Joan L. (1997). A Policymaker's Guide
to Standards-Led Assessment. Denver, CO: Education Commission of the
States. Analysis of policy implications involved in shifting from
norm-referenced assessments (which compare each students' performance
to that of others) to standards-led assessments which incorporate
pre-established performance goals, many of which are based on real-world
rather than 'artificial' exercises.
- Madison, Bernard (1992). Assessment of Undergraduate Mathematics.
In Lynn A. Steen (Ed.), Heeding the Call for Change: Suggestions for
Curricular Action, Washington, pp. 137-149. DC: Mathematical Association
of America. Analysis of issues, benefits, worries, and pressures associated
with the increasing demand for assessment of undergraduate mathematics.
A background paper preceding release of the CUPM report on assessment.
- Mathematical Sciences Education Board (1993). Measuring What Counts:
A Conceptual Guide for Mathematics Assessment. Washington, DC: National
Research Council. Intended primarily as advice for K-12 mathematics
assessment, this report stresses the need for assessment to measure
good mathematics, to enhance learning, and to promote access for all
students to high quality mathematics.
- National Council of Teachers of Mathematics (1995). Assessment Standards
for School Mathematics. Reston, VA: National Council of Teachers of
Mathematics. This third and final volume in NCTM's original set of
standards for school mathematics focuses on six standards: effective
assessment should reflect appropriate mathematics, enhance learning,
promote equity, be based on an open process, promote valid inferences,
and fit together coherently.
- Niss, Mogens (Ed.) (1993). Investigations into Assessment in Mathematics
Education - An ICMI Study. Dordrecht: Kluwer Academic Publishers.
This book is one of two resulting from the ICMI Assessment Study.
The book offers a variety of approaches to the conceptual, philosophical,
historical, societal, and pedagogical investigation of assessment
in mathematics education, by prominent mathematics educators from
Europe, North America and Australia. Both survey chapters and specific
empirical or theoretical studies are included in the book.
- Open University Course Team (1998). Assessment of Key Skills in
the Open University Entrance Suite. MU120, MST121, MS221. Open University,
- Romer, Roy (1995). Making Quality Count in Undergraduate Education.
Denver, CO: Education Commission of the States. Report by the then-Governor
of Colorado on behalf of all U.S. state governors concerning what
parents and students expect of higher education and what research
says about the characteristics of high-quality undergraduate education.
Concludes with recommendations for steps to make higher education
more accountable to its public purposes.
- Schilling, Karen Maitland and Schilling, Karl L. (1993). Professors
Must Respond to Calls for Accountability. Chronicle of Higher Education,
March 24, A40. An op-ed column arguing that faculty must take seriously
the public's demand for evidence that students are learning, and learning
the right things. Suggests portfolio assessment as an effective strategy.
- Schoenfeld, Alan (1997). Student Assessment in Calculus. Washington,
DC: Mathematical Association of America. Report of an NSF working
group convened to support assessment of calculus reform projects by
providing a conceptual framework together with extensive examples.
Emphasizes the 'fundamental tenet' that, since tests are statements
of what is valued, new curricula need new tests.
- Seldin, Peter (1993). The Use and Abuse of Student Ratings of Professors.
Chronicle of Higher Education, July 21, A40. An op-ed column lamenting
the propensity of colleges to misuse student evaluations of faculty.
Gives research-based advice for how to use such ratings intelligently
- Smith, G., Wood, L., Coupland, M., Stephenson, B., Crawford, K.
and Ball, G. (1996). Constructing Mathematical Examinations to Assess
a Range of Knowledge and Skills. Int J Math Edu Sci Technol, 27, 65-77.
- Steen, Lynn Arthur (1999). Assessing Assessment. Preface to Bonnie
Gold, Sanda Z. Keith and William A. Marion (Eds.), Assessment Practices
in College Mathematics, pp. 1-6. Washington, DC: Mathematical Association
of America. An exploration of issues, principles, and options available
to address the wide variety of assessment challenges facing college
- Stevens, Floraline, Lawrenz, Frances and Sharp, Laure (1993). User-Friendly
Handbook for Project Evaluation. Washington, DC: National Science
Foundation. A 'how-to' guide to effective assessment for project directors
who have neither experience in nor enthusiasm for evaluation.
- Tucker, Alan C. and Leitzel, James R. C. (1995). Assessing Calculus
Reform Efforts. Washington DC: Mathematical Association of America.
A 'mid-term' review of the NSF-supported calculus reform movement
in the United States, providing background on the motivation and goals
of the movement, as well as evidence of changes in content, pedagogy,
impact on students, faculty, departments, and institutions.
- Wiggins, Grant (1989). A True Test: Toward More Authentic and Equitable
Assessment. Phi Delta Kappa, May, 703-713. Argues that misunderstanding
about the relation of tests to standards impedes progress in educational
improvement. Suggests that only tests that require the 'performance
of exemplary tasks' can truly monitor students' progress towards educational
- Wiggins, Grant (1990). The Truth May Make You Free, but the Test
May Keep You Imprisoned: Toward Assessment Worthy of the Liberal Arts.
In Assessment 1990: Understanding the Implications, pp.17-31, Washington,
DC: The American Association for Higher Education. (Reprinted in 1992,
Lynn A. Steen (Ed.). Heeding the Call for Change: Suggestions for
Curricular Action, pp. 150-162. Washington, DC: Mathematical Association
of America.) Philosophical reflections on the purposes of education
in the liberal arts or in basic science or mathematics. Focuses on
ten principles of education that testing tends to destroy (e.g., justifying
one's opinions; known, clear, public standards and criteria; self-assessment
in terms of standards of rational inquiry; challenging authority and
asking good questions).
University of Ulster, Northern Ireland