This article
describes the design and impact of a quantitative literacy rubric for a General
Education curriculum at a large, urban University. The rubric has been tested through year-long student portfolio
review, producing a change in curriculum and changes in how mathematics is
approached at an institutional level.
2. BACKGROUND AND GOALS: WHAT DID WE HOPE TO
ACCOMPLISH?
Portland State
University’s General Education Program, University Studies’ mission statement
notes that, “The purpose of the general education program...is to facilitate
the acquisition of the knowledge, abilities, and attitudes that will form a
foundation for lifelong learning among its students. This foundation includes the capacity to engage in inquiry and critical
thinking, to use various forms of communication for learning and expression, to
gain an awareness of the broader human experience and its environment, and
appreciate the responsibilities of persons to themselves, to each other, and to
community.” With this mission statement
in mind, the freshman year at PSU involves a year-long team-taught
interdisciplinary course entitled Freshman Inquiry. One of the primary assignments in Freshman Inquiry is the
year-long portfolio, which seeks to determine student progress through
reflection on the four goals of University Studies: Critical Thinking,
Communication, Diversity of Human Experience, and Ethics and Social
Responsibility. In the portfolio,
students present and reflect on work that is representative of each goal in the
program in digital or “hard copy” formats.
A key component for assessment involves an intensive review of a
representative sample of the portfolios.
Rubrics have been designed for assessment of the four goals, however,
the communication goal contains written, quantitative, oral, and graphic
communication – and no rubric had been designed for quantitative
communication. Our goal was to design
an effective rubric, implement it during the 2002 review, analyze the results,
and implement any needed curricular changes to increase student learning.
The first phase of development involved the creation of an
interdisciplinary team of faculty members from Freshman Inquiry and support
from Portland State University’s Assessment Resource Network (ARN). ARN provided research support and served to
align the Assessment Initiative in Freshman Inquiry with the broader Assessment
Initiative within PSU. The Freshman
Inquiry team first designed student learning objectives after conducting
research. The two basic objectives are:
1)
Understanding
numbers as a natural part of organized, logical thinking. Underlying this objective is the concept
that math is a language. When most
students read a simple sentence in English, they don’t become preoccupied with
the words themselves. Instead, they
perform an almost automatic translation from the words to the meaning that the
words convey. We want them to learn to
react to numbers in a similar way.
2)
Understanding
that quantity matters in making decisions.
To make good decisions on issues of public and private importance, we
must often develop detailed and careful understanding of numerically expressed
information.
We then researched on and designed expected and desirable learning
outcomes for Freshmen at PSU. Expected
outcomes include:
1) Students should be able to critically
evaluate mathematics and statistics in the media including interpreting and
critiquing graphs;
2) Students should be able to communicate
using descriptive statistics in a research paper;
3) Students should be able to display data
with appropriate charts and graphs to communicate information.
Desirable outcomes include:
1) Students should be able to explain the meaning
of statistical significance; explain why significance does not necessarily
imply importance; explain why a well-chosen anecdote can illustrate but not
substantiate a general rule;
2) Students should be able to explain the
meaning of correlation and how the significance test is applied to correlation
– and explain why correlation does not necessarily imply causation;
3) Students should be able to describe an
application of the normal curve to a social and physical phenomena; give an
example of a case that would fall into one of the tails of a normal
distribution (i.e. an outlier) and provide an example of a
distribution that is not normally distributed;
4) Students should also be able to critically
analyze graphic representations of linear regression, interpreting the
significance and slope of regression.
From
the design of outcomes and learning objectives, we began to interrogate the
very nature of our goal: Whereas “numeracy”
had been the operative term in Freshman Inquiry, we began to move towards the notion
of designing a “quantitative reasoning”
rubric. This is a significant
development for the latter term included a greater emphasis on critical
thinking skills necessary for quantitative learning skills in general
education. After further research and
work on the rubric, we realized that the concept of quantitative literacy is more appropriate, for it includes not only
an emphasis on critical thinking and basic quantitative methodology, but also
upon a variety of skills consonant with the goals of the program – and more
importantly, skills that are essential to all undergraduate students,
regardless of their major. The National
Council on Education’s Mathematics and
Democracy: The Case for Quantitative Literacy was particularly helpful in
this regard. The basic skills for
quantitative literacy are in many ways, contained in the goals of University
Studies as well as the objectives and student learning outcomes described
above. An effective programmatic emphasis on quantitative literacy is necessary
for the development of citizenship, understanding and
analyzing the graphs, projections, statistics, and other quantitative data that
are used to justify or reject public policy issues, culture and heritage,
appreciation of the physical world, professional development, personal finance,
and personal health.
The quantitative literacy rubric (also
attached) is designed on a six-point scale to accommodate the needs of students
throughout their undergraduate experience.
The rubric was refined through research, team meetings, consultation
with the faculty at large, circulation throughout ARN and the institution at
large. The rubric was then introduced
into PSU’s Freshman Inquiry portfolio review in 2002. Participants in the review come from a diverse range of disciplines
and are well-calibrated in morning training sessions.
An
essential, albeit obvious, insight that we have gleaned from our efforts is
that perpetual intentional focus on the role of mathematics in undergraduate General
Education can produce a clearer sense of learning outcomes and may produce
fundamental shifts in emphasis and approach.
Such efforts of assessment can and often do result in increasing student
learning.
It
is often said that Assessment can create learning, but how? In addition to
understanding the need for increased emphasis in the curriculum on quantitative
literacy through what may be initially low scores, a focus on the rubric itself
instituted discussions among faculty and students on the importance and role of
mathematics in General Education. Prior
to the portfolio review, faculty had begun to rethink their syllabi for the
following year and began to integrate mathematics-based assignments into their
courses.
The
transformation from a loosely defined focus on “numeracy” to a specific and
student learning-outcome focus on quantitative literacy can be considered a
major success. Rubric development
(which is an on-going, rather than terminal process) can significantly impact
student learning and refine programmatic focus in General Education.
Data
from the portfolio review have recently been generated and a full analysis will
be completed by September 13, 2002.
Preliminary results are mixed, but not without encouragement. If the “benchmark” for success in the first
year of General Education at PSU is a score of four on the average, then the
mean score of 2.55 and median of 2.00 in the first year of the rubric’s
implementation reveals that Freshmen at PSU are not entirely distant from an initial
benchmark of success. Findings also
show that the standard deviation between faculty teams is broader than for any
of our other goals. This is useful for
it can inform teams that are particularly in need of programmatic support in
this area. These data can be
effectively returned into the Assessment “loop” for on-going program
improvement, increasing students’ quantitative literacy and institutional
understanding of the significance of quantitative literacy in General
Education.
NOTE:
In the case study itself, the role, nature, and general applicability of the
quantitative literacy rubric as well as the data will be more closely
examined. Additionally, next steps and
recommendations will also be presented in greater detail.
Acknowledgements:
Paul Latiolais, Toni Levi, Tom Luckett, Georg Grathoff, Alan MacCormack, Judy
Patton, Chuck White, William Becker, Cheryl Ramette, Judy Redder, and Zahra
Baloch.
6. Portfolio
demonstrates evidence of ability to conduct independent research and to
integrate the results with other methodologies in original work. The meaning of
statistical significance, calculus, a comprehensive understanding of causality
and correlation, applications of normal curves and outliers to physical and
social phenomena, and an integrated comprehension of linear regression is
comprehensively displayed.
5. Portfolio demonstrates evidence of ability to conduct independent research
and to integrate the results with other methodologies in original work although
not to the fullest extent possible. The meaning of statistical significance, a
comprehensive understanding of causality and correlation, applications of normal
curves and outliers to physical and social phenomena, and an integrated
comprehension of linear regression is present but not fully displayed.
4. Portfolio contains assignments demonstrating evidence of an ability to read,
understand, and critique books or articles that make use of quantitative
reasoning, using descriptive statistics, understanding the meaning of
statistical significance, and by displaying data using appropriate graphs and
charts. Assignments are included in the
portfolio as separate entities and quantitative reasoning is integrated into
other work.
3. Portfolio demonstrates evidence of an ability to read, understand, and
critique books or articles that make use of quantitative reasoning, using
descriptive statistics (mean, median, mode), understanding the meaning of
statistical significance, and by displaying data using appropriate graphs and
charts. Alternatively, well-designed
and appropriate quantitative reasoning assignments are included in the
portfolio, but as separate entities.
2. Portfolio demonstrates evidence of limited ability to define, duplicate,
label, list, recognize and reproduce mathematical and statistical
elements. Portfolio displays limited or
no evidence of meaningful application of these numerical concepts.
1. Portfolio demonstrates no evidence of ability to evaluate mathematics and
statistics, including no knowledge of basic descriptive statistics.