“Assessing Quantitative Literacy Needs across the University”

Abstract

Portland State University is grappling with what “quantitative literacy” means in terms of student outcomes and faculty expectations of students at the departmental level.  Pilot surveys of faculty and students were developed to attempt to explore these issues.  Initial data from a survey of students with several disciplinary backgrounds was analyzed to develop hypotheses on the mathematical needs of students.

Background and Goals:

Portland State University has engaged in several assessment initiatives aimed at determining what graduates need to know and be able to do when they graduate, both departmentally and in general.  Mathematical skills have consistently come up as a clear need of students, expressed both by faculty and students alike.  What that means as it is translated from discipline to discipline is not yet clear.

The quantitative literacy research team was organized in the Fall of 2001 as part of the University’s assessment initiative.   The first objective of the quantitative literacy research team was to develop strategies to help departments identify the mathematical (and statistical) needs of their students.   The second objective was to articulate the mathematical needs of students in consistent ways to identify the common needs of all students across disciplines. 

Description:

In an effort to meet our goals, the quantitative literacy research team developed and pilot tested a faculty survey (see Appendix A) to generate conversations with departments about what was important and to get an initial sense of what students need to know and be able to do in the mathematical realm.  We wanted to use quantitative literacy concepts, as opposed to concepts articulated in traditional mathematical terms for two reasons.  First, we wanted to have a different conversation about what was important to people and get away from the common response of what math departments are or are not teaching students.  Second, we wanted to get a sense of the context in which the students are being asked to apply mathematical tools.  This process would help us understand more about what mathematics the students really needed and also point out to the faculty that context mattered.   We pilot tested the faculty survey in Biology, Geography, Psychology and University Studies (the core general education unit).  We added statistics to our list of quantitative literacy questions, as a result of feedback we received from respondents.  We are in the process of continuing the conversation with those departments.

Based on our initial responses to the faculty surveys, we decided to create a student version.  The student surveys would ask students what they felt was important and what they thought their skill levels were.  Student survey questions (see Appendix B) were modeled from the quantitative literacy elements in “The Case for Quantitative Literacy” (Steen, 2001, p.8). 

The Student Survey 

In addition to asking students to provide certain demographic information, the survey contained six sections.  The first section addressed student’s attitudes towards Quantitative Literacy.  The next four sections consisted of sub-areas of mathematics identified in Steen that comprise QL: Confidence with Mathematics, Cultural Appreciation of Mathematics, Logical Thinking and Reasoning in Math and Prerequisite Knowledge and Symbol Sense in Math.  The last section on Statistics was added based on feedback we received from prior discussions with faculty.  Each section asked the reader to rate how strongly they either agreed or disagreed with a given statement.  The statements were written to reflect the category they were in (i.e., in the statistics section the statements reflected information regarding statistics; e.g., “I am comfortable with statistics”).

The Students

The survey was distributed to an introductory psychology course during the 2002 summer session.  Forty-seven students completed the survey, 18 men and 29 women.  The mean age of the respondents was 28 with a range of 18 to 41 years.  Due to the wide age range, three age groups were created: (1) 18-25, (2) 26-35 and (3) 36 and older.  The original distribution of majors contained a wide range, so individual majors were grouped in general  disciplinary categories.  The sample distribution by major is, n=13, education; n=3, humanities; n=7, natural sciences; n=22, social science, and n=2, undecided.

Insights: What did we learn?

Quantitative Literacy Section:

The results from these items indicated that the majority of students felt they understood what quantitative literacy meant and believed it to be an important skill because it applies to both their career and daily activities.  

Confidence with Mathematics Section:

Most students felt confident in their mathematical ability, but felt scared of math, and sought courses that were not heavily loaded with mathematics.  Analysis of Variance (ANOVA) results indicated significant gender differences in five of the items in this section (see Appendix C, figure 1).  Women felt less comfortable taking math courses, (P < .052), they were more “scared” of math, (P < .031), used mental estimates less often, ( P < .018), felt like they did not have a good intuition about the meaning of numbers, (P < .05), and felt less confident about their estimating skills, (P < .016).

Logical Thinking and Reasoning Section:

The majority of students felt comfortable in their overall logical and reasoning abilities.  ANOVA results (see Appendix C, figure 2) indicated that women questioned numerical information less frequently, (P < .051), and felt less confident in their ability to construct a logical argument, (P < .04), and felt less comfortable reading graphs,  (P < .016).  It is important to point out that, although women felt less comfortable reading graphs, they did not feel that way about reading maps.  This disparity could be a direct result of perceived negative gender stereotypes (Spencer, 2002 & Steele, 1998) on the part of women.   Spencer and Steele studied experiences of being in a situation where one faces judgment based on societal stereotypes about one’s group, women in our case. That is, the women in our sample may believe that they are not as good at mathematics as men and hence would react more negatively to mathematical terms. The term “graph” likely carries heavy mathematic implications, unlike the word “map.”

Prerequisite Knowledge and Symbol Sense Section:

Most students did not enjoy writing proofs and felt uncomfortable interpreting them.  In particular, although women felt less comfortable constructing “logical arguments,” they were not less comfortable writing proofs.  It may mean that the term “proof” is not as understood as the term “logical argument.”

Statistics Section:

When asked questions about statistics, most students felt comfortable with statistical ideas and in their ability to apply statistical concepts (See Appendix C, figure 3).  ANOVA tests revealed that older students felt significantly more comfortable thinking about information in terms of numbers in order to support claims, (P < .043). 

Cultural Appreciation Section:

This section attempted to capture the role mathematics plays in our culture.  Approximately 95% of the respondents indicated that math is important and believed it to play an important role in science and technology.  ANOVA results showed that women claimed to be less aware of the origins of mathematics than men (P < .046) (see Appendix C, figure 4).

Successes & Failures

In several of the questions the majority of respondents chose the neutral response.  This suggests two possible explanations: 1) that they honestly felt “neutral” with regards to that statement, or 2) they did not understand the statement and that led them to mark the neutral response.   In questions where the latter is likely, we may wish to delete the neutral response and reframe the question so that a neutral response is not reasonable.

  Respondents overwhelmingly indicated that they understood what quantitative literacy meant.  In spite of this finding, it is not possible to gauge whether or not students understand the idea or if they are defining it as mathematics in a more traditional sense.  One possibility is to elicit a definition from students at the onset of the survey in order to see how they define it.  Our sample was limited and biased towards the social sciences.    

Use of the findings

A number of questions have arisen from our results.  Why, for instance, are graphs scarier to students than maps?  The relatively small size of the sample and its bias towards the social sciences made it difficult to discern information as accurately as we hope to in future studies.  Using the survey within departments, we hope to discover:

• Whether the gender differences we report here remain (larger, smaller, nonexistent) in particular disciplines. For example, could it be that women are less confident estimators than men unless they are engineering majors?
• Whether the fear of math is consistent across campus, and hence does the university need to find strategies to address student fear of math courses?
• Whether there are differences in students' understanding of what quantitative literacy means as a function of discipline. For example, do math majors think QL is traditional math skills, political science majors think it means the ability to understand a poll and geology majors think it must include maps?
• Whether differences in comfort level in any of our areas will appear as the disciplines' samples become larger.  That is, will differences between humanities students and others become clearer as the number of humanities students surveyed becomes greater than three, or are humanities students really just as confident about logic as science majors?
• Whether there are differences in students' understanding of quantitative literacy as a function of age or work status. For example, do students working full time and only going to school at night feel more comfortable using data to make everyday decisions than traditional, full-time undergraduates?
• If we added a question on whether students voted in a recent election, would this information correlate with their comfort in using poll data, or reading graphs or something else?

Specific Changes to the Survey Instrument

The survey begins with our definition of quantitative literacy.  To what extent does that determine the respondents' high comprehension of what quantitative literacy means?  We may rewrite the opening section to address possible variations of students' understanding of quantitative literacy and see if this correlates with their majors or other factors.

Based on survey responses, we plan to rearrange some of the questions, delete similar questions whose responses were heavily correlated, and add questions, some from other validated surveys relating to mathematics anxiety.   For example, we will change questions in the "Cultural Appreciation” section.  We received a high number of correlated responses in this section.  Some questions can be eliminated while capturing the same information. We will also re-write questions to make them clearer and more easily comprehensible to students.  For example, “I am aware of the origins of mathematics” and “Math plays an important role in technology” are confusing.  We believe students may not fully understand what is meant by those questions.  We are also considering changing or deleting the questions relating to the important role math plays in Science and Technology.  Students universally agreed that math is important in those areas.  If this response is common, then no information is gained by asking the question.  But this does raise the question, how does that belief cause students to avoid science and technology courses?  Is it because of math anxiety? Or is it due to other factors?

After examining the data from the surveys, we realize we need more information from the respondents related to their math background.  We will revise the demographics section of the survey and add questions about the number of math courses students have taken and the grades they received in those courses.  These new questions will allow us to compare their math grades with their overall grade point average and to make clearer connections between number of courses and their attitudes towards quantitative literacy skills.  A survey designed by the Mathematics and Statistics Department will help us with these questions.

Next steps

One of our goals in this initial phase was to gather data about the survey as an assessment tool.  Are the questions readable?  Are these questions in line with our assessment goals?  Is this the best tool for addressing student attitudes towards Quantitative Literacy?  In phase II, we will administer the survey to a group of students who share the same major.  We will have data specific to a department.  Our goals for phase II are to help departments’ articulate their quantitative literacy goals for their students and to help them map those skills onto the curriculum. 

Unfortunately, due to state budget realities, the funding for the assessment research teams was re-allocated to more direct support for departments to meet the assessment requirements for our accrediting agency.  As such, phase II of this work will have to be done piecemeal.  Hopefully this work will be a model for the assessments that departments will be doing. Our focus for the future is to work with specific departments and help them identify their students’ quantitative literacy needs by administering both the student and faculty survey.  The faculty survey examines which quantitative literacy skills faculty members believe are important for their students and where in the departmental curriculum (or outside the departmental curriculum) students should be gaining these skills.  Administering both surveys will help departments make progress towards identifying and mapping quantitative literacy skills onto their curriculum.

References

Steen, Lynn A., MATHEMATICS and DEMOCRACY: The Case for Quantitative Literacy, The National Council on Education and the Disciplines, 2001.

Spencer, Steven J, Steele, Claude M, & Quinn, Diane M. “Stereotype threat and women's math performance.”  Readings in the psychology of gender: Exploring our differences and commonalities.  Hunter, Anne E.  (Ed); Forden, Carie (Ed). (2002). (pp. 54-68). Needham Heights, MA, US: Allyn & Bacon. xvii, 318pp.

Steele, Claude M. “Stereotyping and its threat are real.” American Psychologist. Vol 53(6) Jun 1998, 680-681.

Appendix A

Description of Faculty Survey

1.        To what extent is the skill above relevant to your discipline? 
Please Circle appropriate spot.

4                                3                                        2                                          1                                              0
|---------------------------|---------------------------------|-----------------------------------|--------------------------------------|

Students need                                          some aspects of this                                                      students do not
advanced                                                   skill are useful                                                                 need this skill
levels of this skill

2.        Where do you expect students to acquire this skill?
Circle ALL that apply:

1. Students should have this skill on entering PSU.
2. This skill is addressed in our departmental curriculum.
3. This skill should be addressed in the University Studies curriculum.
4. This skill should be addressed in courses required for the major taught in other departments.

3.        Please identify specific courses you are aware of that may address this skill.

4.        In what context would you see students using this skill in your discipline? Please explain. Use the back if necessary.

Appendix B

Portland State University

Quantitative Literacy Survey  

Assessment Resource Network

Quantitative Literacy Research Team

General Instructions: 

The purpose of this survey is to explore attitudes students have towards math and its relevance outside of the classroom.

Remember that this is an anonymous survey and that you are under no obligation to answer these questions. 

Please answer the questions to the best of your ability and knowledge.  

Please give only one answer to each question. 

In addition to your opinion about your program, we would like your feedback about the wording of the items or the format of the survey. Please write any comments you have about the questions next to the question or on the back of the survey. 

THANK YOU FOR PARTICIPATING! 

DEMOGRAPHIC INFORMATION 

1.  What is your major?                             

2.  How many credits do you have in your major?                      

3.  Did you transfer to PSU from another school?  (circle one)  Yes   No 

4.  Do you plan on finishing your degree at PSU?  (circle one) Yes   No  

5.  Are you an: (circle one)

                        Undergraduate (e.g., B.A., B.S.)

                        Graduate (e.g., M.A, M.S., Ph.D.)

                        Post-Baccalaureate Student

                        Other (please specify)___________________

6.  What is your GPA? _________

7.  What is your employment status:  (circle one)

      Not Working     0-19 hrs/week        20+ hrs/week           

8.  Gender:  Male   Female  

9.  Ethnic Origin:  (circle all that apply): 

White (Non-Hispanic), Asian/Pacific Islander, Black (Non-Hispanic),   Hispanic,

Native American, Multiple Ethnicities

10.  What is your age?                           

Section A: Attitude Towards Quantitative Literacy

Quantitative Literacy:  Mathematical skills that students use in the context of communicating ideas, either receiving information, providing information or making and communicating conclusions from data.

Section B: Confidence with Mathematics:

Section C:  Cultural Appreciation

Section D:  Logical Thinking and Reasoning

Section E:  Prerequisite Knowledge and Symbol Sense:

Section F:  Statistical Knowledge

Appendix C

Figure 1:  Mean Response by Gender 5 Confidence w/ Mathematics Items

Figure 2:  Mean Response by Gender on 3 Logical Thinking & Reasoning Items

Figure 3:  Mean Response by Grouped Age on Item: “thinking in terms of numbers to support claims.”