SAUM Additional Online Case Studies & Appendices

 


Arapahoe Community College

Mathematics Assessment in the First Two Years

Dr. Erica Johnson – Department Chair
Jeffrey Berg
David Heddens

I.          Abstract

During the last two years, the Arapahoe Community College Mathematics Department participated in a college-wide effort to expand assessment activities to the program and discipline level.  A report at http://www.arapahoe.edu/custom/SAmathematics.html details accomplishments during the first year.  This report details second-year accomplishments that focused on analyses of data from College Algebra entrance and common final exams.

Appendix A

Contents  
Table1 Learning Outcomes Addressed by Departmental Courses
Table 2 Colorado Community College System Core Transfer Program Student Learning Outcomes for College Algebra
Table 3             College Algebra Common Final (Calculator Based Calculus Readiness) Question Learning Outcomes
Table 4 College Algebra Uniform Final Fall 2002 Grading Curve
Table 5          College Algebra Uniform Final Spring 2003 Grading Curve
Table 6             College Algebra Common Final Spring 2003 Rank Order of Student Performance on Questions
Table 7             Distributions of Final Exam Correct Answer Rates, AY 2001
Table 8             Distributions of Final Exam Correct Answer Rates, AY 2002
Table 9             Distributions of Final Exam Correct Answer Rates, Spring 02 and Spring 03
Table 10            Tests of Each Question for Significantly Different Correct Response Rates, Spring 2002 Form 1H and Spring 2003 Form 1D
Table 11            Linkage Across Calculus Readiness (1H) and Calculator-Based Calculus Readiness (1D)  Versions of the Common Final
Table 12            Student Enrollment in College Algebra 2002-2003 Academic Year
Table 13            Student Entrance Exam/Common Final Exam Completion Status
Table 14            Entrance Exam Competency Measurement Distributions
Table 15            Final Exam Competency Measurement Distributions
Table 16            Contingency Table Cross-Classifying Students by Their Entrance and Final Competency  Measures, Fall 2002.
Table 17            Contingency Table Cross-Classifying Students by Their Entrance and Final Competency  Measures, Spring 2003.
Table 18            Student Learning Outcome Strengths and Weaknesses College Algebra Common Final Spring 2003
Table 19            Calculus I Common Project Scoring Rubric
Table 20            Assessment Methods Used to Measure Student Learning Outcomes

Table1  Learning Outcomes Addressed by Departmental Courses

               
 

Students will acquire the ability to read, write, listen to, and speak mathematics

Students will demonstrate a mastery of competencies identified by the competency-based syllabi for specific courses.

Students will use appropriate technology to enhance their mathematical thinking and understanding and to solve mathematical problems and judge the reasonableness of their results.

Students will engage in substantial mathematical problem solving.

Students will acquire the ability to use multiple approaches-numerical, graphical, symbolic, and verbal-to solve mathematical problems.

Fund of Math

x

x

 

x

 

Pre-Algebra

x

x

 

x

 

Intro Algebra

x

x

 

x

x

Math Support

 

x

     

Survey of Algebra

x

x

x

x

x

Tech Lab for Algebra

 

x

x

   

Applied Math I

x

x

 

x

x

Applied Math II

x

x

 

x

x

Math for Pre-Secondary Teachers I

x

x

 

x

x

Math for Liberal Arts

x

x

x

x

x

College Algebra

x

x

x

x

x

Trigonometry

x

x

x

x

x

Finite Math

x

x

x

x

x

Survey of Calculus

x

x

x

x

x

Statistics

x

x

x

x

x

Statistics Lab

 

x

x

x

x

Calculus I

x

x

x

x

x

Calculus II

x

x

x

x

x

Calculus III

x

x

x

x

x

Math for Pre-Secondary Teachers II

x

x

 

x

x

Linear Algebra

x

x

x

x

x

Differential Equations

x

x

x

x

x


Table 2  Colorado Community College System Core Transfer Program

Student Learning Outcomes for College Algebra

http://www.cterc.cccoes.edu/cccns/index.html

A.            Be familiar with set notations, subsets of the real numbers and properties of real numbers.

B.            Perform algebraic manipulations including working with exponents, radicals, polynomial operations, factoring and algebraic fractions.

C.            Present methods for solving first and second degree equations and inequalities and related topics.

D.            Solve the following types of equations: linear, quadratic, equations involving radicals, equations in quadratic form and equations involving absolute value.

E.            Work with formulas including formula evaluation and solving a formula for any of the variables.

F.            Read and analyze problems in the form of word problem applications and obtain solutions using equations.

G.           Solve first degree inequalities, higher degree inequalities and inequalities involving absolute value.

H.            Recognize and graph linear functions, rational functions, absolute value functions, and graph inequalities in two variables.

I.               Work with function notation and demonstrate knowledge of the meaning “function”.

J.             Demonstrate an understanding of function composition, one-to-one functions and inverse functions.

K.            Examine, evaluate and graph exponential functions.

L.             Examine, evaluate and graph logarithmic functions.

M.           Work problems and solve equations containing exponential and logarithmic functions.

N.            Explore a variety of techniques used to solve linear and non-linear systems of equations.

O.           Use at least two of the following techniques to solve linear and non-linear systems of the equations: substitution, addition, Gaussian elimination, Cramer’s rule.

P.            Have some familiarity with matrices and operations involving matrices.

Q.           Graph systems of inequalities.

R.            Graph conic sections including circles, parabolas, ellipses and hyperbolas.

S.            Identify the conic section represented by a given second degree equation.

T.             Introduce various topics related to sequences and series.

U.            Work with series notation and sequence formulas, and counting principles.

V.            Apply the Binomial Theorem.

W.          Demonstrate an understanding of proof by mathematical induction.

X.             Present topics in theory equations.

Y.            Perform synthetic division.

Z.             Use the Remainder Theorem and the Factor Theorem to factor and evaluate polynomials.

AA.       Solve polynomial equations using the Rational Root Theorem and/or approximation techniques.

BB.       Write and speak clearly and logically in presentations and essays.

CC.       Demonstrate the ability to select and apply contemporary forms of technology to solve problems or compile information.


Table 3:  College Algebra Common Final (Calculator Based Calculus Readiness) Question Learning Outcomes

Question

State Core Transfer Program Learning Outcome

MAA Placement Test Competency

1

O.  Use at least two of the following techniques to solve linear and non-linear systems of equations: substitution, addition, Gaussian elimination, Cramer’s rule.

Graphs of Functions, Equations and Factoring

2

F. Read and analyze problems in the form of word problem applications and obtain solutions using equations.

Word Problems, Modeling, Numerical Awareness, Exponential Functions

3

I.  Work with function notation and demonstrate knowledge of the meaning “function”.

Graphs of Functions, Inequalities, Absolute Value

4

E. Work with formulas including formula evaluation and solving a formula for any of the variables.

Geometry and Measurement, Word Problems, Modeling

5

J. Demonstrate an understanding of function composition, one-to-one functions, and inverse functions.

Exponents and Logarithms

6

H.  Recognize and graph linear functions, rational functions, absolute value functions, and graph inequalities in two variables.

Graphs of Functions, Concept Formulation

7

H.  Recognize and graph linear functions, rational functions, absolute value functions, and graph inequalities in two variables.

Graphs of Functions

8

M.  Work problems and solve equations containing exponential and logarithmic functions.

Exponents and Logarithms

9

E. Work with formulas including formula evaluation and solving a formula for any of the variables.

Geometry and Measurement

10

U. Work with series notation and sequence formulas, and counting principles.

Concept Formulation, Numerical Awareness, Exponents and Logarithms

11

K. Examine, evaluate, and graph exponential functions.

Graphs of Functions, Exponential Functions

12

B.  Perform algebraic manipulations including working with exponents, radicals, polynomial operations, factoring, and algebraic fractions.

Equations and Factoring

13

I.  Work with function notation and demonstrate knowledge of the meaning “function”.

Geometry and Measurement, Graphs of Functions

14

B.  Perform algebraic manipulations including working with exponents, radicals, polynomial operations, factoring, and algebraic fractions.

Equations and Factoring

15

I.  Work with function notation and demonstrate knowledge of the meaning “function”.

Function Notation

16

H.  Recognize and graph linear functions, rational functions, absolute value functions, and graph inequalities in two variables.

Graphs of Functions

17

G. Solve first degree inequalities, higher degree inequalities, and inequalities involving absolute value.

Inequalities, Absolute Value

18

F. Read and analyze problems in the form of word problem applications and obtain solutions using equations.

Geometry and Measurement, Word Problems, Modeling

19

H.  Recognize and graph linear functions, rational functions, absolute value functions, and graph inequalities in two variables.

Word Problems, Modeling, Concept Formulation

20

F. Read and analyze problems in the form of word problem applications and obtain solutions using equations.

Geometry and Measurement, Word Problems, Modeling


 

Table 4:  College Algebra Uniform Final Fall 2002 Grading Curve

Exam Score (out of 20)

Curved percentage

20

 

19

98

18

96

17

93

16

90

15

88

14

86

13

83

12

80

11

78

10

75

9

72

8

68

7

65

6

62

5

60

4

Instructor discretion

3

 

2

Instructor discretion

1

 

0

 

Table 5:  College Algebra Uniform Final Spring 2003 Grading Curve

Exam Score (out of 20)

Curved percentage

20

 

19

100

18

98

17

96

16

94

15

92

14

90

13

88

12

85

11

82

10

80

9

77

8

73

7

70

6

67

5

63

4

60

3

56

2

53

1

50

0

instructor discretion


Table 6: College Algebra Common Final Spring 2003 Rank Order of Student Performance on Questions

Question

% Correct

State Core Transfer Program Learning Outcome

19

88.33%

H

11

79.44%

K

15

79.44%

I

3

74.44%

I

17

72.22%

G

9

70.00%

E

18

69.44%

F

16

67.22%

H

4

63.89%

E

7

63.33%

H

12

63.33%

B

14

62.78%

B

5

57.78%

J

6

51.67%

H

10

48.33%

U

20

43.89%

F

13

42.78%

I

1

41.11%

O

8

41.11%

M

2

36.11%

F


Table 7  Distributions of Final Exam Correct Answer Rates, AY 2001

 

exams

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

Fall 01

214

132

140

138

179

74

129

199

131

199

105

193

144

142

201

116

101

86

133

138

177

proportion

 

0.6

0.7

0.6

0.8

0.3

0.6

0.9

0.6

0.9

0.5

0.9

0.7

0.7

0.9

0.5

0.5

0.4

0.6

0.6

0.8

Spring 02

159

100

104

90

121

72

121

154

92

145

91

146

107

110

152

91

70

55

110

103

137

proportion

 

0.6

0.7

0.6

0.8

0.5

0.8

1

0.6

0.9

0.6

0.9

0.7

0.7

1

0.6

0.4

0.3

0.7

0.6

0.9

Table 8  Distributions of Final Exam Correct Answer Rates, AY 2002

 

exams

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

Fall 02

158

49

36

109

111

84

65

94

58

113

68

121

78

48

78

121

89

93

103

128

60

proportion

 

0.3

0.2

0.7

0.7

0.5

0.4

0.6

0.4

0.7

0.4

0.8

0.5

0.3

0.5

0.8

0.6

0.6

0.7

0.8

0.4

Spring 03

180

74

65

134

115

104

93

114

74

126

87

143

114

77

113

143

121

130

125

159

79

proportion

 

0.4

0.4

0.7

0.6

0.6

0.5

0.6

0.4

0.7

0.5

0.8

0.6

0.4

0.6

0.8

0.7

0.7

0.7

0.9

0.4

Table 9  Distributions of Final Exam Correct Answer Rates, Spring 02 and Spring 03

Form 1H

exams

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

S02

159

100

104

90

121

72

121

154

92

145

91

146

107

110

152

91

70

55

110

103

137

proportion

 

0.6

0.7

0.6

0.8

0.5

0.8

1

0.6

0.9

0.6

0.9

0.7

0.7

1

0.6

0.4

0.3

0.7

0.6

0.9

Form 1D

exams

4

3

2

1

6

 

7

8

11

12

 

9

15

16

13

14

20

17

18

19

S03

180

115

134

65

74

93

 

114

74

143

114

 

126

143

121

77

113

79

130

125

159

proportion

 

0.6

0.7

0.4

0.4

0.5

 

0.6

0.4

0.8

0.6

 

0.7

0.8

0.7

0.4

0.6

0.4

0.7

0.7

0.9

Table 10            Tests of Each Question for Significantly Different Correct Response Rates, Spring 2002 Form 1H and Spring 2003 Form 1D

Form 1H

exams

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

Form 1D

exams

4

3

2

1

6

 

7

8

11

12

 

9

15

16

13

14

20

17

18

19

 

z

0.1

1.2

-4

-6

1.2

 

-4

-3

-1

0.9

 

0.4

1.3

-3

-3

3.3

2.2

0.4

0.6

0.2

 

p-val

0.9

0.2

0*

0*

0.2

 

0*

0*

0.2

0.4

 

0.7

0.2

0*

0**

0*

0.03

0.7

0.5

0.8

*  p-value was less than 0.002

** pvalue was less than 0.01


Table 11  Linkage Across Calculus Readiness (1H) and Calculator-Based Calculus Readiness (1D)  Versions of the Common Final

Question 1H

Question 1D

Student Learning Outcome

1

4

E. Work with formulas including formula evaluation and solving a formula for any of the variables.

2

3

I.  Work with function notation and demonstrate knowledge of the meaning “function”.</p>

3

2

M. Work problems and solve equations containing exponential and logarithmic functions.

4

1

O.  Use at least two of the following techniques to solve linear and non-linear systems of equations: substitution, addition, Gaussian elimination, Cramer’s rule.

5

6

H.  Recognize and graph linear functions, rational functions, absolute value functions, and graph inequalities in two variables.

6

None

B.  Perform algebraic manipulations including working with exponents, radicals, polynomial operations, factoring, and algebraic fractions.

7

7

H.  Recognize and graph linear functions, rational functions, absolute value functions, and graph inequalities in two variables.    (category D is possible)

8

8

M.  Work problems and solve equations containing exponential and logarithmic functions.

9

11

K. Examine, evaluate, and graph exponential functions.

10

12

B.  Perform algebraic manipulations including working with exponents, radicals, polynomial operations, factoring, and algebraic fractions.

11

None

B. Perform algebraic manipulations including working with exponents, radicals, polynomial operations, factoring, and algebraic fractions.

12

9

E. Work with formulas including formula evaluation and solving a formula for any of the variables.

13

15

I.  Work with function notation and demonstrate knowledge of the meaning “function”.

14

16

H.  Recognize and graph linear functions, rational functions, absolute value functions, and graph inequalities in two variables.

15

13

I.  Work with function notation and demonstrate knowledge of the meaning “function”.

16

14

B.  Perform algebraic manipulations including working with exponents, radicals, polynomial operations, factoring, and algebraic fractions.

17

20

F. Read and analyze problems in the form of word problem applications and obtain solutions using equations.

18

17

G. Solve first degree inequalities, higher degree inequalities, and inequalities involving absolute value.

19

18

F. Read and analyze problems in the form of word problem applications and obtain solutions using equations.

20

19

H.  Recognize and graph linear functions, rational functions, absolute value functions, and graph inequalities in two variables.


Table 12 Student Enrollment in College Algebra 2002-2003 Academic Year

Section Fall 02

Student Count

Section  Spr 03

Student Count

L01

28

L01

29

L02

30

L02

28

L03

25

L03

29

L04

34

L04

24

L05

32

L05

12

L06

38

L06

31

L07

30

L07

31

L08

26

L08

17

L48

22

L09

20

L85

14

L10

20

TOTAL

279

L66

26

   

TOTAL

267

Table 13 Student Entrance Exam/Common Final Exam Completion Status

Entrance Exam Completed

Final Exam Completed

Student Count Fall ‘02

Student Count Spring ‘03

Yes

Yes

119

168

Yes

No

121

87

No

Yes

39

12

No

No

0

0

 

TOTAL

279

267

Table 14  Entrance Exam Competency Measurement Distributions

Correct Responses on Entrance Exam

Proportion Correct on Entrance Exam

Student Count Fall ‘02

Student Count Spring ‘03

0

0

8

18

1

0.5

41

64

2

1

70

86

 

TOTAL

119

168

Table 15  Final Exam Competency Measurement Distributions

Correct Responses on Final Exam

Proportion Correct on Final Exam

Student Count Fall ‘02

Student Count Spring ‘03

0

0

19

31

1

0.333

28

61

2

0.666

39

45

3

1

33

31

 

TOTAL

119

168

Table 16  Contingency Table Cross-Classifying Students by Their Entrance and Final Competency  Measures, Fall 2002.

Fall ‘02

Correct Responses on Final Exam

 

0

1

2

3

Correct Responses on Entrance Exam

0

3

2

3

0

8

1

6

13

13

9

41

2

10

13

23

24

70

 

19

28

39

33

119

Table 17  Contingency Table Cross-Classifying Students by Their Entrance and Final Competency  Measures, Spring 2003.

Spring ‘03

Correct Responses on Final Exam

 

0

1

2

3

Correct Responses on Entrance Exam

0

6

8

4

0

18

1

16

21

16

11

64

2

9

32

25

20

86

 

31

61

45

31

168


Table 18  Student Learning Outcome Strengths and Weaknesses

College Algebra Common Final Spring 2003


 



Table 19  Calculus I Common Project Scoring Rubric

 

10 points

9 points

7 points

5 points

0 points

Problem Statement

The student(s) provide an accurate statement of the problem in the introduction of the written presentation.  All components of the problem are presented correctly.

The student(s) provide a mostly accurate statement of the problem in the introduction of the written presentation.  Some minor components of the problem are missing or incorrect.

The student(s) provide a fairly complete statement of the problem in the introduction of the written presentation.  Major components of the problem are missing or incorrect.

The student(s) provide a vague statement of the problem in the introduction of the written presentation.  Major components of the problem are missing and the statement may include extraneous components.

The student(s) provide no introduction addressing the statement of the problem in the written presentation.

 

20 points

17 points

15 points

10 points

0 points

Problem Solution

The student(s) correctly and completely apply appropriate mathematical principles and algorithms to obtain solution.  All conditions of the problem are satisfied and strong supporting arguments are exhibited. Efficient, focused, and rigorous.

The student(s) correctly and completely apply appropriate mathematical principles and algorithms to obtain solution.  All essential conditions of the problem are satisfied but some minor conditions are overlooked.  Sufficient supporting arguments are exhibited.  Minor problems with efficiency, focus, and rigor do not detract from solution presentation.

The student(s) apply appropriate mathematical principles and algorithms to obtain a nearly complete and correct solution.  Some essential conditions of the problem are overlooked and/or few supporting arguments are exhibited.  Lack of efficiency, focus, and/or rigor detracts from solution presentation.

The student(s) apply some appropriate and/or unrelated mathematical principles and algorithms to obtain an incomplete and incorrect solution.  Several essential conditions of the problem are overlooked and supporting arguments are virtually nonexistent. Inefficient, unfocused, and superficial.

The student(s) present no solution.

 

15 points

13 points

11 points

7 points

0 points

Mathematical Notation

The student(s) consistently use correct mathematical notation and terminology.

The student(s) use correct mathematical notation and terminology except in a few cases.  Incorrect notation and/or terminology do not substantially detract from the presentation of the solution.

The student(s) use correct mathematical notation most of the time.  Incorrect notation and/or terminology slightly detract from the presentation of the solution.

The student(s) consistently use incorrect mathematical notation and terminology that significantly detracts from the presentation of the solution.

The student(s) use no mathematical notation or terminology.

 

10 points

9 points

7 points

5 points

0 points

Graphs

The student(s) clearly indicate axis labels and scale on all graphs.  All graphs are accurate and use an appropriate scale to show important aspects of presented curves.  All curves and important points are clearly labeled

The student(s) clearly indicate axis labels and scale on all graphs.  The student(s) present a few graphs with minor flaws.  Scale on graphs appropriate to show important aspects of presented curves on all but possibly a few graphs.

The student(s) do not include labels and scale on graphs; however, curves are correctly graphed if scale is inferred OR student(s) include labels and scale on graphs; however, some curves are incorrectly graphed.

The student(s) do not include labels and scale on graphs and most curves are correctly graphed if scale is inferred OR most curves are incorrectly graphed.

The student(s) do not include any supporting graphs.


15 points

12 points

10 points

7 points

0 points

Presentation

The overall appearance of the presentation is neat and the paper is easy to read. All pertinent information can be readily found.  The student(s) communicate effectively and clearly to target audience and include all appropriate graphs and diagrams.

The overall appearance of the presentation is neat and the paper is readable. All pertinent information is in the same area or is appropriately referred to if in a different area, but arrangement could be improved.  The student(s) communicate effectively and clearly to target audience; however, a few supporting graphs and diagrams may be missing.

The presentation is hard to read and/or pertinent information is hard to find.   The student(s) communicate to target audience but the presentation contains gaps in the flow and several supporting graphs and diagrams may be missing.

The presentation is hard to read and pertinent information is hard to find.  The student(s) minimally communicate to target audience.  The presentation contains major gaps in flow and few if any supporting graphs and diagrams.

The presentation is illegible, incoherent, and contains no pertinent information or supporting graphs or diagrams.

 

10 points

9 points

7 points

5 points

0 points

Creativity

The presentation is sophisticated and elegant and includes several insightful interpretations, extensions, and generalizations.

The presentation is thorough and appealing and includes some insightful interpretations, extensions, and generalizations.

The presentation is sound but includes few insightful interpretations, extensions, and generalizations.

The presentation marginally extends mathematical concepts and algorithms to obtain the problem solution but includes essentially no interpretations, extensions, or generalizations.

The presentation exhibits no creativity.

 

10 points

9 points

7 points

5 points

0 points

Conclusion/Reflection

The student(s) provide an accurate summary and a complete reflection of mathematical concepts and algorithms exercised.

The student(s) provide a mostly accurate summary but minor aspects of the project are missing.  Reflection of mathematical concepts and algorithms exercised is nearly complete but some minor points are neglected.

The student(s) provide a reasonable summary but key aspects of the project are missing.  Reflection omits key mathematical concepts and algorithms exercised.

The student(s) provide a minimal summary and little reflection of mathematical concepts and algorithms exercised.

The student(s) provide no summary or reflection.

 

10 points

9 points

7 points

5 points

0 points

English

The student(s) include appropriate and sufficient narrative to provide structure to the presentation using complete sentences, correct spelling, correct grammar, and correct punctuation.

The student(s) include appropriate narrative using complete sentences that may include some minor spelling, grammar, and/or punctuation errors.   Narrative may fall slightly short of providing adequate structure to the presentation.

The student(s) include narrative using a few incomplete sentences and/or committing significant spelling, grammar, and/or punctuation errors.   Narrative may fall somewhat short of providing adequate structure to the presentation.

The student(s) include narrative using very few complete sentences and committing significant spelling, grammar, and punctuation errors.  Narrative falls significantly short of providing adequate structure to the presentation.

The student(s) include no narrative to provide structure to the presentation.


Table 20  Assessment Methods Used to Measure Student Learning Outcomes

Outcome/Tool

Project or Portfolio

Standardized Exam

Pre-test/

Post-test

Faculty Survey

Student Survey

Students will acquire the ability to read, write, listen to, and speak mathematics

CP2003

       

Students will demonstrate a mastery of competencies identified by the competency-based syllabi for specific courses.

CP2003

CF2001

PP2002

   

Students will use appropriate technology to enhance their mathematical thinking and understanding and to solve mathematical problems and judge the reasonableness of their results.

CP2003

CF2001

     

Students will engage in substantial mathematical problem solving.

CP2003

       

Students will acquire the ability to use multiple approaches-numerical, graphical, symbolic, and verbal-to solve mathematical problems.

CP2003

       

CF2001 -  College Algebra common final data collection and analysis began in 2001-2002 academic year

PP2002 -  College Algebra entrance exam/common final data collection and analysis began in 2002-2003 academic year

CP2003 -  Calculus I common projects with scoring rubric beginning in 2003-2004 academic year