Guideline Statement #5

Best Practices for Program Review, Assessment & Accreditation

Each department of mathematical sciences should participate at regular intervals in a process of program review or assessment. These program reviews should focus on internal reflection and improvement, though they may also be part of larger assessment and accreditation protocols. Through the standard cycle (typically between four and ten years) of planning and setting goals, prioritizing and implementing strategies, measuring progress, and reflection [5], departments can respond systematically to needs to update curriculum, pedagogy, technology, approaches to student success, and strategies to promote inclusion and equity. Helpful advice can be found on the MAA’s Committee on Program Review web page, including the Guidelines for Undertaking a Self-Study in the Mathematical Sciences [1] as well as from Hanover Research [3] and in MAA Focus [6].

Participants in the process should include department faculty at all ranks, students, alumni, client departments, deans and other appropriate administrative officials, and external mathematical sciences reviewers. Specific departmental faculty members should be assigned to lead the process, with at least one member of the leadership team having strengths in organization, evaluation, or management.

Reviewers, internal or external, should have sufficient disciplinary knowledge to engage with departmental faculty members and to understand the program’s mission. Guidelines for serving as a reviewer have been published by the MAA’s Committee on Program Review [2]. The cycle should include a self-study “to reflect on where the department has been, where it is now, and where it wants to be” [1]. The cycle should also include a strategic plan that is acceptable to the department and to its dean in order to move forward -- enhancing strengths and addressing opportunities to improve identified in the planning and evaluation process.

The major components of the planning and evaluation process should include:

1) A statement that clearly defines the mission of the undergraduate mathematical sciences department.

2) A delineation of the educational goals of relevant curricula or programs, as well as a statement of how attainment of these goals is expected to fulfill the stated mission of the curricula.

3) Procedures for measuring the extent to which the educational goals are being met. These measures will, of necessity, be multi-dimensional since no single statistic can adequately represent departmental performance with respect to most departmental goals. Measures of student learning and other student outcomes should be included in the procedures. Advice regarding these measures has been published by the MAA Committee on Program Review [1].

4) A process for regularly reviewing (and revising, if necessary) departmental and academic program components in light of measurements of program success.

5) A departmental and institutional plan to procure and allocate, over time, the resources needed to implement the strategic plan agreed to by the department and its dean.

6) Consistency with institutional processes.

The periodic reviews should examine all aspects of the department’s undergraduate academic program. Main components of a review should include, but not be limited to, the following sections.

1. Institutional context  2. Curriculum (as appropriate to institutional policies for review) 3. Community: students, alumni, advising 4. Faculty profiles 5. Assessment: surveys, other means of measuring program quality    

1. Institutional context

a. What is the relevant history of the department/program/curriculum (especially looking back to the time of the last review) and its relation to the college/university?

b. What is the mission of the department/program/curriculum and its student learning goals?

c. What are the resources (faculty, support staff, budget, facilities, technical support, library) of the department/program/curriculum?

2. Curriculum (as appropriate to institutional policies for review)

a. In what ways does the department/program offer a curriculum that serves the general student body? Does this service meet the goals of the institution’s general education requirements? Does this service meet the needs of client departments/programs?

b. What are the curricular goals of the department/program/curriculum? How do the curricular offerings and the structure of the major support those goals, the mission of the department, and its student learning goals?

c. How does the department/program/curriculum judge curricular quality and effectiveness? On that basis, how well is the department/program/curriculum doing?

d. In what ways do the curriculum and structure of the major adhere to or deviate from the most recent CUPM recommendations [4]?

3. Community: students, alumni, advising (as appropriate to institutional policies for review)

a. What is the department’s current student enrollment in each program? How has that changed over time? What trends are anticipated?

b. What is the department’s current student enrollment in general education courses and courses in support of other programs? How have they changed over time? What trends are anticipated?

c. What are the department’s current recruiting, retention, and advising practices? How effective are they, and how is that effectiveness judged?

d. What efforts are made to recruit, retain, advise, and support and students from underrepresented groups? How effective are these efforts, and how is that effectiveness judged?

e. What is the employment profile for graduates of the program? How satisfied are alumni/ae and their employers with their preparation in the program and their ability to find sustained employment or pursue advanced education in their field, and how is that satisfaction judged?

4. Faculty profiles

a. In what ways does the current faculty complement support departmental and institutional missions, including community engagement, university service, and national visibility?

b. Do current faculty (tenured, tenure-track, non tenure-track) complement meet departmental and institutional needs for diversity and representation? If not, what needs are anticipated?

c. What needs are anticipated or currently unmet in terms of specific sub-discipline representation and support for faculty scholarship and professional engagement?

5. Assessment: surveys, other means of measuring program quality

a. What assessment evidence is there to address the components already listed above?

b. Further components questions to consider to measure program quality may include:

i) How have students performed in seminars, departmental comprehensive examinations, course-embedded assessment, undergraduate research activities, internships, consulting experiences, and national competitions and examinations?

ii) What is the opinion of students, obtained through surveys and interviews? (These should address the program broadly, and not individual instructors.)

iii) What accomplishments and awards have been received by the current students and graduates of the department’s programs, as well as the department faculty?

iv) How do numbers of mathematical sciences majors and alumni compare to peer departments and to national averages?

v) How well do associate degree recipients who transfer to four-year colleges succeed in your program? Alternatively, what is the success of these recipients that transfer to your four-year program?

vi) How well do bachelor’s degree recipients succeed in graduate programs and other post-graduate opportunities?

vii) How well do underrepresented minorities (URMs) succeed in your program? How well do URMs who graduate from your program succeed in graduate programs and other post-graduate opportunities?

References

[1] Guidelines for Undertaking a Self-Study in the Mathematical Sciences, MAA Committee on Program Review, 2010.  Available at:  https://www.maa.org/sites/default/files/pdf/ProgramReview/MAA-SelfStudyManual.pdf

[2] Guidelines for Serving as a Consultant in the Mathematical Sciences, MAA Committee on Program Review, 2010.  Available at:  https://www.maa.org/sites/default/files/pdf/ProgramReview/ConsultantManual.pdf,

[3] Best Practices in Academic Program Review, Hanover Research, 2012.  Available at:  https://cpb-us-w2.wpmucdn.com/www.paulsmiths.edu/dist/6/103/files/2017/06/FD_HanoverResearch_BestPractices_ProgramReview-1c7556t.pdf

[4] 2015 CUPM Curriculum Guide to Majors in the Mathematical Sciences, MAA Committee on the Undergraduate Program in Mathematics, 2015.  Available at:  https://www.maa.org/sites/default/files/pdf/pubs/books/members/cupm_guide.pdf

[5] “Ways to Approach the Quality Improvement Process”, The CAHPS Ambulatory Care Improvement Guide: Practical Strategies for Improving Patient Experience, Consumer Assessment of Healthcare Providers and Systems (Agency for Healthcare Research and Quality), 2016.  Available at:  https://www.ahrq.gov/cahps/quality-improvement/improvement-guide/4-approach-qi-process/index.html

[6] “Getting Started with Program Review”, Walker, H., MAA Focus, June/July 2020.  Available at:  http://digitaleditions.walsworthprintgroup.com/publication/?m=7656&i=663226&p=20

Contributors

Ed Aboufadel, Grand Valley State University
Emily Puckette, University of the South
Audrey Malagon, Virginia Wesleyan University
Mary Pilgrim, Colorado State University
Jason Douma, University of Sioux Falls
Benedict Nmah, Morehouse College
Jill Dietz, St. Olaf College
Tim Flowers, Indiana University of Pennsylvania
Debra Lynn Hydorn, University of Mary Washington

October 2020