Academic Program Review Process & Assessment
Stevens Strategy has developed Programs and Resource Optimization (PRO), a transformational academic program review process. PRO is an analytical review of academic programs that engages the institution and involves faculty, staff, alumni and trustees.
The academic program review process has 4 key analytic components:
- Responsibility Center Data-based Analysis
Each academic program is reviewed and measured based upon Mission, Quality, Marketability, Net Income and ranked within three tiers: From extremely successful to extremely unsuccessful. With the analytical output of PRO, institutions can shift resources and develop goals that optimize academic program offerings and strategically manage change. These findings can drive transformational decisions that result in significant shifts in institutional quality and financial performance.
More About the PRO Process from Stevens Strategy
We have developed over the years a trans formative academic program review process we call PRO. It has four interrelated processes.
- A Mission-centeredness Review that identifies the programs that meet the college’s mission and those that don’t. Mission-centeredness is measured through carefully crafted surveys of the institution’s primary constituencies; its students, faculty, staff, alumni and trustees. These groups understand the true mission of the institution and inherently recognize what programs are vital to achieving both the stated and implied mission of the college.
- A Quality Review that identifies those programs that are of high quality and those that aren’t. Quantitative indicators vary by institution, but acceptance rates, retention rates, graduation rates and placement rates are often among the major quality indicators everywhere. Qualitative measures include surveys of key constituencies regarding their opinion of program quality.
- A Market Demand Review that compares, through survey research, demand for current and prospective programs among existing and potential student populations.
- An Institution-wide Responsibility Center Review that identifies the divisions or schools that generate cash and those that use it. It allocates fairly to each revenue-generating department revenue and direct and indirect costs, showing the way to making thoughtful resource allocation decisions.
The most important result of a full academic program review is that presidents have the opportunity to lead transformational change at their institutions and keep their jobs! Other equally important results are the definition of ranges of programs ranked in three tiers from extremely successful ones that provide a surplus of resources to the institution, are highly mission-centered, are of high quality, and experience solid market demand to a few that not only lose money, but do not support the college’s mission, are of questionable quality and have little market support.
Replacing the losses from non-mission-essential, low quality programs that have weak market demand (the bottom few programs) with enhanced enrollments and additional revenues from the top tier programs can cause a dramatic shift in institution financial performance; $500,000 or more for small institutions and millions for larger ones.
Of the factors affecting academic program reviews, competition is the most important driver. Competition in this decade will create simultaneous pressures to reduce price, control costs and adjust program mix. So, institutions need to carefully manage costs; add new revenue programs or revenue sources at least every other year; and weed out poorly performing programs regularly. To respond to the competitive environment, nothing is more important than knowing what programs you should be offering, what programs you should not be, and acting on that information.
Accreditors are increasingly reviewing college efforts towards assessing academic programs as an important element of reaccreditation, as you all know. They are particularly interested in objective measures of student learning outcomes and usually do not require a more comprehensive review, but are impressed with more.
Boards of Trustees want a full program assessment, but don’t usually know how to ask for it. I’ve heard this complaint from many Trustees. I’m sure you have too, “My College is always starting new programs, but I’ve never seen it stop one!” The closest current board practices come to a full program assessment is cycled Academic Program Reviews, which typically result in long narrative reports with recommendations for remediation where weaknesses are identified. It’s unusual for any significant action to be taken, and more unusual for program closures to result. The bottom line is: Current board practices fall short of a full academic program evaluation, but most boards would desire a full program evaluation process if they knew how to get it done.
Most of our faculties want sound assessment of curricular goals and student performance, but our faculties are usually uncomfortable with reviews of their judgments about academic programs. They find financial performance reviews dangerous on their own and are deeply concerned with the idea of closing poor performing programs. Transformative Academic Program reviews account for this sensitivity by engaging faculty intensively with the rest of the institution in a comprehensive review that is required by the board.
A Transformational Academic Program Review requires the cooperation of many constituencies (particularly faculty) and can’t be conducted in a locked room in the basement of the administrative office building. To be conducted effectively and to engender broad institutional support, these reviews require involvement by a broad cross-section of the institution in developing the analysis and considering the ramifications of the results of that analysis:
The programs that require more support, those that require less, and yes, those that should be closed.
- First, the Board should require the Academic Program Review by formal vote. This often is followed by a generative institutional discussion, including the board.
- The President should then carefully select a broadly representative cross-functional work group (composed of a majority of faculty leaders) to conduct the analysis.
- The consultants will help the President select the work group and prepare a careful charge with firm outcome expectations and time-lines.
- Basic research to support the deliberations of the work group should be conducted by the consultants and the college’s institutional research office BEFORE the work group begins its formal deliberations.
- The work group should develop a detailed work plan to conduct its study and submit it to the president for approval.
- The work group should then conduct its deliberations and hold 4 open campus meetings to present its work plan, to share its basic research and to discuss its recommendations as they develop. At these open campus meetings, facilitated by the consultant, the work group should honestly seek feedback and deal directly with issues raised.
- After making final edits resulting form the fourth OCM, the work group’s final report should be presented to the president and accepted.
- The president should present the work group’s report and his or her recommended actions to the board for approval; actions should include the conduct of necessary internal governance reviews on a board required time-line.
The optimum time frame for an academic program review is as follows:
1. Early spring semester
• Board Decision
2. Mid to late spring
• President Appoints, Charges and Announces Work Group
3. Early summer
• Consultants Conduct Research
4. Late summer
• Work group convenes
5. Early September
• First Open Campus meeting
6. Throughout fall semester
• Deliberations, Three Open Campus Meetings, Final Report
As I said earlier, in a highly competitive environment like the one in which we will live this decade, nothing is more important than knowing what programs you should be offering, what programs you should not be, and acting on that information. Stevens Strategy‘s PRO, Programs and Resource Optimization process, provides presidents and boards the necessary information and the broad institutional support they need to act responsibly.
We would be delighted to know your thoughts on this important process.
– John Stevens