Student success is critical to the mission of every higher educational institution. All other things being equal, students who are the most intellectually and psychologically prepared for higher education will be the most successful. While most universities assess the intellectual readiness of their applicants via standardized testing, few measure their psychological readiness.
At Stevens Strategy, we have developed the Applicant Success Assessment (ASA) to accurately measure the psychological preparedness of higher education student applicants. ASA was developed after testing with a national sample of 1,000 college graduates and non-graduates. Based upon this testing, we conservatively predicted ASA’s non-cognitive score combined with institutional cognitive scores would improve retention by 5% at a typical 4-year private college, generating about $3,000,000 in additional income. In previous reports we have discussed how the ASA can be used to improve graduation rates and to reduce costs of student attrition and shown that the ASA accurately predicts first year student GPA.
This report follows up on these findings by examining the degree to which the ASA can predict Fall to Fall retention and cumulative GPA in students from two actual Beta clients. At these clients, we used ASA alone (not combined with cognitive scores like HSGPA or SAT/ACT scores) to predict retention.
The results are significantly better than our conservative prediction of 5%, ranging from 15% to 29% and resulting in multiple millions of dollars in additional income. Combing ASA with cognitive scores will make the predictive value even greater!
Both clients are small, private institutions in the northeastern United States. We sent the ASA to all matriculants at both institutions at the beginning of the fall 2016 semester. We received responses from 180 and 163 students at the two institutions respectively. Using institutional data, we next set out to see how well the ASA predicted fall to fall retention and cumulative (first year) GPA.
Because the ASA consists of 55 items, we can employ sophisticated predictive modeling techniques, such as genetic algorithms, to build models that maximize prediction of student performance on the basis of their responses to all 55 items. For the first client, ASA responses predicted retention at R = .37 and R = .55 (see first Figure below). For the second client, these values were R = .46 and GPA at R = .53 (see second Figure below), respectively.
In real world terms, these values are consequential. A test with predictive validity of r = .40 means that people who score above the average (median) on the test have a 70% chance of scoring above the average (median) on the outcome. Conversely, those who score below the average (median) on the predictor only have a 30% chance of scoring above the average (median) on the outcome. Thus, knowing someone’s score on the ASA dramatically improves your ability to predict that person’s GPA and probability of retention.
Data from two clients at two separate institutions now definitely show that non-cognitive skills are an important component of student success. With proper assessment – such as the ASA – schools can use information about non-cognitive skills to improve admissions decisions, allocate financial aid more effectively, and identify students who are likely to struggle with higher education and develop interventions to assist them. In our view, this is quite impressive for an assessment that takes less than 5 minutes to complete and costs only fractions of the dollars it saves.