Chenna Reddy Cotla - Machine Learning Methods to Measure the Heterogeneity in the Causal Effects of Conditions for Learning on Student Outcomes

Title: Machine Learning Methods to Measure the Heterogeneity in the Causal Effects of Conditions for Learning on Student Outcomes

Abstract: While there are many individual pathways to excellence in learning and development, they are all mediated by social and emotional skills and conditions for learning (Osher & Berg, 2017). Salient conditions for learning include feeling physically and emotionally safe, connected and supported and engaged and challenged.  Contextual variables including demographic characteristics, family circumstances, and neighborhood contexts mediate the ways conditions for learning impact student engagement and performance. Identifying variation in the effects of conditions for learning is crucial because it enables the most efficient use of limited resources by informing how to design and target potential interventions. Data partitioning methods in Machine Learning are well suited for answering questions related to causal effect variation. We illustrate the application of Machine Learning based methods by studying the heterogeneous impacts of the perception of school safety on student outcomes in a large school district in the United States. We leveraged the data on the perception of safety measured as a part of the conditions for learning survey in a large school district consisting of 40,000 students. We combine the survey data with school administrative records and set up a quasi-experimental design to study the causal effects of perceiving low safety on absenteeism and performance measured by GPA. Average treatment effect (ATE) of low safety is +2.20 days on absenteeism and -0.18 points on GPA. We extend this analysis by fitting causal decision trees in Machine Learning to estimate heterogeneity in the causal effects along several covariates including demographics, other perceived conditions for learning, and neighborhood level characteristics. We find that the effects of low safety on absenteeism are the largest for African American males with a low perception of peers’ Social and Emotional Learning (SEL) competency. Conditional Average Treatment Effect (CATE) of low safety on absenteeism for this group is 11 days and represents a 58% increase at the mean. On the other hand, the negative effects of low safety on performance are largest in low-income neighborhoods. Elementary and Middle school students living in low-income neighborhoods who reported low SEL competency of peers saw a drop of -0.55 points on their GPA due to low safety. This effect corresponds to a decline of 22% at the mean for this subgroup.

Presenter: Chenna Reddy Colta, American Institutes for Research

The program coordinator of these seminars is Jiao Wang. If you would like to subscribe to the Melbourne Institute Seminar Series email list, please contact Jiao.