Megastudy finds easy ways to increase math progress

American students have lagged behind mathematics for decades, with a consistently ranked test score of 25% worldwide compared to students in other developed countries, and the situation has been exacerbated by the Covid-19 pandemic.

Previous research has shown that behavioral science-based interventions targeting student motivations are effective in increasing mathematical scores, suggesting that taking a similar “behaviorally informed” approach to teachers is comparable.

A collaborative study currently published in the minutes of the National Academy of Sciences and led by researchers on Behavior Change for Behavior Change (BCFG) at the University of Pennsylvania found a slight improvement in student mathematics progress compared to control messages.

“Our results showed that simple, low-cost nudges help teachers support mathematics students’ progress,” says Professors Angela Duckworth, Rosa Lee and Egbert Chan of Penn’s Faculty of Arts and Sciences and Wharton School. “These nudges worked across different school contexts and lasted eight weeks after teachers stopped receiving nudges.”

The huge key was a partnership with Zearn Math, a non-profit education platform. “Thanks to our partnership with Zearn Math, we have rarely experienced large-scale research on teacher-focused interventions, and we have been able to overcome these challenges.”

The giant is a massive experiment where co-authors Katie Milkman and James G. Dinan are professors of operation, information and decision making at Wharton School and co-director James G. Dinan, professors of decisions, and multiple interventions are tested simultaneously with the same outcome, tournament approach.” “This method allows for a direct comparison of their effectiveness, as all interventions are performed simultaneously and compared to a common control group.”

Involving more than 140,000 teachers and nearly 3 million elementary school students, one of the biggest studies of its kind, researchers compared the impact of 15 different interventions with reminder-only messages.

“These messages were behaviorally informed, meaning they were based on previous insights from the behavioral sciences, for example, an intervention asked teachers to make specific plans for how they would use Zearn Math that week.

Specifically, the researchers found that students improved mathematics progress by 1.89% during the four-week intervention period compared to standard email reminders. The most effective intervention, which increased student mathematics progress by approximately 5.06%, encouraged teachers to log in to Zearn Math Weekly for updated, personalized reports on student progress.

“A particularly promising point is that personalized nudges referring to teachers’ own student progress updates were more effective than impersonal students,” Duckworth says.

Researchers point out that despite their promising, their effectiveness is small. “These results suggest that we need more intensive support than the lite touch mail nudge we tested,” says Milkman. “And they emphasize how difficult it is to change human behavior.”

According to Milkman, these findings suggest several additional valuable tools for future research, such as “a more random allocation field experiment to confirm the causal benefits of nudges targeted at teachers and to confirm the causality of studies to investigate the long-term effects of behavior-based interventions.”

Duckworth said, “Utilizing teachers’ intrinsic motivations to help students to see and explain the benefits of looking at personalized data when tweaking teachers could be a clear and potentially cost-effective approach that can complement other interventions, such as providing performance bonuses and other exogenous insemblies.”

The next step for researchers is to dig deeper into why these types of interventions work and how they can be made even more effective over time. Future research is needed to investigate the long-term effects of nudges and to explore why some interventions are more effective than others.

“The better you understand why something works, the more powerful you can use it to create positive change,” says Duckworth. “Ultimately, this series of research will help shape smarter, more effective educational policies.”