Gemini AI Boosts Math Skills in Sierra Leone Trial

Google DeepMind has released results from a randomized controlled trial in Sierra Leone, evaluating the impact of its AI-powered "Guided Learning" feature within Gemini. The eight-week study involved 1,763 junior secondary students across 12 schools in the Port Loko District.

The trial, conducted in partnership with Fab AI and the Sierra Leone Ministry of Education, aimed to determine if AI could serve as a pedagogical aid without replacing teachers. Minister Conrad Sackey of Sierra Leone's Ministry of Basic and Senior Secondary Education noted, "I am therefore delighted that we now have strong evidence that carefully designed AI can help improve learning outcomes in support of our many hard-working teachers." This initiative underscores Google DeepMind's commitment to building a global evidence base for AI's role in education, as detailed in their latest announcement.

AI as a Learning Partner, Not a Shortcut

A key design principle of Gemini's Guided Learning is to foster critical thinking rather than providing direct answers. This approach, rooted in Google DeepMind's LearnLM research, prioritizes conceptual understanding.

Analysis of over 113,000 interactions revealed students used the tool for building understanding in 91.4% of conversations. Gemini responded with scaffolding questions in 76% of its messages, offering direct solutions in only 2% of cases, mirroring a Socratic method.

Teacher-Led Intervention

The trial emphasized a collaborative model where educators remained central to the learning process. Teachers designed lessons, set objectives, and facilitated classroom discussions.

Focus groups indicated that teachers found Gemini valuable for professional development, discovering new teaching methods for topics like fractions. Many reported a shift from traditional lecturing to facilitating student-led learning journeys.

To support wider implementation, Google DeepMind is releasing a teacher training guide developed with Fab AI.

Quantifiable Learning Gains

Students utilizing Guided Learning showed a statistically significant gain of +0.258 standard deviations in math scores compared to a control group. This translates to approximately 1.2 to 1.7 years of learning progress within the eight-week period.

Students in classrooms where teachers integrated Gemini for roughly half the lessons achieved even greater gains, estimated at 1.8 to 2.5 years of progress. Engagement was notably high, with 69% of students meeting or exceeding usage targets, significantly outperforming the typical 5% engagement rate for educational technologies.

Shifting Student Behavior

Beyond score improvements, the study observed a profound shift in student engagement and behavior. Students reported increased enjoyment of math and participated more actively in learning outside of regular instruction.

Crucially, student queries evolved over time, moving from solution-seeking to skill-building. By the final week, skill-building queries constituted 90% of interactions, a substantial increase from 68% in the first week, while direct solution-seeking questions dropped from 25% to 10%.

Future Directions and Equity Considerations

Google DeepMind plans further pre-registered RCTs globally to build a comprehensive evidence base. They are also releasing a playbook on their RCT methodology to facilitate scalable studies.

The trial highlighted an "achievement gap," with students who entered with stronger math skills benefiting the most. Future efforts will focus on developing tools that offer the most significant gains for students most in need, exploring areas like metacognition and relational intelligence.

By combining teacher-led instruction with AI's personalized scaffolding capabilities, the goal is to ensure technology bridges the gap to meaningful learning opportunities for all students. The project also received support from Google.org and the Gates Foundation, with collaborations from EducAid, Laterite, and Oxford MeasurEd.