Measuring the effect of ARS on academic performance: A global meta-analysis

Abstract

An increasing number of studies have addressed the impact of Audience Response Systems (ARS) on academic performance at all stages of education, although the evidence does not seem conclusive. With the aim of shedding light on the extent and diversity of the research outcomes, we conduct a meta-analysis of studies worldwide on this topic to assess whether the exam scores of students included in ARS experiments achieve better results than others taught using more conventional teaching tools. From an initial sample of 254 studies, data from 51 papers published between 2008 and 2012 (involving 14,963 participants) that set academic quality criteria, were extracted and analyzed following technical protocols for meta-analyses. Their high degree of heterogeneity shows that the effect of ARS on exam scores seems to be moderated by specific features. So, through a randomeffects model, our results provide a positive, although moderated pooled effect of ARS on examination scores that is much greater in experiments performed in non-university contexts (Hedges' g ¼ 0.48; S.E. ¼ .2665) than at the university level (Hedge's g ¼ 0.22, S.E. ¼ .0434). Specifically, the categories of university disciplines in which ARS interventions are implemented seem to influence their usefulness for achieving better academic marks, being more effective when either Pure Soft Sciences or Applied Hard Sciences are considered. These findings might provide guidance for governments, researchers and educators into the effectiveness of learning based on the new interactive technologies.