The folks over at SharpBrains are trying to get a means of evaluating the use of educational games and simulations.
The research overview provided by Tobias, Fletcher, and Dai (this volume) is very helpful in summarizing studies to date on various dimensions of educational games and simulations. The next challenge for the field is to move beyond isolated research in which each group of investigators uses an idiosyncratic set of definitions, conceptual frameworks, and methods. Instead, to make further progress, we as scholars should adopt common research strategies and models—not only to ensure a higher standard of rigor, but also to enable studies that complement each other in what they explore. As this book documents, we now know enough as a research community to undertake collective scholarship that subdivides the overall task of understanding the strengths and limits of games and simulations for teaching and learning. Further, through a continuously evolving research agenda we can identify for funders and other stakeholders an ongoing assessment of which types of studies are most likely to yield valuable insights, given the current state of knowledge.
Research agendas include both conceptual frameworks for classifying research and prescriptive statements about methodological rigor. (For an example of a research agenda outside of gaming and simulation – in online professional development – see Dede, Ketelhut, Whitehouse, Breit, &McCloskey, 2009.) In addition, research agendas rest on tacit assumptions often unstated, but in fact better made explicit, as discussed below. In this chapter, to inform a research agenda for educational games and simulations, I offer thoughts about fundamental assumptions and a conceptual framework that includes prescriptive heuristics about quality. In doing so, my purpose is not to propose what the research agenda should be – that is a complex task best done by a group of people with complementary knowledge and perspectives – but to start a dialogue about what such an agenda might include and how it might best be formulated.http://www.sharpbrains.com/blog/2011/06/11/developing-a-research-agenda-for-educational-games-and-simulations/