Hypotheses generation † Generate hypotheses based on CMO configurations Rephrase CMO configurations into hypotheses Formulate hypotheses to be tested during stage three 3. They can be useful in understanding how a program or policy works, in which settings, and for whom. These configurations or explanatory pathways describe how various contexts (C) work with underlying mechanisms (M) to produce particular outcomes (O). In this way, the effectiveness of the programme is understood, with an explanation of why the outcomes developed as they did and how the programme was able to react to underlying mechanisms and in what contexts. MRTs are specific enough to generate particular propositions to test and general enough to apply across different situations. 0000103900 00000 n More to the point, lots of other people can comment as well, meaning you get richer advice. In this paper, we use a realist evaluation to assess a community-based music initiative designed and implemented to support the well-being of disadvan-taged young people in Scotland. (2000). Journal of Evaluation in Clinical Practice 13(6):935-941. http://onlinelibrary.wiley.com/doi/10.1111/j.1365-2753.2006.00790.x/pdf, Dickinson, H. (2006). Approach primarily intended to clarify differences in values among stakeholders by collecting and collectively analysing personal accounts of change. We were particularly interested in instances where use of feedback had stimulated quality improvement in the form of practice changes, but also sought to … Realist trials and the testing of context-mechanism-outcome configurations: a response to Van Belle et al. The content of this field is kept private and will not be shown publicly. Larger scale or more complicated realist evaluations are ideally carried out by interdisciplinary teams as this usually allows for a braoder consideration of likely mechanisms. Marchal, B., Van Belle, S., Van Olmen, J., Hoerée, T. & Kegels, G. 2012. A realist approach assumes that programs are “theories incarnate”. 0000004399 00000 n 0000096087 00000 n (2007). Realist evaluation researchers orient their thinking to context-mechanism-outcome (CMO) pattern configurations. Would the opponents of realist trials argue that in order to do so, evaluators would have to find other, non-random comparisons to avoid the taint of … 0000069133 00000 n 0000001143 00000 n to the RCT, realist evaluation attempts to explain what about an intervention has worked, for whom, how, why and under what conditions (Pawson and Tilley, 1997). An impact evaluation approach which unpacks  an initiative’s theory of change, provides a framework to collect data on immediate, basic changes that lead to longer, more transformative change, and allows for the plausible assessment of the initiative’s contribution to results via ‘boundary partners’. 0000001272 00000 n Approaches (on this site) refer to an integrated package of options (methods or processes). (2010b). CMO stands for Context, Mechanism, Outcome. Realist synthesis is a literature review methodology for understanding how, for whom, and under what circumstances complex interventions function in complex environments. Critical Public Health 10:125-140. For more details, we refer to Marchal et al. In addition: I am based in Australia and am happy to reply to quick questions on methods and methodology. All kinds of theory in realist evaluation – programme theory, theories about particular mechanisms, CMOs, and formal theory– are most useful if developed at a middle level of abstraction. It proposes a possible elaboration of the ‘trio of explanatory components’ (Pawson and Tilley, 1997: 77) used as a strategy in realist evaluation – the Context-Mechanism-Outcome configuration (CMOc). 0 ��&� endstream endobj 48 0 obj <> endobj 49 0 obj <>stream Tackling health inequalities through partnership working: learning from a realist evaluation. Realist evaluation addresses these questions by developing context-mechanism-outcome (CMO) configurations. Realistic Evaluation. A realist evaluation of a safe medication administration education Programme. Strictly speaking, the term ‘generative mechanism’ refers to the underlying social or psychological drivers that ‘cause’ the reasoning of actors. Doing “hands-on” exercises in CMO configuring and theory building. Realist impact evaluation draws on the realism school of philosophy. trailer <<63BD86A3A70445C6A87D0026D22759CA>]/Prev 192928>> startxref 0 %%EOF 64 0 obj <>stream http://aje.sagepub.com/content/22/3/317.full.pdf+html, http://dx.doi.org/10.1177/1356389012440912, http://www.itg.be/internet/ds/tde/doc/Astbury%20%26%20Leeuw%20.pdf, http://onlinelibrary.wiley.com/doi/10.1111/j.1365-2753.2006.00790.x/pdf, http://onlinelibrary.wiley.com/doi/10.1111/j.1365-2524.2006.00650.x/pdf, Mobius Transform of a regular circle packing, by fdecomite on Flickr, Week 49: The 1st international conference on realist approaches to evaluation: my ‘realist’ take-aways, 3. An approach designed to support ongoing learning and adaptation, through iterative, embedded evaluation. A range of approaches that engage stakeholders (especially intended beneficiaries) in conducting the evaluation and/or making decisions about the evaluation​. An approach designed to support ongoing learning and adaptation, which identifies the processes required to achieve desired results, and then observes whether those processes take place, and how. There are hundreds of members and they are very good about providing advice and support. The difference between realist and other kinds of programme theory-based evaluation approaches is that a realist programme theory specifies what mechanisms will generate the outcomes and what features of the context will affect whether or not those mechanisms operate. For example, 'Randomized Controlled Trials' (RCTs) use a combination of the options random sampling, control group and standardised indicators and measures. 0000002159 00000 n The evaluations done so far have not been realist and have not assessed CMO configurations. Finally, the context may provide alternative explanations of the observed outcomes, and these need to taken into account during the analysis. 0000002763 00000 n This resulting CMO configuration is then compared with the initial programme theory, which is modified (or not) in light of the evaluation findings. A realist evaluation of community-based participatory research: partnership synergy, trust building and related ripple effects Justin Jagosh1*, Paula L. Bush2, Jon Salsberg2, Ann C. Macaulay2, Trish Greenhalgh3, Geoff Wong3, Margaret Cargo4, Lawrence W. Green5, Carol P. Herbert6 and Pierre Pluye2 Abstract Background: Community-Based Participatory Research (CBPR) is an approach in … As with any evaluation, the scope of the realist evaluation needs to be set within the boundaries of available time and resources. The second CMO configuration can be summarised as ‘keeping up standards of excellence through organisational culture’. Collecting data to test realist programme theories. In a realist evaluation, the CMO configuration is used in different ways. 0000093179 00000 n 0000003279 00000 n This temporarily diverts attention away from social programs to focus on the pre-existing context of action and related mechanisms that may be operating within it. A research design that focuses on understanding a unit (person, site or project) in its context, which can use a combination of qualitative and quantitative data. 0000069036 00000 n Using a realist approach to evaluation is not necessarily more resource or time-intensive than other theory-based evaluations, but it can be more expensive than a simple pre-post evaluation design. Whether parents can put their new learning into practice will depend on a range of factors – perhaps the time they have available, their own beliefs about parenting, or their mental health. Is realist evaluation keeping its promise? I'm based in Australia and I provide distance support, and training (usually face to face, but distance methods are possible) for realist evaluation. We formulated the initial MRT as follows: “Hospital managers of well-performing hospitals deploy organisational structures that allow decentralisation and self-managed teams and stimulate delegation of decision-making, good flows of information and transparency. If the programme theory/MRT is made explicit together with the main actors, it can lead to a better, shared understanding of the intervention. 0000003532 00000 n Realist evaluations are based on an assumption that projects and programmes work under certain conditions and are influenced by the way that different stakeholders respond to them. Define ethical and quality evaluation standards, Document management processes and agreements, Develop planning documents for the evaluation or M&E system, Develop programme theory / theory of change, activities, outcomes, impacts and context, Combine qualitative and quantitative data, Check the results are consistent with causal contribution, Investigate possible alternative explanations, Sustained and emerging impacts evaluation (SEIE), Technology and evaluation in insecure settings, Evaluation practice in Aboriginal and Torres Straight Islander settings. American Journal of Evaluation 22: 317-324. http://aje.sagepub.com/content/22/3/317.full.pdf+html. Their HRM bundles combine employment security, adequate compensation and training. We initially selected ‘organisational commitment’ and ‘trust’ as proximal outcomes of human resource management, because our initial literature review indicated that these outputs are often found to explain the effect of high commitment management. Hence, we refer to the middle range theory (MRT) rather than the programme theory. Co-produced by BetterEvaluation, DFAT and ODI as part of the Methods Lab project, this publication explains when a realist impact evaluation may be most appropriate or feasible for evaluating a particular programme or policy, and outlines how to design and conduct an impact evaluation based on a realist approach. mechanisms only fire when the context is conducive), realists are modest in their claims, stating that an evaluation cannot produce universally applicable findings. An participatory approach to value-for-money evaluation that identifies a broad range of social outcomes, not only the direct outcomes for the intended beneficiaries of an intervention. Realist evaluations are carried out from different disciplinary perspectives and using a plurality of methods that are fit for purpose. A strengths-based approach to learning and improvement that involves intended evaluation users in identifying ‘outliers’ – those with exceptionally good outcomes - and understanding how they have achieved these. (2005) and Byng et al. évaluation réaliste, santé rurale, santé mondiale, enfants dâge scolaire. Realist evaluation is a form of theory-driven evaluation, but is set apart by its explicit philosophical underpinnings. �k a CMO configuration as a whole is not applied to the process of evaluation. Post-doctoral Researcher, Department of Public Health, Institute of Tropical Medicine, Antwerp (Belgium) and Visiting Researcher, School of Public Health, University of the Western Cape, Cape Town (South Africa). Realist methodology is increasingly used in the evaluation of complex health care interventions, to facilitate a deeper understanding of ‘what works, how, for whom, in what circumstances and to what extent’ [1,2,3,4].The initial step of realist studies aims to identify programme theories underlying the intervention . An impact evaluation approach based on contribution analysis, with the addition of processes for expert review and community review of evidence and conclusions. I heartily endorse Alice's suggestion of joining the Rameses discussion list. It draws on the … A realist evaluation design is well suited to assess how interventions in complex situations work because it allows the evaluator to deconstruct the causal web of conditions underlying such interventions. To construct CMOs, the researchers explored the coded data to identify instances where NCA feedback was reported to have been used by providers.