Publication date

Evaluating the implementation and effects of a multilevel quality collaborative in hospital care: a research strategy.

Dückers, M., Wagner, C., Groenewegen, P. Evaluating the implementation and effects of a multilevel quality collaborative in hospital care: a research strategy. In: J. Ovretveit; P. Sousa. Quality and safety improvement research: methods and research practice from the International Quality Improvement Research Network. Lissabon: Escola Nacional de Saúde Pública, 2008. p. 105-126.
Download the PDF
Phase of the research when this was written: in the middle of the process. Before the data collection of the final part of the study, but after reviewing the literature and collection and anlysis first data. Main research question: How does the participation by hospitals in a multilivel quality collaborative (MQC) result in enhanced quality of care and the development of an organizational infrastructure for improvement, stimulating the adaption and sustainable spread of best practices?

General design and sample
Monitoring of process and effects of implementation and spread of breakthrough projects by multidisciplinary hospital teams joining six quality improvement collaboratieves (QIC's). Data are gathered among actors within hospital (e.g. executives, programme coordinators, managers, doctors and project leaders of improvement teams) as well as external change agents responsible for supporting hospital actors. Data from a national survey are used to compare the state and effects of the improvement infrastructure in the intervention group (the MQC group) and control group (non-MQC). Main findings (expected):

The study wil clarify how leadership and support by the parties involved relate to the implementation, spread and sustainability of innovations within hospitals. Furthermore, it will illustrate how the intended improvement infractucture connects national health care targets to care delivery by individual professionals.

Methodological challenges
1) Analysis require a certain amount of statistical power. The limited number of hospitals and project teams combined with self report bias (too positive) and non response reduces the sample size and availability and reliability of necessary data.
2) Developing an instrument to monitor the degree to which relevant conditions are met during the implementation of breakthrough projects in a QIC.
3) Conceptualization of the emerging organizational improvement infrastructure is needed to formulate an evaluation strategy in which intervention and control group are compared.
4) controlling for hosptial characteristics while comparing, all the more because the 24 MQC hospitals are not randomly selected.
5) Aggregating data at unit (micro level to institutional (meso) level.

Practical challenges
Many actors and conditions play a role in implementing the MQC. The setting of the evaluation study is highly politicized. Evaluation inevitably means influencing the implementation, a process based on communicating success stories, as it also generates less positive insights.

Main lessons for other researchers
Besides general advice, a number of tips and recommentations is listed for conducting an independent evaluation study in a political environment.