Blog
Measuring outpatient safety at scale

Primum non nocere — first, do no harm. This most basic tenet of medical care is routinely violated in clinics and hospitals around the world today. But the absence of routine data means that when it comes to improving patient safety, we are often in the dark. Available studies frequently rely on self-reported data from healthcare workers, focus on a single domain such as hand hygiene and are based on small samples: one review found that only 10 of 41 studies on hand hygiene interventions were collected in more than one hospital!

Beyond the problems of generalizing from self-reported data in small samples, the focus on single domains can impede much needed epidemiological modelling given multiple pathways of nosocomial (i.e., healthcare-associated) infections and a range of violations to safety practices that can occur in these settings. To address these problems, in 2015 we conducted the largest patient safety survey across low- and middle-income countries, covering 1,035 facilities in three Kenyan counties and a total of 1,680 healthcare workers and 14,328 patients.

These data — collected as part of the larger Kenya Patient Safety Impact Evaluation or KePSIE trial — built on substantial investments by WHO in developing and validating the tools required to study violations across multiple domains such as safe injection practices and hand hygiene. As a start, in 2013, we visited several facilities to understand the ‘average’ patient experience. Apart from glaring violations (see picture), we noticed that consultations in Kenya typically involve visits to different parts of a facility. For instance, a patient may start off in the consultation room, then be advised to get his or her blood tested, then return to the consultation room, and finally, be sent to the injection room for treatment.

Clearly, restricting ourselves to a single domain would not present the full picture — patients could face violations at any point, and each of these safety violations could contribute to nosocomial infections. The new tool that we implemented specifically addressed this point.

What did we do?

We identified three key procedures for observation: physical examination, injection and blood sampling; trained assessors to record the interaction between healthcare workers and patients in the consultation room, the laboratory, and the injection room. We measured compliance with IPC practices across five domains (i) hand hygiene; (ii) use of protective gloves; (iii) injection and blood sampling safety; (iv) disinfection of reusable equipment; and (v) waste segregation.

These domains are recognized as critical for outpatient safety by the WHO, the United States Center for Disease Control and Prevention, and the Kenyan Ministry of Health. Finally, we measured healthcare workers’ knowledge of these practices and the availability of the equipment and supplies needed to implement them. For each procedure we recorded indications, or situations in which a safety action should be undertaken to prevent the risk of pathogen transmission (e.g., before touching a patient) and then whether the correct action to avoid such risk was taken by the healthcare worker (e.g., washing hands with water and soap or an alcohol-based hand rub). 

What did we find?

There were four key findings.

• First, overall compliance with the 20 IPC practices analysed was low (31.8%).  Consequently, outpatients faced on average 5.1 safety violations during their visit.
• Second, there was significant variation in compliance across domains, ranging from 2.3% in hand hygiene to 87.1% in injections and blood sampling.
• Third, like other studies summarized here, we found significant know-do gaps across most domains: the proportion of healthcare workers that had the requisite knowledge or access to the needed supplies was always higher than compliance — sometimes notably so. In the most dramatic example (Figure 1), compliance with hand hygiene practices increased from 2.4% to only 4.2% when healthcare workers had the required knowledge and supplies.
• In general, the association between compliance and most characteristics of healthcare workers and facilities were weak. Whether looking at the facility level (specialization, ownership type), healthcare worker level (age, education, gender), or IPC emphasis (availability of supplies, availability of Kenyan IPC practices manual, training on IPC in the last year), nothing was strongly correlated with what providers actually did in the clinic.

What does this imply?

Most importantly, real progress has been made in some domains. For instance, in our sample compliance was 100% for the actions ‘using new needles and syringes for injections and blood sampling’. WHO has tracked this progress, but to see how far these efforts have improved the lives of patients in some of the most remote and rural settings of a country like Kenya is a sign that the safe injection campaign has had a real and measurable impact on the behaviour of healthcare workers. At the same time, the weak association between compliance and healthcare worker knowledge as well as facility characteristics is consistent with the widely-discussed concept that patient safety is driven more by behavioural norms and biases than by technical knowledge, training, or the availability of supplies. How to engender similar behaviour change in other domains, particularly hand hygiene, remains the single biggest challenge for patient safety today.

To be sure, our tool has limitations: we cannot currently link these compliance indicators to health outcomes and it could be that healthcare workers change their behaviour when they are being observed (the Hawthorne effect). Our observations are also centred on the clinical interaction, leaving out equally important issues such as waste management (11.1% of facilities had a standard operating procedure for waste management, and 26.1% had an on-site incinerator or contract with a company for incineration). But as the tool receives more attention and is applied to diverse settings, we will be able to move towards routine data on multiple domains. This can help us better understand pandemics and, ultimately, reduce the danger that patients may face in their search for a cure.

KePSIE is funded by the following World Bank programs and units: the Strategic Impact Evaluation Fund of the Health (SIEF); the Competitiveness Policy Evaluation Lab (ComPEL); the Impact Evaluation to Development Impact (i2i) fund; the Korea World Bank Group Partnership Facility (KWPF); the Development Economics Research Group Department; and the Kenya Health in Africa Initiative. The KePSIE team includes Guadalupe Bedoya, Jorge Coarasa, Jishnu Das, Amy Dolinger, Ana Goicoechea, Njeri Mwaura, Khama Rogo, and Frank Wafula, supported by Benjamin Daniels, Rebecca de Guttry, and Seungmin Lee from the World Bank Group. The team works together with the Kenya Ministry of Health, the regulatory boards, and councils.

The views expressed in this piece are those of the author(s), and do not necessarily reflect the views of the Institute or the United Nations University, nor the programme/project donors.

Read more on the papers published in the WHO Bulletin theme issue 'Measuring quality of care' at the UNU-WIDER's lauch event website.