All FSI Projects

Standardized Patients

screen shot 2014 07 31 at 4 32 04 pm
screen shot 2014 07 21 at 2 30 30 pm

In the past decade, China has rolled out a large number of reforms and invested heavily in providing care for the more than 600 million Chinese that live in rural areas and migrant communities.


Problem

The new investments have sought to improve access to health care at all three tiers of the rural health care system: county hospitals, township health centers and village clinics. Government investment in healthcare reached $155 per capita in 2011. Most of this expenditure went to expanding healthcare coverage. By the end of 2010, around 96% of rural residents had insurance coverage. While China has undoubtedly made great strides in improving access to health care in rural areas since the start of reforms, the quality of this care is very much in question. For example, there is little evidence that the reforms have improved actual health outcomes or lowered the risk of catastrophic health expenditures for patients. A key reason for this could be low quality healthcare in rural areas. Although government attention has recently shifted toward improving care delivered by providers in rural areas, there is little objective evidence on provider quality to guide policy interventions. 


Goals

Our goal in conducting this study is to provide policymakers with objective evidence on the quality of healthcare delivered in rural areas of Northwest China. We will also compare the quality of care delivered at different tiers of the health system and assess what factors are correlated with better care. 


Approach

In order to measure the quality of health care in rural China, we adopted an innovative survey methodology known as Standardized Patients (SP).

Standardized Patients (SP) have been used in Western medical schools since the 1960s and are considered the “gold standard” of assessing clinical practice. The Standardized Patients are confederates who pose as regular patients to an unsuspecting doctor. They undergo training to present consistent (standardized) symptoms of chosen illnesses, and then visit a local doctor in their guise as patients. The SPs interact with doctors normally and then – after the conclusion of the visit – these interactions are compared to a series of pre-determined standards of clinical care.

Why use the SP approach instead of directly interviewing the doctors? The reason is that doctors, if aware that they are being evaluated, are likely behave differently than they would in a normal interaction with a patient (a so-called “Hawthorne effect”). Therefore, direct interviews and observation do not reflect clinical practice under normal circumstances. In addition to avoiding such Hawthorne effects, the SP approach allows for comparisons between doctors who typically face different types of patients and diseases.

While long-used in developed country medical schools, researchers have begun to use the SP approach to measure health care quality in developing countries. Our project is the first in China using this innovative approach.

To implement the SP approach in China, we trained individuals recruited from the local community to represent consistent cases of two diseases (childhood dysentery and unstable angina) to village and township clinicians. SPs memorized “case scripts” (developed with the assistance of consulting doctors in China) corresponding to these diseases complete with detailed medical histories, symptoms and individual background stories.

Following training, our patients and staff visited 12 township clinics and 36 village clinics in southern Shaanxi province. Between the two diseases (which were presented to each doctor), the sample included a total of 96 clinical interactions. Interactions between the SPs and doctors were recorded using concealed recording devices. The SPs also purchased any drugs that were prescribed by the doctors and paid any service fees.  Recordings were then transcribed and scored against pre-determined checklists of recommended questions that clinicians should ideally ask of patients presenting with the symptoms of the SPs as well as exams that should be performed (given the resources available in village and township clinics).  Diagnoses and treatments given by doctors were also evaluated based on pre-determined standards.

A doctor administers to a parent and child.

Results

We used the interaction transcripts to evaluate rural doctors on three primary dimensions of quality: process (questions asked and tests performed), diagnoses, and treatment.

Process: We found process quality to be low among both village and township doctors. On average village and township doctors asked or performed only 18% of the questions and exams for the childhood dysentery case. For the unstable angina case, village doctors asked or performed 19% of the recommended items and township doctors asked or performed 27%.

Diagnosis: Village and township doctors gave diagnoses approximately 75% of the time in the unstable angina case and 35% of the time in the dysentery case.  Of these, correct diagnoses were more frequent than expected given the low performance of doctors on process quality. When a diagnosis was given, village doctors correctly diagnosed dysentery 64% of the time and township doctors correctly diagnosed 75%. The numbers were higher for unstable angina: this disease was correctly diagnosed 65% of the time in village clinics and 75% in township health centers.

Treatment: Overall we found that correct treatment was rare, particularly in village clinics. Across all SP interactions with clinicians, only 53% of treatments were judged to be correct or partially correct. In other words, a visit to a rural health care provider only gives patients a 50-50 chance of appropriate treatment—that’s about the same as a coin toss!

Even more alarming, REAP’s consulting physicians determined that 64% of the drugs dispensed by rural clinicians as part of their treatment plan were either unnecessary or harmful.

Overall, the results suggest that process quality of physician-patient interactions was low and, more often than not, the course of treatment was incorrect. The bottom line is: don’t get sick in rural China!

DSC_0782

Next Steps

China is home to the world’s second largest tuberculosis (TB) epidemic, with over 1.3 million new cases each year. Despite good intentions from both the Chinese government and a number of international health organizations, TB control within China is still inadequate. The estimated proportion of new cases of sputum smear-positive tuberculosis that were diagnosed and treated—a key indicator of efforts to control tuberculosis—has stagnated at around 30%, far below the 70% target set by World Heath Organization. Moreover, recent estimates put the incidence of multi-drug resistant TB—a serious public health problem that ignores national borders—at dangerous highs. More must be done to target this potentially devastating public health problem.

Building on experience from our first SP study, REAP is now working hand-in-hand with the Chinese Ministry of Health to tackle TB and raise the quality of the greater rural healthcare system in the process. Our latest project will be carried out in three phases:

Phase 1: Using Standardized Patients and Vignettes, we will assess both the knowledge and practice of rural doctors in treating TB.

Phase 2: Based on our results from Phase 1, which will improve our understanding of obstacles to high quality TB treatment in rural China, we will work with the Ministry of Health to design a series of two to three potentially scaleable interventions that address these obstacles. In a randomized controlled trial (RCT), we will test these interventions to identify which is most effective.  Although the final set of interventions has not yet been decided, initial ideas include physician training, clinical checklists, and payment reform.

Phase 3: Once we have determined which interventions are successful, we will work with the Ministry of Health to scale up these interventions to cover the full rural population. Beginning in Shaanxi province (the pilot site and proposed site of the RCT), we will then expand to other provinces with high rates of TB infection, including Gansu, Ningxia, Qinghai, Sichuan, Guizhou, and Shandong.  As part of the scaling up process, we will hold a series of training meetings with national- and provincial-level Ministry of Health officials across the country to share our proven training protocols and materials.  We will also work directly with the Ministry of Health to organize and run official training programs to educate rural doctors.