Quality in Primary Care Open Access

Keywords

Quality assurance; PEPFAR; Health system strengthening; Botswana

Introduction

Tuberculosis (TB) is the leading cause of death from infectious disease worldwide. In 2014 in Botswana, TB was responsible for 13% of adult mortality [1]. Botswana has one of the highest reported TB rates in the world, with an incidence of 385 cases per 100,000 people in 2014, more than three times the global rate. A primary driver of the epidemic is HIV. The TB/HIV co-infection rate is between 59-75% and TB is responsible for 40% of mortality among people living with HIV [1,2].

The Botswana National Tuberculosis and Leprosy Programme (BNTP) leads national TB control efforts and aims to strengthen data systems through standardized recording, reporting, and monitoring and evaluation (M&E) [3]. This aligns with the World Health Organization’s (WHO) Stop TB strategy which emphasizes strengthening M&E systems to improve TB programmes [4].

In the current data system, patient data is first entered into hard-copy facility TB registers, then manually transcribed into hard-copy district registers. Districts enter data into electronic TB registers, which are sent to the national office by email or portable media [5]. Although quality assurance procedures are built into the system, challenges remain, including incomplete facility-level data recording and reporting; incomplete transfer of data from facility to district level; weak routine M&E activities; minimal use of data to improve programming [6]; delayed reporting and late submission of reports [5]; and under-estimation of successful treatment [7].

Quality TB data are critical for determining population health outcomes and wellbeing and are an essential component of evidence-based decision-making. Data quality is multi-dimensional including factors of completeness, accuracy, and consistency. To strengthen data quality, continuous quality improvement (CQI) activities were conducted at 62 health facilities in 12 districts with high HIV prevalence. We review health facility activities, assess their impact on TB data completeness, describe current data accuracy, and report insights from TB focal staff.

Methods

Data collection

CQI facility visits were conducted by small teams of TB nurse specialists and M&E specialists who reviewed patient records from specified time periods. At all visits, assessment information on TB data quality was immediately shared with facility leadership and TB focal people. During the visits, teams provided one-on-one mentoring on TB data recording and reporting, supported updating of TB registers, and discussed challenges to data quality. Teams worked collaboratively with facility staff to develop plans for improving data quality. Challenges were categorized and addressed using a systems-thinking approach with the following topics: health workforce, health information, service delivery, supply chain, and leadership and governance.

The first CQI facility visit assessed data completeness, the second monitored progress towards completeness, and the third investigated data accuracy. In the first two visits, completeness data were collected for two cohorts on 29 variables, grouped into seven sections: demographics, diagnosis, treatment, sputum, adherence, TB/HIV co-infection, and outcome. Additionally during the second visit, teams interviewed focal people on challenges to data quality, specifically exploring facility characteristics, access to manuals, and facilitators and barriers for TB data quality. The third visit, focusing on accuracy, collected data from a new sample of TB patients at each facility, proportional to patient volume. Accuracy was assessed through comparing secondary source data (facility TB register) with primary source data (TB treatment card, contact tracing form, Mycobacteriology request form) on the following seven variables: contact screening, pre-treatment sputum, month 2/3 sputum, month 6/8 sputum, adherence, treatment stop date, and outcome. Completeness data were not collected.

Data analysis

During facility visits, data completeness was categorized as required and complete, required but incomplete, required but missing, or not required by national guidelines and correctly left blank (e.g. no stop date for patients not yet completed treatment). For analysis, data were dichotomized into complete (required and complete, not required) or incomplete (required and incomplete, required and missing). Completeness was operationalized as 100% complete. Number and percent of complete records were calculated by district, facility type, and data section. Change in total record completeness for the first cohort over time was assessed with a paired McNemar test and between the first and second cohort with a chi-squared test.

Accuracy was operationalized as a match between secondary and primary data sources. Accuracy data was recorded as missing (no primary data source available), accurate (matching primary source), or inaccurate (not matching primary data source). Number and percent of accurate records were calculated for the third cohort at the third visit. All analyses were conducted using STATA version 14.2. Information from focal person interviews was summarized and grouped by theme. Ethical approvals were obtained by the Health Research and Development Committee at the Botswana Ministry of Health (MOH) and the University of Washington Institutional Review Board.

Results and Discussion

Completeness and accuracy data

Three CQI visits were conducted over 12 months in 2015-16. The first two visits were conducted at 62 facilities in 12 high-burden HIV districts. At both, records were reviewed for completeness from a same cohort of 1,418 patients initiated on treatment in the 10 months prior to the first visit. Across the visits, these records had a statistically significant increase in complete data from 5.5% to 16.4% (p<0.001) (Table 1).

Variables	N=1,418	Visit 1		Visit 2
		%	(n)	%	(n)
Location
Southeast	57	0	0	36.8	21
Francistown	139	5.8	8	28.1	39
Gaborone	549	10.9	60	23.1	127
Tutume	55	0	0	10.9	6
Serowe	90	2.2	2	12.2	11
Mahalape	109	0	0	8.3	9
Palapye	133	4.5	6	6.8	9
Good Hope	71	0	0	5.6	4
Northeast	20	0	0	5.0	1
Kweneng East	89	2.3	2	4.5	4
Southern	56	0	0	1.8	1
Kgatleng	50	0	0	0	0
Facility type
Clinic	1,203	6.1	73	17.1	206
Hospital	215	2.3	5	12.2	26
Section
Treatment		88.1	1,249	93.4	1,324
Outcome		76.5	1,085	84.1	1,193
Adherence		52.8	748	73.1	1,037
TB Diagnosis		56.2	797	66.9	948
Demographics		50.4	715	65.0	921
Sputum		42.3	600	52.8	748
TB/HIV		32.1	455	46.5	659

Table 1: TB data completeness from cohort 1 (1,418 records) reviewed at CQI visit 1 and CQI visit 2 in Botswana during 2015-2016.

In the second visits, an additional 837 records were reviewed for patients who initiated treatment prior to the second visits. The second cohort compared to the first had statistically significantly more complete data (15.1% vs. 5.5%; p<0.001).

The third visits were conducted at a subset of previous facilities: 37 facilities in 6 districts. Records were reviewed for accuracy for a new sample of 308 patients who initiated treatment in 2015. For most indicators, the majority of primary source data was unavailable (Figure 1). Among records where a primary source was available, data accuracy was high, ranging from 41.5% to 90.2% per aspect.

Figure 1: TB data accuracy from 308 records reviewed at CQI visit 3 in Botswana in 2016.

Focal people interviews

During the second visits, 54 interviews were conducted with TB focal people in 12 districts on routine processes for TB data documentation and facilitators and barriers to data quality. Only one-third (32.2%) of healthcare providers had been trained in TB case management. Over 80% of respondents indicated that they had access to a TB manual and of those with available manuals/guidelines, 80% said they referred to the documents. Facility QI committees were present at half of the facilities; TB was part of the scope of work for slightly more than half (53.3%) of these committees. Of the 48 TB focal people who discussed frequency of TB record updates, 22.9% updated TB data records monthly, 33.3% weekly, 16.6% two or three times per week, and 27.1% daily. About one-quarter had specific time set aside to regularly update TB records.

TB focal people noted a number of facilitators to high data quality, including systems in place for screening highrisk patients, providers’ positive attitudes toward learning new techniques, routine visits by district TB coordinators to support the work, easy availability of national guidelines, and general support from management for TB coordination activities. TB focal people also noted a number of barriers to implementation, including no systems to transfer patient information among providers, no phones available for tasks such as locating missing patients, providers lacking motivation to update registers, inadequate number of staff trained on TB case management due to lack of initial and refresher trainings and lack of TB data orientation for new staff, no scheduled time to review TB records, lack of job aids to guide providers, inadequate data documentation supervision, no prioritization of TB programme data, and lack of feedback to providers regarding data quality.

The most common recommendations made were to implement more training for TB case management, data quality, and reporting, including both initial training for new staff and refresher training for existing staff; schedule regular times for updating TB records; provide transportation to assist with patient follow-up on missing information or tests; hold regular meetings to share best practices regarding data quality; create QI teams to monitor routine data quality and management; and maintain logs to track communications with patients and share information among staff members.

Conclusion

Overall, three core CQI activities made a substantive difference in increasing staff knowledge on TB variables and national guidelines and ownership for TB data. Resulting in a statistically significant increase in TB data completeness. First, providing direct training and feedback from the CQI team to the TB focal person, facility management team, and DHMTs. Many staff had not previously been trained on TB data documentation and had genuine confusion over variable definitions. In addition, these activities built the programmatic knowledge of facility management teams. Second, engaging with clinicians who initiate care to record all necessary information at initiation. A recommendation was provided to the BNTP to require clinicians to record their names on data documents to further increase a sense of ownership and accountability. Third, including national-level staff in the activities engaged them in providing feedback on data quality on a routine basis, rather than waiting for annual reports and helped build multi-level support for the initiative.

Competing Interests

The authors declare that they have no competing interests.

Funding

This work was funded by the U.S. President’s Emergency Plan for AIDS Relief (PEPFAR), through the Health Resources Services Administration (HRSA) of the U.S. Department of Health and Human Services (DHHS), under Cooperative Agreement U91HA06801. The funding body had no role in study design, data collection, data analysis, data interpretation, and manuscript writing.

Authors Contributions

SM, BK, HM, RM, BS and JL conceptualized the programme. HM and RM conducted CQI visits and collected data. JB, SM, and RM conducted initial data analysis and developed an in-country report. NJK condensed and revised the report into a manuscript draft. All authors read and approved the final manuscript.

Acknowledgements

The authors would like to acknowledge the CQI teams, TB focal people, facility management, DHMTs, and Botswana Ministry of Health.

References

1. Botswana Ministry of Health. Botswana National Tuberculosis and Leprosy Programme (BNTP). Combined Annual Report, 2013–14 ; Gaborone, Botswana, 2014.
2. World Health Organization: African Health Observatory. Botswana Analytical Summary – Tuberculosis, 2017.
3. Botswana Ministry of Health. National Tuberculosis Programme Manual, Seventh Edition. Gaborone, Botswana, 2011.
4. WHO. Electronic recording and reporting for tuberculosis care and control. Geneva, Switzerland, 2012.
5. Botswana Ministry of Health. TB data quality challenges in Botswana: A rapid review. Gaborone, Botswana, 2015.
6. Botswana Ministry of Health. Rapid data quality assessment report. Gaborone, Botswana, 2013.
7. Botswana Ministry of Health. Assessment of community TB care implementation. Gaborone, Botswana, 2010.