Quality in Primary Care Open Access

Keywords

Sweden; Primary health care providers; Equity; Health services research

How this fits in with quality in primary care?

What do we know?

On the central attributes of primary care and different patient groups, there are many studies and seeking after new knowledge continues. Previous research has shown that necessary efforts are not easy to implement within primary care regarding themes as patients’ gender, ethnicity, age, mental health and physical functioning. Previous research also has shown that communication among colleagues, collaboration within the team, routines, personnel and time, and the financial situation are important for equitable care, but communication is not easy to establish on a level which primary care needs.

What does this paper add?

This paper adds knowledge about how Primary Health Care Units (PHCU) developed and tested activities to promote a more equitable care. Combination of learning seminars and team works between seminar periods, and involvement of researchers in the project from the beginning to the end has led to development of some working methods to handle some difficulties and weaknesses mentioned above. Using a perspective called “ongoing evaluation” in conjunction with a program-theoretical evaluation perspective during development of working methods is also a contribution from our study.

Introduction

The goal of the Swedish health care system is to provide good care on equal terms to the whole population [1]. Health care in Sweden is a public responsibility, financed primarily through taxes that are levied by county councils and municipalities. During the past decade, several reports from different agencies, as well as from the government and the Swedish Association of Local Authorities and Regions (SALAR), have revealed that health care is unequal, lacking in accessibility and not offered on equal terms to the whole population [2-4]. Other reports [5,6] show that differences in health between different groups in the population have increased with respect to factors such as education level, gender and country of birth.

Providing health care on equal terms has become a challenge for the health care system, and especially for primary health care. The central attributes of primary care are first-contact (accessibility), continuity (person-focused preventive and curative care over time), patient-oriented comprehensiveness and coordination (including navigation towards secondary and tertiary care) [7]. This means that the primary health care team deals with continuous care for all unselected health problems in all patient groups, irrespective of social class, gender, ethnicity, etc. Thus, primary health care makes an important contribution to equitable care [8] and developing methods for the analysis of equity in health care delivery are crucial. In Sweden, residents are encouraged to visit their primary health care centers first. According to the Swedish health care act, paragraph 5, primary health care is part of the outpatient care and is responsible for meeting the population’s need for basic medical treatment, care, prevention and rehabilitation that does not require hospital and technical resources, or other special skills, without restrictions due to illness, age or patient group [1].

Both the government and regional and local agencies have initiated projects aiming to increase equality in health care. As part of one such project, in June 2011 the government signed an agreement with The Swedish Association of Local Authorities and Regions (SALAR), for a three year project to develop and implement “Health care on equal terms.” The project, which involved seven Primary Health Care Units (PHCU) from five county councils in different parts in Sweden, was completed in early 2014. The aim of the project was to develop methods and activities that could promote more equal health care provision in socio-economically disadvantaged areas. The project used the Breakthrough model from IHI (Institute for Health Care Improvement), which is a structured improvement methodology designed to help health-care organizations reduce the “know-do gap”, where evidence from research is rarely implemented despite having the potential to improve outcomes while reducing costs at the same time [9].

During the project implementation, a total of eight learning seminars were arranged, and between seminar sessions, eight periods of time were scheduled for team working at the PHCUs. The participating PHCUs developed and tested a total of 48 activities for more equitable care with a variety of solutions to different identified problems [10,11].

Equity research in health care often focuses on either financing or delivery of health care while, in this study the focus was to assess and compare the health care providers’ experiences and perceptions of achieving more equitable health care before and after the project “Health care on equal terms”. This study was a part of a larger evaluation report financed by SALAR [10].

Material and Methods

The study design was explorative and cross-sectional with "unbalanced panels" at two time-points. With "unbalanced panels" we mean that some participants appeared at both times, but not all participants. This type of cross-sectional design differs from both true panels/pooled cross-sectional times and repeated cross-sectional designs [12].

A questionnaire was constructed and distributed to the health care providers in the nine PHCUs participating at the beginning of the project, and the seven PHCUs participating at the end. The questions were based on previous international and national research questionnaires on health equality and on questions used in evaluations of the Breakthrough methodology [13-16]. The survey consisted of four parts. Part 1 was about professional information (occupation, age, education, number of years in the profession and in the current activity). Part 2 dealt with workplace environment and working conditions. Part 3 dealt with the respondents’ experiences and perceptions of factors related to health care on equal terms. Part 4 of the survey dealt with the respondents’ perceptions about the factors related to changes and improvements in achieving equitable health care. The survey consisted of both closed response questions with multiple choice options and open-ended questions. Four of the questions were based of a Likert scale in the range 1-5 where 1 was Very strong and 5 was No impact, answer No idea was 0. A pilot web survey was sent to individuals working at the participating PHCUs. The online survey was further refined after the responses from this pilot survey to make it more understandable and easier to answer.

The web survey (baseline) was sent to all staff at the participating PHCUs at the beginning (2012) and the end (2013) of the project by SALAR Statistics Center. The center was also responsible for sending out two reminders. The response rate for 2012 was 60.25% and 62.80% in 2013.

Due to small sample size as well as the used study design there panels in 2012 and 2013 were not identical we did not analyze the data with inferential statistics. Instead, we allow the changes between these two occasions to speak through percentages in the tables. Descriptive statistics were employed by using SPSS 11.0 software. The percentages are based on “valid percentages” which exclude the missing cases. We have combined response options 1 and 2, as well as 4 and 5 (see online questionnaire) in order to present a clearer picture of the results.

Open-ended responses were analyzed with content analysis. Content analysis involves identifying, coding and categorizing the primary patterns that emerge from the collected data [17,18]. The first phase was to read through the printouts several times to get a sense of the whole. The second phase was to choose sentences or phrases that contained information of relevance to the study’s aims. The third phase was a “systematic analysis of the chosen sentences or phrases.” The researchers coded claims, cut them out, and sorted them according to the different themes chosen. Eventually, the researchers were able to merge several codes into subcategories. Main categories were developed through comparative analysis of these subcategories [19,20].

Results

Participants

The questionnaire survey was completed voluntarily and anonymously by health care providers in the participating PHCUs, totaling 141 respondents in 2012 (baseline) and 147 on the second occasion in 2013. In both 2012 and 2013, those who participated were mostly nurses, physicians, assistant nurses and medical secretaries. Table 1 show that most respondents were 56-65 years of age and had higher education. Most were experienced professionals who had worked in their occupation for more than 10 years. Most respondents were working at primary health centers and/or child health centers.

Age	N (%) 2013	N (%) 2012
26-35	7 (6.2)	9 (6.7)
36–45	29 (25.7)	33 (24.4)
46–55	36 (31.9)	35 (25.9)
56–65	41 (36.3)	58 (43.0)
Education
Primary school	2 (1.7)	1 (0.7)
University degree	91 (77.8)	102 (73.9)
Other higher education	11 (9.4)	13 (9.5)
High school degree	13 (11.1)	22 (15.9)
Years in the profession
0-9 Years	17 (14.8)	15 (11.3)
10-19 Years	30 (26.1)	47 (35.3)
20-29 Years	23 (20.0)	20 (15.0)
30-39 Years	36 (31.3)	36 (27.1)
40-47 Years	9 (7.8)	15 (11.3)
Years at the current primary health care unit
0-9 Years	65 (55.6)	77 (56.2)
10-19 Years	28 (23.9)	35 (25.5)
20-29 Years	20 (17.1)	17 (12.4)
30-39 Years	4 (3.4)	8 (5.8)
Workplace
Primary care unit	95 (79.8)	109 (75.2)
Primary care for children	29 (24.4)	35 (24.1)
Primary care for mothers	6 (5.0)	8 (5.5)
Hospital	4 (3.4)	4 (2.8)
Primary care for youth	3 (2.5)	3 (2.1)
Other	5 (4.2)	7 (4.8)

Table 1: Demographic information about the participants.

The response rate to the open-ended questions ranged between 10 and 124 respondents. Analysis of the responses to the open-ended questions does not show any major changes in respondents’ perceptions between 2012 and 2013.

What is care on equal terms?

In answering an open-ended question about what “care on equal terms” means to them, the respondents answered that “everyone is given the care they need” or "everyone, regardless of origin, social and economic status, and sexual orientation, and regardless of other illnesses should have the right to equitable care.” Some answered “no opinion” or “do not know.”

Some respondents expressed that care on equal terms means “everyone should be entitled to the same care,” while others thought “every person who seeks care should be individually assessed and have their needs met, which does not always mean that all patients receive the same care.” Another summed up their viewpoint as “equal treatment, according to individual needs.” Another aspect highlighted by some respondents is that “everyone should be able to get the help they need, even if they have difficulty expressing themselves or understanding the culture or regulations.” This aspect is emphasized by another respondent who thought that “patients/clients who are easy to talk to, or have higher social status, should not be given priority over those who do not have these characteristics.”

One of the respondents highlighted the importance of adapting health care to the patients’/users’ needs: “adapting care to each individual’s circumstances, such as people with ADHD, physical disabilities, who have been traumatized, etc. and not the other way around.” “Trying to find those not seeking care” is also mentioned as an important factor in providing care on equal terms.

Factors that contribute to access to health care

As shown in Table 2, the proportion of respondents who considered that sexual orientation, age and area of residences are important factors that influence access to health care had increased in 2013 compared to 2012. On the other hand, the proportion who considered that physical functioning is significant for access to care had decreased in 2013. The proportion who considered gender to be significant for access to health care remained almost unchanged.

Factors that are important for access to health care	Year	Very strong impact 1	2	3	4	No impact 5	No idea 0	N
Patients’ education level	20122013	21.115.5	16.724.1	21.123.3	13.214.7	24.619.8	3.52.6	114116
Patients’ ethnicity or country of origin	20122013	17.710.3	19.517.2	26.521.6	14.223.3	18.624.1	3.53.4	113116
Patients’ gender	20122013	9.67.8	3.510.3	21.920.7	15.817.2	43.041.4	6.12.6	114116
Patients’ age	20122013	9.79.5	15.016.4	23.031.0	19.516.4	28.325.0	4.41.7	113116
Patients’ sexual orientation	20122013	7.18.6	4.46.0	12.417.2	12.47.8	53.150.9	10.69.5	113116
Patients’ area of residence	20122013	8.812.1	7.19.5	19.519.8	15.912.1	41.640.5	7.16.0	113116
Patients’ mental health	20122013	16.811.3	19.527.0	24.825.2	18.614.8	15.020.9	5.30.9	113115
Patients’ physical health and level of functioning	20122013	8.16.0	15.317.2	25.222.4	18.021.6	27.031.0	6.31.7	111116

Table 2: Factors that contribute to access to health care (%).

Respondents had different opinions about whether the influence of patients’ levels of education, ethnicity and mental health were related in access to health care. In 2012, the percentage of participants who thought that these factors have no or very little significance was equal in size to the proportion who thought these factors have a high impact on access to health care. In 2013, the percentage of those who considered patients’ ethnicity to have no or very little impact on access to care had increased. Respondents who had no idea about the impact of patients’ gender, age and mental and physical health on access to health care had decreased.

In answer to the question about patients/clients who may find it difficult to be heard within the PHCU, most respondents perceived foreign-born people (immigrants, asylum seekers, and a specific foreign-born group mentioned by some respondents, Somalis), the elderly and mentally ill patients/clients to be among “those who have difficulty making their voices heard” or “those who are not strong enough to be heard in the ‘right way’ and are lacking in communication skills.” Other patient groups that were mentioned were those with multiple or chronic sicknesses, those with mental or physical disability, older people without families and those with addiction problems.

One participant described patients who may find it difficult to be heard within the PHCU as “those who do not speak Swedish, those who do not understand the system and organization.” Another added that “immigrants, especially women who are illiterate” or “foreign-born patients” “may have to wait a little longer when an interpreter must be booked.” Some respondents felt that foreign-born patients had no difficulty being heard: “Patients from other countries find it difficult to understand self-care; they are accustomed to always going to the doctor for the slightest little ailment.”

In order to be heard, the patient/client should adapt to the demands of organization of care. This was explained clearly by one respondent:

"Calling the PHCU to make an appointment is the main point of access to care...Those who cannot speak Swedish or English...the elderly or those who hear or see poorly...may have difficulty making a phone call. They may have difficulty understanding telephone instructions and which button to press...The mentally ill and people with a low income (cannot afford phone expenses) also have difficulty contacting the PHCU by phone...Illiterate people find it hard to call...All of these people come to the clinic and speak with a nurse or doctor".

Respondents who answered the supplementary question about what other factors affect patients’/clients’ access to health care on equal terms mentioned some organizational factors within health care. They thought that “access to health care where you consistently can meet the same physician,” “access to an interpreter when necessary” and “information about what the health care services can offer” may have implications for access to care. Some thought that there are structural/organizational factors in society that affect access to care on equal terms. They mentioned “the amount of money in your wallet,” “unemployment” and “access to a car.” One of the respondents explains “especially the wallet...if you have the economic means you can seek private-sector care and see a doctor or nurse faster.”

A number of respondents believed that language can affect access to health care on equal terms. One explained it as follows: “Language. If you can’t speak Swedish, it’s an obstacle to making contact. It may take longer when the interpreter needs to be booked.” Another referred to the patients’/clients’ levels of education and their “health literacy.” The respondent explained:

"Education is an important factor; especially important is knowledge about how the body works. We expect the patient to describe what they want help with, how they feel and also be able to understand information, explanations, etc., and to participate in a discussion with the professional...It’s not easy if you understand your body in completely different ways".

Factors and criteria affecting PHCUs’ ability to offer health care on equal terms

As shown in Table 3, in 2012, patients’ confidence in health care providers was perceived to be the most important factor for offering health care on equal terms by more respondents than any other factor, while in 2013 having enough time for patients and availability of staff were considered the most important factors. The proportion of those who had no idea about the influence of the factors asked about, for example concerning issues of leadership, lack of patient follow-up and patient confidence in the health care providers, had increased in 2013.

Factors affecting PHCUs’ ability to offer care on equal terms	Year	Very strong impact 1	2	3	4	No impact 5	No idea 0	N
Access to care (e.g. telephone times, opening hours, etc.)	20122013	43.437.9	15.011.2	12.419.8	12.414.7	13.314.7	3.51.7	113116
Respectful treatment by staff	20122013	42.037.9	14.310.3	14.316.4	17.912.9	8.920.7	2.71.7	112116
Information to patients	20122013	42.936.2	18.817.2	16.119.0	9.812.1	8.912.1	3.63.4	112116
Access to health care providers or PHCU staff	20122013	38.146.6	19.516.4	15.07.8	13.312.9	11.515.5	2.70.9	113116
Time for patients (e.g. consultation)	20122013	41.647.0	21.216.5	15.011.3	10.612.2	7.112.2	4.40.9	113115
Staff competence	20122013	38.941.2	20.411.4	10.614.9	12.412.3	14.218.4	3.51.8	113114
Health care providers’ attitudes	20122013	46.042.6	16.813.0	8.810.4	14.213.0	10.620.0	3.50.9	113115
Lack of patient follow-up	20122013	33.332.7	17.117.7	21.618.6	15.314.2	8.110.6	4.56.2	111113
Leadership at PHCUs	20122013	35.435.1	17.715.8	15.918.4	14.213.2	15.0 13.2	1.84.4	113114
Patients’ trust in health care providers	20122013	48.244.7	16.110.5	10.712.3	13.413.2	9.816.7	1.82.6	112114

Table 3: Factors affecting PHCUs’ ability to offer health care on equal terms.

In both 2012 and 2013, the respondents considered the most important organizational/ structural criteria for providing equal health care to be continuity of care, allocation of financial resources to match the needs of the catchment area and patient groups, trained health care providers, availability of care, and adapting care to users’ needs. According to the results, the majority of respondents considered having modern technology and medical equipment and free choice of primary health care unit/physicians to be not at all important or of lesser importance for providing equal health care (Table 4).

Organizational criteria for providing care on equal terms	Year	Very strong impact 1	2	3	4	No impact 5	N
Free choice of primary health care unit	20122013	9.10.0	9.10.0	9.140.0	27.30.0	45.560.0	115
Free choice of physician	20122013	16.70.0	0.011.1	16.722.2	33.311.1	33.355.6	69
Accessibility	20122013	21.721.1	21.724.4	22.818.9	18.518.9	15.216.7	9290
Modern medical equipment	20122013	0.00.0	0.00.0	7.710.0	23.110.0	69.280.0	1310
Trained health care providers	20122013	10.618.6	26.616.7	29.826.5	23.420.6	9.617.6	94102
Not-for-profit health care	20122013	21.220.7	15.227.6	18.26.9	18.213.8	27.331.0	3329
Allocation of resources according to the needs of the medical district or patient group	20122013	26.023.5	14.022.5	27.021.6	16.016.7	17.015.7	100102
Political management of health care	20122013	20.021.4	16.03.6	16.014.3	20.021.4	28.039.3	2528
Continuity in care	2012 2013	27.0 23.6	24.0 25.5	14.0 20.0	24.0 20.0	11.0 10.9	100 110
Adaptation of care to patients’ needs	2012 2013	19.0 17.6	24.1 17.6	11.4 18.7	15.2 27.5	30.4 18.7	79 91

Table 4: Main organizational/structural criteria for providing equitable health care (%).

The results show that in both 2012 and 2013 most respondents agreed that the three most important clinic-level factors for providing equitable health care were respectful attitude, awareness of one’s own biases/behaviors and trust between patients and health care workers. The fourth factor which was considered important was the patients’ participation in consultations with the care providers (Table 5).

Three key factors for providing care on equal terms	Year	Very strong or strong impact 1-2	Medium impact 3	No or little impact 4-5	N
Respectful treatment	20122013	57.151.1	29.733.0	13.216.0	9194
Awareness of one’s own biases/behavior	20122013	39.138.6	27.528.1	33.333.3	6957
Patients’ participation in decision making in consultations with physicians	20122013	12.118.2	34.533.3	53.448.5	5866
Information to patients	20122013	13.024.0	39.128.0	47.848.0	2325
Attractive and welcoming environment	20122013	50.020.0	0.00.0	50.080.0	25
Trust between patients and health care provider	20122013	23.630.7	40.338.7	36.130.7	7275
Openness toward other people	20122013	25.016.0	37.544.0	37.540.0	2425

Table 5: Clinic-level factors for providing equitable health care (%).

Some respondents answered “do not know” or “unclear issue” to the question about other factors that affect health care facilities that offer care on equal terms, or did not respond at all. Some mentioned factors such as “economic resources”, “shortage of doctors/specialists,” “public policy”. Some pointed to the allocation of resources in immigrant-dense areas where appointments take much more time, especially if an interpreter is needed – and time is costly. A lack of continuity of physicians and specialists was described by one respondent as an important factor in being able to offer care on equal terms. Some respondents felt that having “different doctors all the time” reduces the clinic’s ability to offer care on equal terms. Continuity of care is a factor that is mentioned "to meet with the doctor, especially for chronic or for example patients with multiple problems" and "access to fixed-employed doctors' to provide health care on equal terms”. Access to interpretation and “quality of interpreters” and “staff who are knowledgeable about different cultures” are mentioned as other important factors that may be important to offering care on equal terms.

Regarding political governance and leadership, one respondent said that “the leadership’s ignorance and unwillingness” can be an obstacle to offering care on equal terms. One of the respondents thought that the health care system and the rest of society are intertwined: “Health care is linked to the rest of society. The patient/client should have a secure social situation through housing, income, the opportunity to acquire knowledge about how the body works, a good Swedish education, and good interpreters. This makes it easier for us to do our part.” Some respondents raised the issue that it is the health care organization that should adapt to the patient/client and not vice versa, which is a decisive factor in being able to offer care on equal terms. To provide various contact ways and possibilities to communicate with the care has also been mentioned.

They mentioned other factors that may be important in offering care on equal terms “Time and again TIME.” Another respondent explains that “some patients need more time for consultation, which we find difficult to provide.” Another adds, “we don’t have enough staff answering the phones”, and another respondent believes “the resources are not adequate.” “Too much administration and documentation” is mentioned as another factor that may be significant for offering care on equal terms.

Discussion and Conclusion

Statement of principal findings

The results show an increased awareness of the issue of equitable care and the factors behind it among health care providers. A process of change has begun, and if this is to lead to more equitable care and provide lasting effects, it is necessary that the process continues and develops over time.

The results show that the health care providers at the seven participating PHCUs considered the factors affecting clients’/patients’ access to care on equal terms to be composed of three categories: Structural/organizational factors in health care (e.g. shortage of doctors and resources), in the community (e.g. residential area) and patients’/clients’ situation/characteristics (such as education, occupation, ethnicity, age). It also emerged that the five most important subcategories that can be seen as criteria for offering care on equal terms are continuity of care, allocation of financial resources to match the needs of the catchment area and patient groups, availability of trained staff, accessibility, and care that is adapted to clients’ needs. Access to interpreters is mentioned as an important factor for changing and improving the care provided. In addition, other factors that affect PHCUs’ ability to provide care on equal terms are resources in the form of financing, more staff and time, access to permanently-employed physicians and specialists (continuity in care), and adaptation of the services to patients’/clients’ needs. The evaluation shows that the proportion of health professionals who are aware of the action plan for equal care within their occupation had increased in 2013, in comparison with 2012.

The changes in the variables between the measurement time points may be caused by different factors. One cause may be the activities within the project “Health care on equal terms.” The participating PHCUs developed and tested a total of 48 activities to promote more equitable care. These activities vary widely. Not only clients/patients, but also health care providers were in focus in these projects, which may have changed their perceptions between measurement points. The results of the surveys from eight learning seminars and seven team work periods, interviews with operational managers and the project supervisor, and other documentation about the process during the implementation of the project also show that all PHCUs have developed towards providing more equitable care [10,11].

Strengths and weaknesses of the study

There are two strengths in this study. Activities that were considered to influence the changes in the variables between measurement points were based on a modified form of the Breakthrough series with its key components and tools. Another strength is that the researchers were involved in the project from the beginning to the end, and applied a perspective called (among other things) “ongoing evaluation” [21,22] in conjunction with a program-theoretical evaluation perspective [23,24].

The weakness of study is the limitation of time. A follow-up in 2014 could contribute with a more deep analysis and results. Another weakness may be that the study was conducted as a cross-sectional study and included those working at the health care centers at the respective time for the survey. Changing in results in 2012 and 2013 may therefore be reading in relation to this design which we call cross-sectional with “unbalanced panels”. Strength had been on the questionnaire instead, in the form of a panel study, where the same persons responded to the two questionnaires. This aspect can be as a challenge for further studies.

The findings in relation to other studies

The results show the proportion of those who reported that they had “no idea” that patient’ gender, age, mental health and physical functioning were significant for access to care was lower in 2013 than in 2012. The main conclusion is that it was possible to implement changes aiming for more equitable care through projects with a focus on learning. We find some parallels with previous research [25-27] regarding these themes. The results also have some parallels with previous research about the significance for equitable care of communication among colleagues, collaboration within the team, routines, personnel and time and the financial situation [28,29].

Recommendations and implications for clinicians and policy makers

Several aspects of the results can be noted that are relevant to health care units and policy makers. The study shows that it is possible to implement changes for more equitable care with a method centered on learning. The changes that have taken place during this project are at both unit level and staff level. Nevertheless, it is important that the processes which have begun are able to continue and develop over time so that they can lead to sustainable effects and more equal health care in the future. To achieve this, it is also necessary to have appropriate management and compensation systems that can support and promote further development. Hopefully, such mechanisms will enable the process of change to lead to more equitable care for clients /patients and a more manageable working situation for staff.

Ethical Approval

The study was approved by the Ethical Committee in Uppsala (Dnr: 2013/461).

Acknowledgement

The authors wish to thank all the participants in the study; without their contribution, it would not have been possible to undertake the research. Our thanks go to the project leader, Hanna Wallin and to Mälardalen University research team.

Source of Funding

The study was supported by the Swedish Association of Local Authorities and Regions (SALAR).

References

1. Healthcare. Social Department, Stockholm 1982.
2. National Board of Health. Health Care - Situation Reports, 2007. National Board of Health Stockholm 2008.
3. National Board of Health. Illness and health care 1980-2005. National Board of Health, Stockholm 2006.
4. Swedish Municipalities and County Councils. Care on (o) equal terms. A knowledge overview about social differences in Swedish health and social care. Sveriges Kommuner og Landsting, Stockholm 2009.
5. National Board of Health & Public Health. Public health in Sweden. Annual report. National Board of Health, Stockholm 2013.
6. Public Health Authority. Public Health in Sweden, Annual Report 2014. Public Health Agency, Stockholm 2014.
7. Starfield B. Is primary care essential? Lancet 1994; 344: 1129-1133.
8. De Maeseneer J, Willems S, De Sutter A, Van de Geuchte I, Billings M. Primary health care as a strategy for achieving equitable care: A literature review commissioned by the health systems knowledge network. 2007.
9. https://www.ihi.org/resources/Pages/IHIWhitePapers/TheBreakthroughSeriesIHIsCollaborativeModelforAchievingBreakthroughImprovement.aspx
10. Akhavan S, Aytar O, Bogg L, Söderlund A, Tillgren P. Was it a breakthrough? Evaluation of the national learning project - Equal care. Mälardalen University, Eskilstuna/Västerås 2014.
11. Aytar O, Söderlund A. To evaluate with process thinking for learning. Soc Med J 2015; 92: 162-175.
12. Lebo MJ, Weber C. An effective approach to the repeated cross-sectional design. Am J Pol Sci 2015; 59: 242-258.
13. Carter-Pokras O, Baquet C. What is a “health disparity”? Public Health Rep 2002; 117: 426-434.
14. Gwatkin RD. 10 best resources on health equity. Health Policy Plan 2007; 22: 348-351.
15. Graham H. Understanding health inequalities. Open University Press, Berkshire, UK 2009.
16. Ainalem I, Bengtsson E. How has the improvement work been done? A follow-up of some of the development center's education programs. Region Skåne, Malmö 2008.
17. Krippendorf K. Content analysis: An introduction to its methodology. Sage, Beverly Hills, CA 1980.
18. Patton MQ. Qualitative evaluation and research method. Sage, Newbury Park, CA 1990
19. Holme IM, Solvang BK. Research methodology - About qualitative and quantitative methods. Student Literature, Lund 1997.
20. Graneheim UH, Lundman B. Qualitative content analysis in nursing research: Concepts, procedures and measures to achieve trustworthiness. Nurse Educ Today 2004; 24: 105-112.
21. Baklien B. Evaluation in practice. Norwegian Educational Journal 1987; 87: 250-259.
22. https://www.archiwum.ewaluacja.gov.pl/ewaluacja_wstep/Documents/INDICATIVEGUIDELINESONEVALUATIONMETHODSEVALUATIOND.pdf
23. Donaldson SI.Program theory-driven evaluation science: Strategies and applications. Lawrence Erlbaum Associates, New York 2007.
24. Aytar O. "Yes, we also think..." - A program theory perspective on the National Learning Project - Equal care. Soc Med J 2015; 92: 116-131.
25. Beach MC, Price EG, Gary TL, Robinson KA, Gozu A, et al. Cultural competency: A systematic review of health care provider educational interventions. Med Care 2005; 43: 356-373.
26. Smith EM, Johnson SR, Guenther SM. Health care attitudes and experiences during gynecologic care among lesbians and bisexuals. Am J Public Health 1985; 75: 1085-1087.
27. U.S. Department of Health and Human Services. Healthy People 2010: Under-standing and improving Health. U.S. Government Printing Office, Washington, D.C 2000.
28. Brandstorp H, Halvorsen PA, Sterud B, Haugland B, Kirkengen AL. Primary care emergency team training in situ means learning in real context. Scand J Prim Health Care 2016; 34: 295-303.
29. Trubek LG, Das M. Achieving equality: Health care governance in transition. Am J Law Med 2003; 29: 395-421.