Guest Editorial - (2011) Volume 19, Issue 1
General Practitioner, Jessop Medical Practice, Church Farm Primary Care Centre, Ripley, UK
Received date: 16 March 2010; Accepted date: 22 December 2010
Impact factor (IF) is a contentious issue. Academics have become obsessed by it. And little wonder. It has weighed their performances, measured their publi-cations and even, in part, influenced funding and the scoring of research units through the research assess-ment exercise (RAE). But while the RAE, which was used by the Higher Education Funding Council of England to survey the quality of research in British universities, was based on peer review of an academic department’s research, it is being replaced by the Research Excellence Framework (REF), which will rely far more heavily on statistical measures of research quality, such as IF.[1] Publish consistently in high IF journals and an academic’s research grants will stead-ily grow, their international profile burgeoning. Fail to do so and the flow of grant funding will slow and, perhaps, research quality will diminish. The old maxim ‘publish or perish’ still holds true.
The concern of many, especially those in the field of primary care research, is that an overemphasis on IF may not give a true measure of the impact of research on society as a whole. There have been a number of attempts at developing alternative scoring systems to include a measure of ‘societal impact’. A British team developed a ‘pay back model’ which outlines five di?erent categories of outcome from research, journal IF being only one of a number of measures.[2] Based on this work an Australian study attempted to develop a feasible way of assessing the impact of primary health-care research by assessing four funded research projects. They concluded that the project with the highest impact overall was the one which had no peer reviewed publications. They considered that the strongest path-ways to impact in primary healthcare research are formed by strong collaborative links, personal relation-ships and the involvement of practitioners, healthcare managers and policy makers in defining the research question and process, as well as good dissemination and the involvement of credible champions. Based on their study they expanded the Buxton and Hanney Payback Framework for measuring the impact of research.[3]
So what is IF? IF was introduced in 1963 by Garfield and Sher to improve the management of library journal collections. It is measured by dividing the number of times articles published in a particular journal over a two-year period were cited in indexed journals in the third year by the number of citeable items. So if the journal published 100 citeable articles and these articles were overall cited in indexed journals 100 times in the third year, it would have an IF of one. Here are a few examples. The New England Journal of Medicine has an IF of 47.05. Publish here and you could say your academic career was sorted. The Lancet has an IF of 30.758, the British Medical Journal (BMJ) 13.66 and the British Journal of General Practice 2.442.
But does IF influence general practice? I suggest that high IF medical journals have a significant influence over the routine practice of general practitioners, but not always directly or immediately. Let me explain. Many factors a?ect the journal IF. Basic science research and fields with rapid scientific change, where papers are likely to be cited within the next two years, tend to have high IF. Although it is sometimes felt that such journals are remote from influencing society, the ulti-mate goal of all basic medical science ought to be, through its eventual incorporation into clinical trials, the improvement in health and quality of life of indi-viduals and societies. It has even been suggested that leading high IF journals may be read infrequently by general practitioners.[4] This may well be true concern-ing basic science journals, but I believe that much of the relevant content of general medical journals with high IF filters down inexorably to grass roots general practice, shaping the knowledge base for clinical practice. This occurs through at least three di?erent routes. First, prominent clinically relevant studies often hit news headlines, informing both patients and general practitioners. Second, the BMJ, an example of a high IF medical journal, has a total average issue readership (TAIR) of 69% of UK general practitioners. Therefore, not only are its research and review papers read by the majority of UK general practitioners, but through its ‘Short Cuts’ section it references other significant publications from high IF journals. Third, I suggest that leading high IF medical journals influence evi-dence-based guideline development. Although I could find no UK based papers to support this conclusion, one study I found analysed the bibliographic citations of articles used to support the annual Guides to Clinical Preventive Services by the US Preventive Services Taskforce (USPSTF). These are generally thought to reflect the highest level of evidence. The study found that the number of citations by the USPSTF guidelines roughly paralleled the IF for the respective journals, although journals with low IF were also acknowledged as providing important evi-dence.[5]
In the UK evidence-based guidelines, based on systematic reviews of the literature, are now being incorporated into the Quality and Outcomes Frame-work (QOF) – the largest healthcare pay for perform-ance programme in the world, through which by earning the maximum number of performance indi-cator ‘points’ an average sized general practice can earn about £125 000 (e185 000; $252 000) in addition to its usual sources of income.[6] Whether we like it or not, payment shapes general practice work patterns. It has already been noticed that QOF may lead to general practitioner consultations being taken over by the agendas of well-meaning single disease interest groups.[7] The incorporation of these guidelines into QOF is therefore not all good. In addition, many of these guidelines still do not meet the internationally ac-cepted quality criteria, as defined in the AGREE instrument,[8] and there is the risk of substantial bias in guideline development, many of the recommen-dations being for ‘ideal patients’ (e.g. adults without co-morbidity). Furthermore, these guidelines are ex-ceptionally time-consuming and expensive to develop
– between about e100 000 and e200 000.[9]
In summary, IF may have its limitations. It meas-ures a journal’s impact on the scientific community rather than on clinical practice, where the widespread practical application of a new finding gains no citations. Strategic editing of a journal’s contents can increase its IF. Basic science journals, which are not immediately clinically relevant, tend to have higher IF than primary care journals. And yet IF does appear to influence the development of knowledge which is core to general practice. Whether it is eventually replaced by a biblio-metric, which would have the facility of incorporating the societal impact of research, remains to be seen. This would be a step well received by primary care researchers, a measure capable of quantifying the hugely beneficial impact they themselves are having on shaping general practice.
Not commissioned; internally peer reviewed.
