Knowledge Share - (2008) Volume 16, Issue 2
Evidence based/Knowledge Management Librarian, KnowledgeShare, The Library, Audrey Emerton Building, Royal Sussex County Hospital, Brighton, UK
Received date: 20 December 2007; Accepted date: 8 January 2008
Lately it seems that the term ‘evidence-based practice’, which has often been misunderstood and maligned, is falling even further out of favour, with ‘clinical effec-tiveness’ perhaps regaining prominence to take its place. Evidence-based practice (EBP) has often been seen, wrongly, as a form of medicine-by-numbers; a policy-driven straightjacket that elevates randomised trials and systematic reviews at the expense of all other knowledge.[1] Yet no matter what one chooses to call it, it seems self-evident that clinical staff need to factor research evidence into their decision making. Time and again it has been shown that ignoring research and relying on intuition can lead to suboptimal treat-ment,[2] misdiagnoses,[3] and effective therapies being ignored for years.[4]
The key to reconciling these viewpoints is the understanding that EBP is about integrating the best available evidence with one’s clinical experience and expertise. A randomised controlled trial (RCT) is almost always the most convincing way of demon-strating the effectiveness of a therapy, but if an RCT simply hasn’t been conducted then it is perfectly legitimate to look for a cohort study or some other form of research. Different types of question require different research methodologies, and even the humble case series has an important place in the development of healthcare knowledge. Similarly, clinical experience is paramount, as without it, it is impossible to inter-pret and adapt the evidence to fit an individual patient’s history, pReference and combination of conditions.
But the phrase ‘best available’ is also what makes evidence-based practice so difficult, in practice. You shouldn’t base a treatment decision on a cohort study if an RCT has been published, and you shouldn’t base it on a poor RCT if a larger, less-biased trial is avail-able. Healthcare staff need to know that they have the best, most up-to-date information on a subject, but often don’t have the time or confidence to effectively search the literature and evaluate their findings. One solution to this problem is to refer to systematic reviews and practice guidelines, which are based on exhaustive searches and critical appraisal of the literature. Un-fortunately, systematic reviews are highly focused on specific interventions, tending to be poor at providing an overview of a subject area, and guidelines take a long time to produce, so rapidly go out of date.
Thankfully, there are now various resources that collect together all of the best evidence on the inci-dence, risk factors, diagnosis, treatment and prognosis of particular conditions; evidence summaries that are regularly updated, fully Referenced, and checked by experts in the field. Some have been purchased nationally for use by NHS staff, but there are others that may be purchased by individuals or trusts. The NHS libraries in Brighton and Haywards Heath re-cently ran a joint trial of a number of these resources and invited comment from clinicians. The following comments are based on the views of a small number of health librarians and responses from clinical staff.
Clinical Knowledge Summaries: https://cks.library.nhs.uk
Many primary care staff will already be familiar with the National Library for Health’s Clinical Knowledge Summaries (CKS) website, even if they are used to calling it by another name: Prodigy. In 2007, all of the content from the Prodigy site was migrated across to CKS, and work is going on to restructure the Prodigy guidance into a new topic review format. Around 500 conditions are covered, all of which describe clinical scenarios commonly encountered in primary care. The resource is funded by the NHS.
The topic reviews provide information on goals, outcome measures, audit criteria, and indicators for the Quality and Outcomes Framework (QOF). They are based where possible on existing guidelines and systematic reviews, or alternatively the authors will conduct their own systematic search of the healthcare literature. Everything is fully Referenced and is reviewed prior to publication by professional organisations, patient groups, and other experts. Updates are triggered by the release of new national guidance, Cochrane reviews, Health Technology Assessments, safety alerts, and, more rarely by the publication of important new original research.
The information is presented clearly; by clicking on ‘Clinical Knowledge’ you can browse for a condition and see different levels of detail depending on how much time is available to you. The ‘In Summary’ information is designed to be read in 15 seconds or less. In addition to these topic reviews, CKS includes patient information leaflets on conditions, tests and treatments produced by NHS Direct. There is also a ‘KnowledgePlus’ section, which provides briefings on regularly changing topics, such as highlights from new Cochrane reviews, a test of the week, medicolegal issues and drug safety updates.
DynaMed (access via Clinical Knowledge Summaries, see below)
DynaMed is an American collection of evidence-based summaries that has been purchased for use by NHS staff. It can be accessed by staff in England and Wales via the CKS site (above). There is a small link in the top right-hand corner that takes you through to the DynaMed topic list. Eventually all of the content in DynaMed will be searchable through CKS and the National Library for Health, but at the time of writing it is necessary to access the sites independently.
One of the reasons that you may want to access DynaMed is that it contains close to 2000 topic sum-maries, far more than CKS itself. In addition, the content is updated more frequently. The producers look through the contents of 500 journals, plus sys-tematic review databases, and update the database daily to incorporate significant new evidence. All articles are evaluated to assess the validity and reliability of the methodology as well as their clinical significance. Per-haps most importantly, the layout of the topic sum-maries is extremely user friendly, with large amounts of information clearly categorised and subdivided so that it is easy to jump quickly to the section of interest. Sections on general information about a condition, causes and risk factors, complications and associated conditions, history, physical assessment, diagnosis, prognosis, treatment and prevention are further broken down, using a bullet-point style that keeps things detailed but concise.
Although the format of DynaMed is appealing, our survey showed that the resource isn’t quite what clinical staff are looking for. About half of those who looked at it said that DynaMed ‘partly’ answered their question, and one user said ‘DynaMed is okay but doesn’t have comprehensive cover in oncology’. All the resources in the trial scored highly for accuracy of content, but DynaMed was not rated quite as well as either Clinical Evidence or Up-To-Date, and the same is true of ‘value of content’. Despite this, a study published in the Annals of Family Medicine in 2005 randomised primary care clinicians to search for answers to their clinical questions using DynaMed or without it, and found that they ‘answered more questions and changed clinical decisions more often, without in-creasing overall search time’ when using DynaMed.[5]
Map of Medicine
If you would like to see evidence-based information about a condition presented in a more graphical, algorithmic format, the Map of Medicine (MoM) is ideal. This is yet another online service that has been purchased nationally by the NHS, although it is being implemented and accessed slightly differently around the country. Your local NHS librarian should be able to advise you on how to access the resource in your area.
The Map, which has been created by clinicians, presents patient pathways as flow diagrams. These begin with the presentation of symptoms in primary care, indicating which tests to perform and what should be done depending on the results. Obviously these algorithms need to be adapted to suit individual patients, but they do provide extremely helpful guid-ance for generalists and those working in an area that is new to them. In most parts of the county the pathways will still be generic, based on National Institute for Health and Clinical Excellence (NICE) guidelines in many cases, but the added value of MoM is that it can be adjusted to reflect practice in the local area. Where patient pathways deviate from the guidance, perhaps because of the availability of certain services or the clinical judgement of local staff, or where national guidance is not available, the Map can be localised. In addition, clinic times and contact numbers can be added so that where the Map suggests referral to secondary care this process can be seamless. Questions remain about who will fund this localisation process, but the potential for standardising care within a region is very exciting.
Clinical Evidence: https://clinicalevidence.bmj.com
Clinical Evidence is published by the BMJ Publishing Group. Until last year, access to this resource was made available to all NHS staff, but this is now no longer the case, with only those in Scotland and Wales still eligible (via an Athens password). NHS institu-tions in England can opt to subscribe individually to Clinical Evidence by contacting BMJ Publishing through the above website.
As with all the sites covered here, Clinical Evidence provides reviews that summarise the current state of knowledge with regard to the prevention and treat-ment of medical conditions. These are based on the best available evidence; randomised trials and system-atic reviews where available, but including less-con-vincing research methodologies where necessary. One of the biggest selling points of Clinical Evidence is the clarity of presentation, which groups potential ther-apies according to the strength of evidence behind them, identifying those that are ‘beneficial’, ‘likely to be beneficial’, ‘likely to be ineffective or harmful’, and so on. There is summary information for each con-dition, reporting incidence, risk factors, etc, although this tends to be less comprehensive than some of the other resources indicated in this article, and there is little emphasis on diagnosis. Links to relevant national guidelines are presented alongside the review, in add-ition to the obligatory Reference and an opportunity for users to add their own responses to the text.
Clinical Evidence has always been a well-liked resource, and in our recent trial, most of those who looked at it said that they found an answer to their question. It scored extremely well for ease of use and came second (after UpToDate; see below) for both accuracy and value of the content. Comments include ‘very impressed with Clinical Evidence’, with about one-quarter of respondents indicating that it was their preferred resource. The only drawbacks seem to be the heavy focus on therapy, to the exclusion of other aspects of a condition, and the fact that some clinicians seem to prefer more concrete guidance on how to treat.
UpToDate: www.uptodate.com
UpToDate seems to be the most popular of the evidence-based summary resources, at least according to our small survey, despite its more American slant. The producers of UpToDate are based in the US and most of the 3600 clinicians who contribute to the content are drawn from their top medical schools. The content is updated every four months, and involves the authors reading and reviewing hundreds of health-care journals and incorporating that research which is well supported by the data and clinically useful.
The format of UpToDate is more textbook like, less concise, than other resources listed here, and does not lend itself as easily to quick identification of relevant chunks of information. However the coverage is superb. A search for ‘Crohn’s disease’ turns up articles on the condition’s management in children, in adults, and in pregnancy. Separate articles give detail on clinical manifestations and diagnosis, various different medication-specific articles, complications, nutritional considerations and more. Each of these topics seem exhaustive, bringing together all the convincing evi-dence in one place. What is lost in ease of use is made up for in depth of content.
It is this comprehensiveness that made UpToDate the clear winner in our small-scale trial, as well as its ‘excellent’ drugs interaction programme and ‘well-written’ copy. UpToDate scored most highly for value and accuracy of content, and it was considered just as easy to find information here as in Clinical Evidence. Around three-quarters of respondents marked it as their preferred resource.
Evidence Matters: www.evidencematters.com
The last resource we looked at was Evidence Matters, also from North America. This resource differs quite substantially from the others discussed above, firstly in the way that you find the information, and secondly in the way the information is presented. Instead of searching for pre-written topic reviews, the user can construct their own question to a high level of sophis-tication by choosing a condition, then selecting a therapy from a list of options, indicating the outcome measure that they are interested in and whether there is some other specific intervention they would like to compare. Evidence Matters then retrieves from its database all studies that have answered this particular question and presents the results, stratified according to study design. The results can be further refined according to the participants’ sex, age range, race, or co-existing diseases, and by the type of research, country of origin, blinding characteristics or whether there is a declared interest by an author or pharma-ceutical company.
The results of each study in the database are presented in a very systematic, tabular and graphical format. There is no explanatory text at all, just the numerical results and answers to standard questions such as ‘dose frequency’, ‘blinding characteristics’, ‘interpretation of significance’. This presentation takes some getting used to and could be off-putting to anyone who struggles to interpret spreadsheets of data. Others may appreciate the effort to just focus on the bare essentials of a study.
Whether this presentation of data is helpful or not, Evidence Matters is seriously undercut by the paucity of content. Although cancer is reasonably well served, many other disease areas are barely covered. In gastro-enterology only gastro-oesophageal reflux disease and peptic ulcer are included; the only gynaecological condition is polycystic ovary syndrome; and so on. This makes it difficult to justify purchasing this resource on an institution-wide basis, although it may be appre-ciated by oncologists. Evidence Matters is also let down by the absence of overview information on diseases. The focus is solely on the effectiveness of interventions, and so the resource says nothing about incidence, diagnosis or risk factors. None of the clinical staff who looked at the system in our trial found exactly what they were looking for, and the site scored poorly for ease of use.
No attempt is made to suggest that our small-scale trial of these resources provides definitive results, and readers are encouraged to take out their own trial subscriptions of these evidence-based summary data-bases to judge for themselves. However, the range of topics searched for by our clinicians was quite wide-ranging, from non-steroidal anti-inflammatory drugs (NSAIDs) to prostate-specific antigen (PSA) screening to candidiasis and renal issues. UpToDate was the most well-received resource, particularly with regard to the extent of its coverage, but Clinical Evidence scored equally well in terms of presentation of infor-mation, and in my own opinion is better in this regard.
Thanks to Erica Rae, Assistant Librarian at Brighton and Sussex University Hospitals NHS Trust for her contributions.
