Appraise - understand results
What is critical appraisal?
Critical appraisal involves critically examining a study to determine its validity, reliability and relevance. It is an important and essential part of the EBP 5 step process. For a more comprehensive overview refer to the critical appraisal for health guide.
It is useful to ask the following three questions when conducting an appraisal:
- Are the results of the study valid?
- Are the results of the study reliable?
- Are the results applicable to my situation? (Saewert & Hagler, 20151).
There are many different appraisal tools that can be used depending on the type of study. See below for some examples.
1. Saewert, K. J., & Hagler, D. (2015). Generating the evidence for health professional education: The five As of the scholarship of learning and teaching - ask, answer, access, appraise and apply. In T. Brown & B. Williams (Eds.), Evidence-based education in the health professions: Promoting best practice in the learning and teaching of students (pp. 67-80). London: Radcliffe.
Critical appraisal tools & guides
There are a number of checklists / tools available Internet to assist in the critical appraisal of evidence.
- CASP checklistsThis set of eight critical appraisal tools are designed to be used when reading research, these include tools for Systematic Reviews, Randomised Controlled Trials, Cohort Studies etc.
- Centre for Evidence Based MedicineUniversity of Oxford - contains useful tools and worksheets for the critical appraisal of different types of medical evidence.
- How to use the Evidence (Australian National Health and Medical Research Council)This handbook focuses on how to evaluate and use the evidence gathered from a systematic literature review to inform the development of evidence-based clinical guidelines
- International Centre for Allied Health Evidence - University of South AustraliaA comprehensive list of critical appraisal tools available for use.
- PRISMASystematic reviews & meta-analyses. Includes links to PRISMA flow diagram and checklist.
Key EBP statistics
The probability that a particular result could have happened by chance. Used to assess whether results are statistically significant (where the P-value is less than 5% - P<0.05).
P-values and Type I Error by Terry Shaneyfelt (YouTube)
An estimate of treatment effect which accounts for the sampling error between the study population and the larger population that the study represents. Indicates that in the wider population, there is e.g. 95% confidence that results would fall within a given range. A narrow range indicates a higher degree of precision.
Confidence Intervals by Terry Shaneyfelt (YouTube)
A measure of the effectiveness of interventions which indicates clinical significance. The number of patients who need to be treated to prevent one bad outcome.
How to calculate the Number Needed to Treat by Terry Shaneyfelt (YouTube)
Recommended resources
- Understanding health research: a tool for making sense of health studiesThis tool will guide you through a series of questions to help you to review and interpret a published health research paper.
- Duke University: Explaining the resultsIncludes videos & articles on ARR, NNT, RR, RRR Odds Ratio & more
- What is …?Brief explanations of hot topics in evidence-based healthcare. Statistics topics include confidence intervals and p-values, hazard ratios, sensitivity analysis and more
- CEBM GlossaryFind definitions for key statistical terms used in Evidence-Based practice
- CEBM - University of OxfordNumber Needed to Treat (NNT)
- The CONSORT StatementRecommendations for reporting randomized trials, to facilitate complete and transparent reporting, and to aid critical appraisal and interpretation.