Study design
Learn how to critically appraise the evidence, determine the validity of a study, and identify the best kind of study design for different types of questions such as diagnosis, therapy, prognosis and harm/etiology.
Study design
It is important to look at study design for two reasons:
- Different types of questions are best answered by different study designs
- The criteria you use to appraise the research will vary depending upon the study design
Type of question & study design
The best study design to answer your question will depend on the type of question you have. For each type of question, there is a best type of study - if this is not available you could consider moving down the evidence hierarchy.
| Most common type of questions: | Type of study: |
| Diagnosis - how to select and interpret diagnostic tests | Prospective, blind comparison to a gold standard or cross-sectional |
| Therapy - how to select treatments that do more good than harm and that are worth the efforts and costs of using them | Randomized controlled trial > cohort study |
| Prognosis - how to estimate the patient’s likely clinical course over time and anticipate likely complications of disease | Cohort study > case control > case series |
| Harm/Etiology - how to identify causes for disease (including iatrogenic forms) | Cohort > case control > case series |
(Source: University of North Carolina-Chapel Hill Health Sciences Library, Introduction to Evidence Based Practice)
Determining the validity
To determine the validity of a study, consider these key issues:
Diagnosis
- Any diagnostic uncertainty?
- Was there an independent, blind comparision with a reference ('gold') standard?
- Did each patient receive both tests?
Therapy (e.g. randomised controlled trial)
- Were patients randomised?
- Was group allocation concealed?
- Are there similar baseline characteristics?
- Was the study blinded, and to what extent?
- Was follow-up complete?
- Was there an intention-to-treat?
Prognosis (e.g. cohort study)
- Was the sample of patients well-defined?
- Did patients have a similar prognosis?
- Was follow-up complete?
Harm/etiology (e.g. cohort or case control study)
- Are the comparison groups similar?
- Are the outcomes and exposures measured the same for both groups?
- Was follow-up complete?
Systematic review
- Is it a well focussed question?
- Is the literature search thorough?
- Does it include validated studies?
- Is the study reproducible?
Qualitative research (Source: CASP Qualitative checklist)
- Are the aims of the research clearly stated?
- Was qualitative methodology appropriate?
- Was the recruitment strategy appropriate?
- Was data collected in a way that addressed the research issue?
- Was the relationship between researcher and participants adequately considered?
- Was the data analysis sufficiently rigorous?