Study design

It is important to look at study design for two reasons:

  • Different types of questions are best answered by different study designs
  • The criteria you use to appraise the research will vary depending upon the study design

Type of question & study design

The best study design to answer your question will depend on the type of question you have. For each type of question, there is a best type of study - if this is not available you could consider moving down the evidence hierarchy.

Most common type of questions: Type of study:
Diagnosis - how to select and interpret diagnostic tests Prospective, blind comparison to a gold standard or cross-sectional
Therapy - how to select treatments that do more good than harm and that are worth the efforts and costs of using them Randomized controlled trial > cohort study
Prognosis - how to estimate the patient’s likely clinical course over time and anticipate likely complications of disease Cohort study > case control > case series
Harm/Etiology - how to identify causes for disease (including iatrogenic forms) Cohort > case control > case series

(Source: University of North Carolina-Chapel Hill Health Sciences Library, Introduction to Evidence Based Practice)

Determining the validity

To determine the validity of a study, consider these key issues:

Diagnosis

  • Any diagnostic uncertainty?
  • Was there an independent, blind comparision with a reference ('gold') standard?
  • Did each patient receive both tests?

Therapy (e.g. randomised controlled trial)

  • Were patients randomised?
  • Was group allocation concealed?
  • Are there similar baseline characteristics?
  • Was the study blinded, and to what extent?
  • Was follow-up complete?
  • Was there an intention-to-treat?

Prognosis (e.g. cohort study)

  • Was the sample of patients well-defined?
  • Did patients have a similar prognosis?
  • Was follow-up complete?

Harm/etiology (e.g. cohort or case control study)

  • Are the comparison groups similar?
  • Are the outcomes and exposures measured the same for both groups?
  • Was follow-up complete?

Systematic review

  • Is it a well focussed question?
  • Is the literature search thorough?
  • Does it include validated studies?
  • Is the study reproducible?

Qualitative research (Source: CASP Qualitative checklist)

  • Are the aims of the research clearly stated?
  • Was qualitative methodology appropriate?
  • Was the recruitment strategy appropriate?
  • Was data collected in a way that addressed the research issue?
  • Was the relationship between researcher and participants adequately considered?
  • Was the data analysis sufficiently rigorous?