What is Critical appraisal?

Critical appraisal is the systematic evaluation of clinical research papers in order to establish:

  1. If the study addresses a clearly focused question?
  2. Are valid methods used to address this question?
  3. If the study results are important?
  4. Are these results applicable to my patient or population?

Resources for critical appraisal and reporting guidelines are available in The Equator Network, JAMA, Oxford Evidence based medicine, CASP, SQUIRE and TREND.

How to do Critical appraisal?

The steps involved in critical appraisal are

  1. Choose a journal article relevant to your research question
  2. Read the article in detail, especially the section on methods
  3. Answer the critical appraisal questions
  4. Make a presentation on critical appraisal of the article

Quick references for each study design:

The critical appraisal questions for each study design is listed below. The questions are based on JAMA Users guide to Medical Literature. However, we strongly recommend reading the guide in detail prior to critical appraisal.

How serious was the risk of bias?

Did intervention and control groups start with the same prognosis?
Were patients randomized?
Was randomization concealed?
Were patients in the study groups similar with respect to known prognostic factors?
Was prognostic balance maintained as the study progressed?
To what extent was the study blinded?
Were the groups prognostically balanced at the study’s completion?
Was follow-up complete?
Were patients analyzed in the groups to which they were randomized?
Was the trial stopped early?

What are the results?

How large was the treatment effect?
How precise was the estimate of the treatment effect?

How can i apply the results to patient care?

Were the study patients similar to my patient?
Were all patient-important outcomes considered?
Are the likely treatment benefits worth the potential harm and costs?

Was the Process Credible?

Did the Review Explicitly Address a Sensible Clinical Question?
Was the Search for Relevant Studies Detailed and Exhaustive?
Was the Risk of Bias of the Primary Studies Assessed?
Did the Review Address Possible Explanations of Between-Study Differences in Results?
Did the Review Present Results That Are Ready for Clinical Application?
Were Selection and Assessments of Studies Reproducible? Did the Review Address Confidence in Effect Estimates?

Were Eligibility Criteria for Inclusion in the Systematic Review Appropriate?

Are results likely to be similar across the range of included patients (eg, older and younger, sicker and less sick)?
Are results likely to be similar across the range of studied interventions or exposures (eg, for therapy, higher dose or lower dose; for diagnosis, test results interpreted by experts or nonexperts)?
Are results likely to be similar across the range of ways the outcome was measured (eg, shorter or longer follow-up)?

Rating Confidence in the Estimates (The Quality ofEvidence) The GRADE Approach How Serious Is the Risk of Bias in the Body of Evidence?

Are the Results Consistent Across Studies?
How Precise Are the Results? Do the Results Directly Apply to My Patient?
Is There Concern About Reporting Bias?
Are There Reasons to Increase the Confidence Rating?

How serious is the risk of bias?

Did participating patients constitute a representative sample of those presenting with a diagnostic dilemma?
Did investigators compare the test to an appropriate, independent reference standard?
Were those interpreting the test and reference standard blind to the other result?
Did all patients receive the same reference standard irrespective of the test results?

What are the results?

What likelihood ratios were associated with the range of possible test results?
How can i apply the results to patient care?
Will the reproducibility of the test results and their interpretation be satisfactory in my clinical setting?

Are the study results applicable to the patients in my practice?

Will the test results change my management strategy?
Will patients be better off as a result of the test?

How serious is the risk of bias?

Did the investigators enrol the right patients?
Was the patient sample representative of those with the disorder?
Was the definitive diagnostic standard appropriate?
Was the diagnosis verified using credible criteria that were independent of the clinical manifestations under study?
Were clinical manifestations sought thoroughly, carefully, and consistently?
Were clinical manifestations classified by when and how they occurred?

What are the results?

How frequently did the clinical manifestations of disease occur?
How precise were these estimates of frequency? When and how did this clinical manifestations occur in the course of disease?

How can I apply the results to patient care?

Are the study patients similar to my own?
Is it unlikely that the disease manifestations have changed since this evidence was gathered?

Are the Results Valid?

Did the Study Patients Represent the Full Spectrum of those With This Clinical Problem?
Was the Diagnostic Evaluation Definitive?

What Are the Results?

What Were the Diagnoses and Their Probabilities?
How Precise Are the Estimates of Disease Probability?

How Can I Apply the Results to Patient Care?

Are the Study Patients and Clinical Setting Similar to Mine?
Is It Unlikely That the Disease Possibilities or Probabilities Have Changed Since This Evidence Was Gathered?

Is the clinical question clear and comprehensive?

Is the recommended intervention clear and actionable?
Is the alternative clear?
Were all of the relevant outcomes important to patients explicitly considered?

Was the recommendation based on the best current evidence?

Are values and preferences associated with the outcomes appropriately specified?

Do the authors indicate the strength of their recommendations?

Is the evidence supporting the recommendation easily understood?

For strong recommendations, is the strength appropriate?
For weak recommendations, does the information provided facilitate shared decision making?
Was the influence of conflict of interests minimized?

Are the results valid?

Did the recommendations consider all relevant patient groups, management options, and possible outcomes?

  • Are the results valid?
  • Did investigators adopt a sufficiently broad viewpoint?
  • Are results reported separately for relevant patient subgroups?

Is there a systematic review and summary of evidence linking options to outcomes for each relevant question?

  • Were costs measured accurately?
  • Did investigators consider the timing of costs and consequences?

What are the results?

  • What were the incremental costs and effects of each strategy?
  • Do incremental costs and effects differ between subgroups?
  • How much does allowance for uncertainty change the results?

How can I apply the results to patient care?

  • Are the treatment benefits worth the risks and costs?
  • Can I expect similar costs in my setting?

Are the Results Credible?

Was the Choice of Participants or Observations Explicit and Comprehensive?
Was Research Ethics Approval Obtained? Was Data Collection Sufficiently Comprehensive and Detailed?
Did the Investigators Analyze the Data Appropriately and Corroborate the Findings Adequately?

What Are the Results?
How Can I Apply the Results to Patient Care?

Does the Study Offer Helpful Theory?
Does the Study Help Me Understand Social Interactions in Clinical Care?

Are the recommendations valid?

Did the recommendations consider all relevant patient groups, management options, and possible outcomes?
Is there a systematic review linking options to outcomes for each relevant question?
Is there an appropriate specification of values or preferences associated with each of the outcomes?
Do the authors indicate the strength of their recommendations?

What are the results?

What are the key recommendations?

Will the recommendations help you in caring for your patients?

Do the recommendations make sense in your practice setting?

How serious is the risk of bias?

Did the cases and control group have the same risk for being exposed in the past?
Were cases and controls similar with respect to the indication or circumstances that would lead to exposure (or did statistical adjustment address the imbalance)?
Were the circumstances and methods for determining exposure similar for cases and controls?

What are the results?

How strong is the association between exposure and outcome? How precise was the estimate of the risk?

How can i apply the result to patient care?

Were the study patients similar to the patient in my practice?
Is the exposure similar to what might occur in my patient? What is the magnitude of the risk?
Are there any benefits that are known to be associated with exposure?

How serious is the risk of bias?

In a cohort study, aside from the exposure of interest, did the exposed and control groups start and finish with the same risk for the outcome?
Were patients similar for prognostic factors that are known to be associated with the outcome (or did statistical adjustment address the imbalance)?
Were the circumstances and methods for detecting the out-come similar?
Was the follow-up sufficiently complete?

What are the results?

How strong is the association between exposure and outcome?
How precise was the estimate of the risk?

How can i apply the result to patient care?

Were the study patients similar to the patient in my practice?
Was follow-up sufficiently long?
Is the exposure similar to what might occur in my patient?
What is the magnitude of the risk?
Are there any benefits that are known to be associated with exposure?

How serious is the risk of bias?

Is there randomized control trial evidence addressing the impact of the intervention in asymptomatic people?

What are the recommendations, and will they help you in caring for patients?

Were the data identified, selected, and combined in an unbiased fashion?
What are the benefits?
What are the harms?
How do benefits and harms compare in different people and with different screening strategies?
What is the effect of individuals’ values and preferences?
What is the effect of uncertainty associated with the evidence?
What is the cost-effectiveness?

BACK