**Literature search ** **Literature search **
**Selection criteria**
**Statistical analysis **
**Heterogeneity**
**Publication bias**
**Narrative procedure (conventional critical review method)** **Narrative procedure (conventional critical review method)**
**Vote-counting method (significant results marked “+”, converse “–” and no significant results “neutral”)**
**Combined tests (combining the probabilities obtain from two or more independent studies)**
**Systematic review is the entire process of collecting, reviewing and presenting all available evidence ** **Systematic review is the entire process of collecting, reviewing and presenting all available evidence **
**Meta-analysis is the statistical technique involved in extracting and combining data to produce a summary result**
**A meta-analysis is also possible without doing a systematic review** **A meta-analysis is also possible without doing a systematic review**
**With no attempt to be systematic about the particular studies were chosen**
**To increase power** **To increase power**
**To improve precision**
**To answer questions not posed by the individual studies**
**To settle controversies arising from apparently conflicting studies or **
**To generate new hypothesis**
**Assessment of strength of evidence**
- To determine whether an effect exists in a particular direction
**Statistical pooling of results**
- To obtain a single summary result
**Investigation of heterogeneity**
- To examine reasons for different results
**A meta-analysis is a two-stage process** **A meta-analysis is a two-stage process**
**Stage 1** - Extraction of data from individual study
- Calculation of a result for that study (point estimate)
- Estimation of chance variation (confidence interval)
**Stage 2** - Deciding if it is appropriate to calculate and pool average results across studies
- If so, calculate and present the results.
## What are the main comparisons in your view? ## What are the main comparisons in your view? ## How will you summarise the results of the outcomes for each study? ## How will you decide whether to combine the results of the separate studies? ## Do you plan any subgroup or sensitivity analyses?
**Dichotomous data** (e.g. dead or live) **Dichotomous data** (e.g. dead or live)
**Counts of events** (e.g. no. of pregnancies)
**Short ordinal scales** (e.g. pain score)
**Long ordinal scales** (e.g. quality of life)
**Continuous data** (e.g. cholesterol con.)
**Censored data or survival data** (e.g. time to 1st service)
**Continuous data** **Continuous data**
- Calculation of overall effect size (standardised mean difference)
**Rate data**
- Measures of effect (difference between incidence in the population of exposed
*vs* not exposed) - Relative risk
- Odds ratio
- Risk difference
**Fixed effect models** **Fixed effect models**
- Mantel-Haenszel (MH)
- Peto test (modified MH method)
- Recommended for non-experimental studies
**Random effect models**
- DerSimonian & Laird method
- Bayesian method
**Regression models (Mixed model)**
## This model is based on a mathematical assumption that every study is evaluating a common treatment effect ## This model is based on a mathematical assumption that every study is evaluating a common treatment effect ## In this model, the **true treatment** difference is considered to be the same for all trials ## The SE of each trial estimate is based on sampling variation within the trial ## The summary results are specific to the trials included ## The summary results can not be generalised to the population
**Mantel-Haenszel approach** **Mantel-Haenszel approach**
- Odd ratio
- Risk ratio
- Risk difference
- Not recommended in review with sparse data (trials with zero events in treatment or control group)
**Peto method**
- Odds ratio
- Used in studies with small treatment effect and rare events
- Not a very common method
- Used when the size of groups within trial are balanced
## In this model, the assumption is that the **true treatment effects** in the individual studies may be different from each other ## In this model, the assumption is that the **true treatment effects** in the individual studies may be different from each other ## In this model, the **true treatment difference** in each trial is itself assumed to be a realisation of random variable, which is usually assumed to be normally distributed ## The SE of each trial estimate is increased due to the addition of this between-trial variation
**Odd ratio** **Odd ratio**
**Risk ratio**
**Risk difference**
**Fixed effects assumption**
- “
**did** the treatment produce benefit on average in the studies in hand”? - “what is the
**best estimate** of the treatment effect”?
**Random effects assumption**
- “
**will** the treatment produce benefit on average”? - “what is the
**average treatment** effect”?
**Choice between fixed and random effects may be decided**
- By a formal chi-square test of homogeneity
- That is whether the between study variance component is zero or not
**Risk ** **Risk **
- A chance or probability of having a specific event (no of participants having the event in a group divided the total no. of participants)
**Odds**
- The ratio of events to not-events (risk of having an events divided by the risk of not having it)
**Odds Ratio (OR)**
- The odds of the event occurring in one group divided by the odds of the event occurring in the other group
**Relative risk or Risk Ratio (RR)**
- The risk of the events in one group divided by the risk of the event in the other group
**Risk difference (RD; -1 to +1)**
- Risk in the experimental group minus risk in the control group
**Confidence interval (CI)**
- The level of uncertainty in the estimate of treatment effect
- An estimate of the range in which the estimate would fall a fixed percentage of times if the study repeated many times
## Odds ratio (OR) will always be further from the point of no effect than a risk ratio (RR) ## Odds ratio (OR) will always be further from the point of no effect than a risk ratio (RR) ## If event rate in the treatment group ## If event rate in the treatment group
**When the event is rare** **When the event is rare**
- OR and RR will be similar
**When the event is common**
**1. metan** **1. metan**
**2. labbe**
**3. metacum**
**4. metap**
**5. metareg**
**6. metafunnel**
**7. confunnel**
**8. metabias**
**9. metatrim**
**Relative Risk (Fixed and Random effect model)** **Relative Risk (Fixed and Random effect model)**
**Fixedi= Fixed effect RR with inverse variance method**
**Fixed= M-H RR method**
## metan evtrt non_evtrt evctrl non_evctrl, rr fixed second(random) ## favours(reduces pregnancy rate # increases pregnancy rate) ## lcols(names outcome dose) by(status) sortby(outcome) force ## astext(70) textsize(200) boxsca(80) xsize(10) ysize(6) ## pointopt( msymbol(triangle) mcolor(gold) msize(tiny) ## mlabel() mlabsize(vsmall) mlabcolor(forest_green) mlabposition(1)) ## ciopt( lcolor(sienna) lwidth(medium)) rfdist rflevel(95) counts **Saving the graph in different formats**
## graph export "D:\Forest plot.gph", replace ## graph export "D:\Forest plot.gph".png", replace ## graph export "D:\Forest plot.gph".eps", replace
**Meta-analysis should only be considered when a group of trials is sufficiently homogeneous in terms of participations, interventions and outcomes to provide a meaningful summary** **Meta-analysis should only be considered when a group of trials is sufficiently homogeneous in terms of participations, interventions and outcomes to provide a meaningful summary**
## Examination for “**heterogeneity**” involves determination of whether individual differences between study outcomes are greater than could be expected by chance alone. ## Examination for “**heterogeneity**” involves determination of whether individual differences between study outcomes are greater than could be expected by chance alone. ## Analysis of “**heterogeneity**” is the most important function of MA, often more important than computing an “average” effect.
**By different investigators** **By different investigators**
**In different settings**
**In different countries**
**In different ways**
**To look at different outcomes**
**Etc.**
**Clinical diversity**: Variability in the participants, interventions and outcomes studied **Clinical diversity**: Variability in the participants, interventions and outcomes studied
**Methodological diversity**: Variability in the trial design and quality
**Statistical heterogeneity**: Variability in the treatment effects being evaluated in the different trials. This is a consequence of clinical and/or methodological diversity among the studies
**Study location and setting** **Study location and setting**
**Age, sex, diagnosis and disease severity of cases**
**Timing of the treatments**
**Dose and density of the intervention**
**Definition of the outcomes**
## Conventional chi-square (χ2) analysis (P>0.10) ## Conventional chi-square (χ2) analysis (P>0.10) **I****2= [(Q-df)/Q x 100% **(**Higgins et al. 2003**), where
**Graphical** test-forest plots (OR or RR and confidence intervals)
**L’Abbe **plots (outcome rates in treatment and control groups are plotted on the vertical and horizontal axes)
**Galbraith plot **
## Regression analysis ## Comparing the results of fixed and random effect models (a crude assessment of heterogeneity)
## Check again that the data are correct ## Check again that the data are correct ## Do not do a meta-analysis ## Ignore heterogeneity (fixed effect model) ## Perform a random effects meta-analysis ## Change the effect measure (e.g. different scale or units) ## Split studies into subgroups ## Investigate heterogeneity using meta-regression ## Exclude studies
**A process for re-analysing the same data set** **A process for re-analysing the same data set**
**A range of principles used, depends on**
- Choice of statistical test
- Inclusion criteria
- Inclusion of both published and unpublished
**To investigate whether heterogeneity among results of multiple studies is related to specific characteristics of the studies (e.g. dose rate)** **To investigate whether heterogeneity among results of multiple studies is related to specific characteristics of the studies (e.g. dose rate)**
**To investigate whether particular covariate (potential ‘effect modifier’) explain any of the heterogeneity of treatment effect between studies**
## **It is appropriate to use meta-regression to explore sources of heterogeneity even if an initial overall test for heterogeneity is non-significant**
## **metareg _ES bcalving acalving full_lact monen_other bstcode apcode, wsse(_seES) bsest(reml)** ## **metareg _ES bcalving acalving full_lact monen_other bstcode apcode, wsse(_seES) bsest(reml)** ## Meta-regression Number of obs = 23 ## REML estimate of between-study variance tau2 = .04357 ## % residual variation due to heterogeneity I-squared_res = 65.24% ## Proportion of between-study variance explained Adj R-squared = 51.05% ## Joint test for all covariates Model F(6,16) = 3.50 ## With Knapp-Hartung modification Prob > F = 0.0209 ## ------------------------------------------------------------------------------------------------------------- ## _ES | Coef. Std. Err. t P>|t| [95% Conf. Interval] ## -------------+----------------------------------------------------------------------------------------------- ## bcalving | -.0028578 .0031445 -0.91 0.377 -.0095239 .0038083 ## acalving | -.0007429 .0013228 -0.56 0.582 -.0035472 .0020613 ## full_lact | .3517979 .2544234 1.38 0.186 -.1875556 .8911513 ## other s| .3403943 .1278838 2.66 0.017 .0692928 .6114959 ## bstcode | -.0333014 .1370641 -0.24 0.811 -.3238642 .2572614 ## apcode | .4589506 .1391693 3.30 0.005 .1639249 .7539762 ## _cons | -.3564435 .2521411 -1.41 0.177 -.8909586 .1780717 ## -----------------------------------------------------------------------------------------------------------
## **metareg _ES full_lact monen_other apcode, wsse(_seES) bsest(reml)** ## **metareg _ES full_lact monen_other apcode, wsse(_seES) bsest(reml)** ## Meta-regression Number of obs = 23 ## REML estimate of between-study variance tau2 = .04134 ## % residual variation due to heterogeneity I-squared_res = 66.02% ## Proportion of between-study variance explained Adj R-squared = 53.55% ## Joint test for all covariates Model F(3,19) = 6.63 ## With Knapp-Hartung modification Prob > F = 0.0030 ## --------------------------------------------------------------------------------------------------------- ## _ES | Coef. Std. Err. t P>|t| [95% Conf. Interval] ## -------------+------------------------------------------------------------------------------------------- ## full_lact | .3138975 .117573 2.67 0.015 .0678144 .5599807 ## others | .3640601 .124601 2.92 0.009 .1032672 .6248529 ## apcode | .4385834 .1253647 3.50 0.002 .1761921 .700974 ## _cons | -.4478162 .1598295 -2.80 0.011 -.7823432 -.1132892 ## -------------------------------------------------------------------------------------------------------
**+ve results more likely** **+ve results more likely**
- To be published (publication bias)
- To be published rapidly (time lag bias)
- To be published in English (language bias)
- To be published more than once (multiple publications bias)
- To be cited by others (citation bias)
## Bias arising from the studies included in the review ## Bias arising from the studies included in the review ## Bias arising from the way the review is done ## Publication bias is only one of the possible reasons for asymmetrical funnel plot **Funnel plot** should been seen as a means of examining “**small study effect**”
**Funnel plot** **Funnel plot**
- Publication bias exists (asymmetrical)
- Publication bias doesn’t exists (symmetrical)
- For continuous data- Effect size plotted
*vs* SE or sample size - For dichotomous data- LogOR or RR
*vs* logSE or sample size
**Fail Safe Number (F)**
- Z= (∑ ES/1.645)2-N: (where N= no of papers; ∑ ES is summed of effect size over all studies)-
- for calculation of unpublished studies that would be required to negate the results of a significantly positive ES analysis.
*Cochrane group suggests that *that tests for funnel plot asymmetry should be used in only a minority of meta-analyses (Ioannidis 2007) *Cochrane group suggests that *that tests for funnel plot asymmetry should be used in only a minority of meta-analyses (Ioannidis 2007)
**Begg’s rank correlation test** (adjusted rank correlation-low power)
- This test is
**NOT** recommended with any type of data
**Eggers linear regression test** (regression analysis-low power)
- This test is mainly recommended for
**continuous** data
**Peters **(2006) & **Harbord **(2006) **tests** **Peters **(2006) & **Harbord **(2006) **tests**
## These tests are suitable for dichotomous data with odds ratios ## False-positive results may occur in the presence of substantial between-study heterogeneity **For dichotomous outcomes with risk ratios (RR) or risk differences (RD)**
## Firm guidance is not yet available
## Trim and fill method (tail of the side of the funnel plot with smaller trials chopped off) ## Trim and fill method (tail of the side of the funnel plot with smaller trials chopped off) ## Fail safe N (required studies to overturn positive results) ## Modelling for the probability of studies not published **Conclusion**: there is no definite answer for assessing the presence of publication bias
## www.stata.com/support/faqs/stat/meta.html ## www.stata.com/support/faqs/stat/meta.html ## Cochrane Collaboration Open learning material for reviewers (2002) ## Higgins et al. (2001). BMJ 327: 557-560 ## Sterne et al. (2001). BMJ 323: 101-105 ## Whitehead A (2002). Meta-analysis of Controlled Clinical Trials
**Do'stlaringiz bilan baham:** |