The use of weighted averages of Hedges' d in meta-analysis: Is it worth it?

Hedges' d is a measure of effect size widely used to standardize results across different studies in ecological and evolutionary meta-analyses. When summarizing Hedges' d or other effect sizes across multiple studies, it is considered a best practice to use weighted averages, with weights inversely proportional to the variances of the estimated effect sizes. Importantly, the within-study variance of Hedges' (Formula presented.), (Formula presented.), is a function of sample size, (Formula presented.), and (Formula presented.), the effect size itself. Since true effect sizes are unknown, (Formula presented.) is also unknown and needs to be estimated, for which numerous approaches have been proposed. We examined the behaviour of inverse-variance weights, specifically the performance of 14 different weighted and unweighted variants of the Hedges' (Formula presented.) estimator, to test the conditions under which weighting improved performance. We found that when sample sizes (per group) were >10, there was little difference between the estimators and weighting did not notably improve performance; bias was <5% and differences in root mean squared error were <10% for all estimators, except when the between-study variance for effect size, (Formula presented.), was relatively small compared to the within-study variance ((Formula presented.)). We also propose a new estimator that was found to be (a) the most efficient of a subset of minimally biased estimators and (b) had the least bias when estimating τ². Contrary to current guidance, we contend that the simpler unweighted Hedges' d estimator is generally acceptable. Exceptions are when the between-study variance is small relative to within-study variance ((Formula presented.)) and there is a large disparity (>3X) in sample sizes for a large proportion of effect sizes in the meta-analysis. Using unweighted averages may enable more robust inferences by allowing studies for which variance cannot be extracted (a common practical limitation) to be included. That said, weighted averaging is necessary where estimation of (Formula presented.) is itself of interest.