Preprint reviews by David Angeles-Albores
Casey W Dunn, Felipe Zapata, Catriona Munro, Stefan Siebert, Andreas Hejnol
Review posted on 12th February 2017
I would like to thank the authors for an extremely clear and interesting exposition on the benefits of phylogenies versus pairwise comparisons. The thoroughness of the analysis and the clarity of the writing make this paper a joy to read.
My only question regards the Levin reanalyses. Although the authors stated (correctly) that the KS test is sensitive to the shapes of distributions, they then go on to suggest that a Wilcoxon test should be used instead. However, my understanding was that the Wilcoxon test is a test that is also sensitive to the shape of distributions. I thought the strongest null hypothesis that can be tested using a Wilcoxon statistic was that 'the two samples come from identical distributions'. I thought that, given that the two distributions are identical, and one is shifted away from the other, only then the Wilcoxon test indicates whether the median is statistically different between the two distributions. Is this incorrect? If not, then it is necessary to perform an additional test before the Wilcoxon statistic to ensure that the two distributions are in fact the same, up to a shift.
It appears to me that the only method that is insensitive to the shape of a distribution is to perform a bootstrapped permutation test. Using a non-parametric bootstrap, we can test differences in the mean or the median, regardless of whether the samples are normally distributed or not.