Completed on 10 Feb 2016 by Guillaume Rousselet . Sourced from http://biorxiv.org/content/early/2016/02/08/039065.
Login to endorse this review.
Author response is in blue.
The results are very interesting but difficult to assess in their current format. The presentation of the results could be strengthen by using scatterplots instead of bar graphs. In particular, when comparing a patient group to a control group, it is essential to be able to estimate group overlap. Also, for repeated-measure effects, scatterplots of the pairwise differences would help assess effect sizes and inter-participant differences.
The confidence intervals are incorrect, spanning undefined values of percent correct inferior to 0 and larger than 100. That’s probably because the confidence intervals were derived from a t-test formula, which assumes unbounded variables, and it is not the case for percent correct data.
The SD rule to remove outliers is not robust. A simple alternative is to use the median as a measure of central tendency for reaction times. The mean is anyway inappropriate for skewed distributions.
These references could help improve the presentation of the results:
Allen, E.A., Erhardt, E.B. & Calhoun, V.D. (2012) Data visualization in the neurosciences: overcoming the curse of dimensionality. Neuron, 74, 603-608.
Weissgerber, T.L., Milic, N.M., Winham, S.J. & Garovic, V.D. (2015) Beyond bar and line graphs: time for a new data presentation paradigm. PLoS Biol, 13, e1002128.
Wilcox RR, Keselman HJ. 2003. Modern Robust Data Analysis Methods: Measures of Central Tendency. Psychological Methods 8: 254-74
Thank you for your comments on our manuscript, and particularly for
pointing out better ways to visualize the results. This will be very
helpful when revising the manuscript.
Regarding the SD rule to
remove outliers, I agree that using the median RT may be a good
alternative. However, as we are reporting vocal reaction times measured
by a voice key (microphone) in the current study, we found it better to
trim the data using a +/- 2,5 SD criterion, so that responses where the
voice key was accidently set off (e.g., by breathing in), or not
triggered (speaking to quitely), would be removed, without the need to
set an arbitrary time-criterion for such responses.