Completed on 10 Mar 2016 by NIH/NHGRI preprint journal club and John Didion. Sourced from http://biorxiv.org/content/early/2016/01/28/038174.
Login to endorse this review.
Author response is in blue.
We discussed this paper in our preprint journal club on 3/3/16. Our comments:
We appreciate that the authors corrected for demographic covariates and batch effects. However, there are additional potential confounding factors for which we are not confident that the authors have properly corrected:
• RNA quality: samples from individuals with heart failure are subject to hypoxic stress and increased apoptosis, and thus may display substantially different expression profiles from healthy tissue due to non-biological causes (or at least not the biological causes you are interested in).
• Medications: those with failing hearts are more likely to be on medications that may alter expression profiles.
• Cell type composition: failing hearts are likely to have high infiltration of immune cells, which would change the make-up of the tissue you are profiling, and thus the expression profile.
We urge you to report RIN scores and summary phenotypes for your case and control samples. Additionally, RNA degradation is correlated with duration ex vivo, so it would be nice to see data showing whether there were any differences in surgical conditions, sample handling, etc. between the healthy and failing hearts. There are various strategies for estimating cell type composition, and/or for estimating the composition by computational deconvolution (e.g. DeconRNASeq).
It was also not clear as to the criteria for selecting case and control samples. Were control samples rejected for transplantation, or was the tissue sample taken from explanted hearts during the transplantation procedure? If the controls were rejected, what were the reasons, and might they constitute additional confounding factors? It would be nice to see additional detail on the methods of RNA isolation (including how much tissue was used and how much RNA was isolated).
A general comment about the figures is that font sizes should be increased for readability. Figure 2 a,b are not intuitive, and it is unclear what additional information they convey beyond figure 1c; we already know that the healthy network is more interconnected, so it seems obvious that there should be more trans effects. If you want to be able to claim that trans effects are more significant for disease versus healthy hearts, you need to normalize by network size/connectedness.
Figure 3 is the strong point of the paper, and we found it to be very effective in conveying the points you seem to be trying to make with the paper. We also find the method to be generally useful, and we urge you to release the source code used to perform this analysis as supplementary material. However, we are confused as to your end goal. Are you trying to create a community resource and starting point for investigating genes underlying heart failure? If so, then you should publish the entire list of genes that meets some significance threshold, not just the top 10.
In figure 4 b,f,g, it is unclear what the scales mean. How are we supposed to interpret normalized expression difference of 0.850 versus 0.875? Is that a big difference? It would be nice to know the magnitude of difference that constitutes significance. The legend for figure 4 is difficult to follow, and we don’t see where you even discuss panels F and G.
Thank you and your journal club members for discussing our paper and giving us careful suggestions! We really appreciate both the kind words and the constructive feedback. We have already taken them into consideration for the next draft of the manuscript. Here’s some commentary on that.
Regarding covariate correction, we did not correct for RNA quality since the sample RINs did not co-vary with case/control status. In addition, when building the networks for the individual cohorts we did not find evidence for differential expression of processes/networks relating to RNA degradation (e.g. apoptosis). The RIN values were consistent with what’s expected for human tissue collection with a mean of 8.1. At your suggestion, we are adding a table with RIN values into the supplement.
Addressing medication use is another good question but more challenging. Typically, most heart failure patients enter a final ‘common pathway’ that involves admission to the intensive care unit and support with adrenergic agonists pre-transplant. Whereas, heart donors are by definition without heart failure or a prolonged course. Their hearts function normally or near normally without the need for any such support. Thus, this differential is an unavoidable aspect of the collection of human tissue (the benefits of which we believe outweigh the limitations). We will add a short section to the manuscript to explain this rationale.
Estimating cell type composition and in particular immune cell infiltration is a fascinating idea. At your suggestion, we are currently testing in silico decomposition algorithms using various immune signatures. While interesting, it is unclear whether any such signal should be classified as a covariate to be controlled for or would better be considered a signal of interest. In addition, there might be a different inflammatory response in the unused donors that obfuscates any effect in the failing hearts.
Regarding criteria for selecting non-failing controls, control samples were healthy hearts obtained from organ donors that were not transplanted due to logistical reasons (e.g. a snowstorm). We are updating the manuscript to include this information as well as more information on the samples in the supplement.
Regarding the figures, thanks for the comments! We are reformatting and in some cases remaking them (e.g. Figure 2) to make them more intuitive following your feedback. What we find interesting, and what we tried to convey in Figure 2, is that the failing network has a wider variety of ‘network QTLs’ despite it being less connected. We have indeed corrected for network size and connectedness and used a permutation test to test this relationship. This was briefly mentioned in the manuscript -- we will look at rewording this section to highlight it more.
As you mention, Figure 3 is critical to explaining the novelty of our approach and at your suggestion we are rewording portions of the manuscript to increase focus on the global/local connectivity strategy for gene prioritization. All in all, our paper is a community resource, and we will be making available our results and networks, as well as updating our public codebase, before we submit the next draft.
Thanks so much again for your comments which have already improved the paper!